Skip to content

'sort_index"does not work with 'pd.set_option('mode.use_inf_as_na',True)' #29687

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
asmcat666 opened this issue Nov 18, 2019 · 7 comments · Fixed by #41712
Closed

'sort_index"does not work with 'pd.set_option('mode.use_inf_as_na',True)' #29687

asmcat666 opened this issue Nov 18, 2019 · 7 comments · Fixed by #41712
Labels
good first issue Needs Tests Unit test(s) needed to prevent regressions
Milestone

Comments

@asmcat666
Copy link

asmcat666 commented Nov 18, 2019

In [12]: pd.set_option('mode.use_inf_as_na',False)

In [13]: df
Out[13]:
                            B
2018-09-25 14:47:31.500   0.0
2018-04-03 22:55:13.000   1.0
2018-09-04 09:10:07.500   2.0
2018-11-30 10:32:08.000   NaN
2018-03-27 14:21:12.500   4.0
2018-09-28 14:30:36.500   6.0
2018-07-16 21:56:48.000   7.0
2018-12-05 23:53:59.500   8.0
2018-09-04 09:46:58.500   9.0
2018-11-09 09:14:08.500  10.0

In [14]: df.sort_index()
Out[14]:
                            B
2018-03-27 14:21:12.500   4.0
2018-04-03 22:55:13.000   1.0
2018-07-16 21:56:48.000   7.0
2018-09-04 09:10:07.500   2.0
2018-09-04 09:46:58.500   9.0
2018-09-25 14:47:31.500   0.0
2018-09-28 14:30:36.500   6.0
2018-11-09 09:14:08.500  10.0
2018-11-30 10:32:08.000   NaN
2018-12-05 23:53:59.500   8.0

In [15]: pd.set_option('mode.use_inf_as_na',True)

In [16]: df.sort_index()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-16-02a473b4132d> in <module>
----> 1 df.sort_index()

g:\py372\lib\site-packages\pandas\core\frame.py in sort_index(self, axis, lev
 ascending, inplace, kind, na_position, sort_remaining, by)
   5088
   5089             indexer = nargsort(
-> 5090                 labels, kind=kind, ascending=ascending, na_position=n
osition
   5091             )
   5092

g:\py372\lib\site-packages\pandas\core\sorting.py in nargsort(items, kind, as
ding, na_position)
    244
    245     items = extract_array(items)
--> 246     mask = np.asarray(isna(items))
    247
    248     if is_extension_array_dtype(items):

g:\py372\lib\site-packages\pandas\core\dtypes\missing.py in isna(obj)
    120     Name: 1, dtype: bool
    121     """
--> 122     return _isna(obj)
    123
    124

g:\py372\lib\site-packages\pandas\core\dtypes\missing.py in _isna_old(obj)
    177         return _isna_ndarraylike_old(obj)
    178     elif isinstance(obj, ABCGeneric):
--> 179         return obj._constructor(obj._data.isna(func=_isna_old))
    180     elif isinstance(obj, list):
    181         return _isna_ndarraylike_old(np.asarray(obj, dtype=object))

AttributeError: 'DatetimeArray' object has no attribute '_constructor'

@TomAugspurger
Copy link
Contributor

Can you provide a reproducible example, including the code to create the dataframe?: http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports

@simonjayhawkins
Copy link
Member

closing as can't reproduce

@annhak
Copy link

annhak commented Nov 13, 2020

Bumped into the same problem @simonjayhawkins
Minimal reproducible example:

import pandas as pd

d = {'col1': [1, 2, 3], 'col2': [3, 4, 5]}
df = pd.DataFrame(data=d)
now = pd.Timestamp.now()
df.index = [now, now - pd.Timedelta('1m'), now + pd.Timedelta('2m')]

try:
df1 = df.sort_index()
print('test1, success')
except Exception as ex:
print('test 1 failed: ', ex)

pd.set_option('use_inf_as_na', True)

try:
df2 = df.sort_index()
print('test2, success')
except Exception as ex:
print('test 2 failed: ', ex)

Produces:
test1, success
test 2 failed: 'DatetimeArray' object has no attribute '_constructor'

@simonjayhawkins
Copy link
Member

Thanks @annhak . You code sample raises on 0.25.3, but appears to be working on 1.0.5 and master.

will reopen for now with needs tests label but will also add closing candidate if it can be determined which commit fixed this and whether appropriate tests were added.

@simonjayhawkins simonjayhawkins added Closing Candidate May be closeable, needs more eyeballs good first issue Needs Tests Unit test(s) needed to prevent regressions labels Nov 22, 2020
@simonjayhawkins simonjayhawkins added this to the Contributions Welcome milestone Nov 22, 2020
@ankushduacodes
Copy link
Contributor

Hi @simonjayhawkins, I would love to contribute to this issue, Please let me know if I may :)

@weikhor
Copy link
Contributor

weikhor commented Dec 17, 2020

Hi Guys I will start doing this.

@weikhor
Copy link
Contributor

weikhor commented Dec 17, 2020

@TomAugspurger @simonjayhawkins @annhak @asmcat666
Guy have created test cases for this issues.

@mroeschke mroeschke mentioned this issue May 29, 2021
15 tasks
@mroeschke mroeschke modified the milestones: Contributions Welcome, 1.3 May 29, 2021
@lithomas1 lithomas1 removed the Closing Candidate May be closeable, needs more eyeballs label May 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Needs Tests Unit test(s) needed to prevent regressions
Projects
None yet
8 participants