Skip to content

sort_index with sort_remaining=False does not work with int level #21052

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kdebrab opened this issue May 15, 2018 · 5 comments · Fixed by #21043
Closed

sort_index with sort_remaining=False does not work with int level #21052

kdebrab opened this issue May 15, 2018 · 5 comments · Fixed by #21043
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Milestone

Comments

@kdebrab
Copy link
Contributor

kdebrab commented May 15, 2018

Code Sample, a copy-pastable example if possible

import pandas as pd
mi = pd.MultiIndex.from_tuples([[2,2],[2,1],[1,2],[1,1]], names=list('AB'))
a = pd.Series([1,2,3,4], index=mi)
a.sort_index(level=0, sort_remaining=False)

returns:

A  B
1  1    4
   2    3
2  1    2
   2    1
dtype: int64

Problem description

Besides A (level 0), also B (level 1) is sorted. This, despite the fact that sort_remaining is set to False.

Expected Output

By comparison, the correct result is obtained when specifying the level by its name:

a.sort_index(level='A', sort_remaining=False)

with

A  B
1  2    3
   1    4
2  2    1
   1    2
dtype: int64

Exactly the same result should be expected when using level=0 instead of level='A'.

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.13.final.0
python-bits: 32
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 78 Stepping 3, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None
pandas: 0.22.0
pytest: None
pip: 9.0.3
setuptools: 36.4.0
Cython: None
numpy: 1.14.1
scipy: 1.0.0
pyarrow: None
xarray: 0.10.0
IPython: 5.3.0
sphinx: 1.6.3
patsy: 0.5.0
dateutil: 2.6.1
pytz: 2018.3
blosc: None
bottleneck: 1.2.1
tables: None
numexpr: None
feather: None
matplotlib: 2.1.2
openpyxl: 2.5.0
xlrd: 1.0.0
xlwt: 1.3.0
xlsxwriter: 1.0.2
lxml: 4.1.1
bs4: 4.6.0
html5lib: 0.999999999
sqlalchemy: 1.2.3
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@WillAyd
Copy link
Member

WillAyd commented May 15, 2018

Thanks for the bug. This is probably related to #21043, though the work done there does not appear to solve it.

Interestingly enough, the difference in sorting is ONLY observed at the first level. If your items were say at the second level there would be no difference.

In [2]: mi = pd.MultiIndex.from_tuples([[0,2,2],[0,2,1],[0,1,2],[0,1,1]], names=list('CAB'))
In [3]: a = pd.Series([1,2,3,4], index=mi)

In [5]: a.sort_index(level=1, sort_remaining=False)
Out[5]: 
C  A  B
0  1  2    3
      1    4
   2  2    1
      1    2
dtype: int64

In [6]: a.sort_index(level='A', sort_remaining=False)
Out[6]: 
C  A  B
0  1  2    3
      1    4
   2  2    1
      1    2
dtype: int64

@kdebrab
Copy link
Contributor Author

kdebrab commented May 15, 2018

Thus, the problem occurs only when level=0. Hmm, that sounds familiar. Indeed, a simple a.sort_index?? already reveals the culprit:

if level:
   [..]

In other words, level=0 is handled in the same way as level=None, which means that all levels are sorted...

The solution seems simple:

if level is not None:
   [..]

@WillAyd
Copy link
Member

WillAyd commented May 15, 2018

Perhaps - that's the same issue I found in the referenced PR #21043 though trying on that branch didn't resolve your issue. May be a similar issue somewhere else - investigation and PRs welcome!

@kdebrab
Copy link
Contributor Author

kdebrab commented May 15, 2018

I should have checked your link. I think you solved it for DataFrames, but not yet for Series...

@kdebrab
Copy link
Contributor Author

kdebrab commented May 16, 2018

Thanks for the quick response and fix @WillAyd !

@jreback jreback added Bug Indexing Related to indexing on series/frames, not to indexes themselves Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels May 17, 2018
@jreback jreback added this to the 0.24.0 milestone May 17, 2018
@jorisvandenbossche jorisvandenbossche modified the milestones: 0.24.0, 0.23.1 Jun 5, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants