Skip to content

Sorting within groupby causes duplication of index level #11802

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tdhopper opened this issue Dec 9, 2015 · 2 comments
Closed

Sorting within groupby causes duplication of index level #11802

tdhopper opened this issue Dec 9, 2015 · 2 comments
Labels
Apply Apply, Aggregate, Transform, Map Bug Groupby

Comments

@tdhopper
Copy link
Contributor

tdhopper commented Dec 9, 2015

When I run this code on 0.17.0 on Python 3.5.0

import pandas as pd
d = pd.DataFrame(data=[[2, 1]])
print(d.index)
print((d.
 groupby(level=0).
 apply(lambda d: 
       d.sort_values(0)
      )
 ).index)

I get

Int64Index([0], dtype='int64')
MultiIndex(levels=[[0], [0]],
           labels=[[0], [0]])

The index gets duplicated (and turned into a multiindex) as a result of the sort within the groupby.

Appears to be a bug to me.

@jreback
Copy link
Contributor

jreback commented Dec 9, 2015

this is the same issue as in #9946 (same because the same underlying cython routine is used)

you will notice that using lambda x: x.copy() is the same result.

pandas is trying to infer whether you have mutated things (or not), by comparing indexes. This is prob a bug, but this is a bit of a rabbit hole as I think the best thing to do would actually to ban this entirely (mutation). So the outputs can be more predictible. You are welcome to dig-in!.

@jreback jreback added Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode Difficulty Advanced labels Dec 9, 2015
@jreback jreback added this to the Next Major Release milestone Dec 9, 2015
@mroeschke mroeschke added Apply Apply, Aggregate, Transform, Map Groupby and removed Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Apr 21, 2021
@mroeschke mroeschke removed this from the Contributions Welcome milestone Oct 13, 2022
@rhshadrach
Copy link
Member

This can now be controlled by specifying group_keys=False in groupby.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Apply Apply, Aggregate, Transform, Map Bug Groupby
Projects
None yet
Development

No branches or pull requests

5 participants