Skip to content

BUG: groupby apply on single group transposes series output #45187

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 of 3 tasks
seanv507 opened this issue Jan 4, 2022 · 4 comments
Closed
2 of 3 tasks

BUG: groupby apply on single group transposes series output #45187

seanv507 opened this issue Jan 4, 2022 · 4 comments
Labels
Apply Apply, Aggregate, Transform, Map Bug Duplicate Report Duplicate issue or pull request Groupby

Comments

@seanv507
Copy link

seanv507 commented Jan 4, 2022

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the master branch of pandas.

Reproducible Example

import pandas as pd
df = pd.DataFrame({'A': 'a a'.split(),
                   'B': [1,2],
                   'C': [4,6]})
df.groupby("A").apply(lambda x: x["C"]>x["B"])

# output
#       0     1
# A            
# a  True  True
# dtype: bool
df2 = pd.DataFrame({'A': 'a a b b'.split(),
                    'B': [1,2, 3, 4],
                    'C': [5,6, 7, 8],
                    'D': [5,6, 7, 8]
}
)

df2.groupby("A").apply(lambda x: x["C"]>x["B"])
# output 
# A   
# a  0    True
#    1    True
# b  2    True
#    3    True
# dtype: bool

Issue Description

the apply function is returning a series. if a single group is output the series is treated as a row vector, if multiple groups it is returned as a column vector as expected.

Expected Behavior

df.groupby("A").apply(lambda x: x["C"]>x["B"])
# output 
# A   
# a  0    True
#     1    True

Installed Versions

python 3.10.1
pandas 1.3.5

show versions crashed (created by running conda create -n test_pandas pandas on Mac OSX)

@seanv507 seanv507 added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Jan 4, 2022
@takarzyna
Copy link

I have confirmed this bug exists on the master branch of pandas (I was going to report this issue when I noticed this report).
I would add that in the case with one group the result's type is a DataFrame which can cause issues on its own if a Series is expected.

My current workaround is applying squeeze(axis=0) to the result of apply in the case of one group.

@mroeschke mroeschke added Apply Apply, Aggregate, Transform, Map Groupby and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Jan 13, 2022
@rhshadrach
Copy link
Member

The issue is with all_indexed_same in pandas.core.groupby.generic._wrap_applied_output_series. We try to determine how the output should be stacked. If the results are all indexed the same it is usually because the operation is a reducer producing a Series where the index is the columns of the input frame. In such a case, we want to turn the Series into a row of the result rather than stacking it vertically. However, if the indexes in the results are not all the same, we stack them vertically (the desired behavior here).

Trouble is, when there is only one group, the indices of course are all the same. I'm not seeing a better way of determining how to wrap the result, but currently still searching.

@jsamoocha
Copy link

Adding the squeeze=True argument to groupby() seems like a viable workaround. However, that argument is deprecated since version 1.1??

@simonjayhawkins
Copy link
Member

closing as duplicate of #42749

@simonjayhawkins simonjayhawkins added the Duplicate Report Duplicate issue or pull request label Jun 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Apply Apply, Aggregate, Transform, Map Bug Duplicate Report Duplicate issue or pull request Groupby
Projects
None yet
Development

No branches or pull requests

6 participants