You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
(optional) I have confirmed this bug exists on the master branch of pandas.
Code Sample
In [1]: df=pd.DataFrame(dict(a=[nan], b=[0], c=[1]))
In [2]: dfabc0NaN01# groupby drops groups with null by defaultIn [3]: df.groupby(['a', 'b']).first()
EmptyDataFrameColumns: [c]
Index: []
# pass dropna=False to keep groups with nullsIn [4]: df.groupby(['a', 'b'], dropna=False).first()
cabNaN01# This doesn't work if the groups are in a multi index.In [5]: df.set_index(['a', 'b']).groupby(['a', 'b'], dropna=False).first()
EmptyDataFrameColumns: [c]
Index: []
Problem description
It seems like setting dropna=False in groupby doesn't work when the group columns are part of a multi index. I'm not sure if this is intentional, but it is certainly confusing.
Expected Output
I would expect the output of [5] above to match that of [4].
importnumpyasnpimportpandasaspddefmain():
df=pd.DataFrame(dict(a=[np.NaN], b=[0], c=[1]))
print('df', df, sep='\n')
print('\ngroupby() drops groups with null by default')
print(df.groupby(['a', 'b']).first())
print('\npass dropna=False to keep groups with nulls')
print(df.groupby(['a', 'b'], dropna=False).first())
print("\nThis doesn't work if the groups are in a multi index.")
print(df.set_index(['a', 'b']).groupby(['a', 'b'], dropna=False).first())
if__name__=='__main__':
main()
The output I'm seeing is as below, but I would expect that the last two outputs should be the same.
df
a b c
0 NaN 0 1
groupby drops groups with null by default
Empty DataFrame
Columns: [c]
Index: []
pass dropna=False to keep groups with nulls
c
a b
NaN 0 1
This doesn't work if the groups are in a multi index.
Empty DataFrame
Columns: [c]
Index: []
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
(optional) I have confirmed this bug exists on the master branch of pandas.
Code Sample
Problem description
It seems like setting
dropna=False
ingroupby
doesn't work when the group columns are part of a multi index. I'm not sure if this is intentional, but it is certainly confusing.Expected Output
I would expect the output of [5] above to match that of [4].
Output of
pd.show_versions()
The text was updated successfully, but these errors were encountered: