Pandas multiindex unstack fillna not working

[SeunghyeonTonyYu]

When we unstacking the multi-indexed pandas dataframe, the method fillna does not work.

Here is an example.

import numpy as np
import pandas as pd

df = pd.DataFrame(np.random.randint(0,10,(5,4)),
                 columns=['c1','c2','c3','c4'])
df.iloc[1,2] = np.nan
df.iloc[0,0] = np.nan
df['ind1'] = ['a','a','b','b','c']
df['ind2'] = [1,2,1,2,1]
df = df.set_index(['ind1','ind2'])
print(df)

Now we have df with some NaN values.

            c1  c2   c3  c4
ind1 ind2                  
a    1     NaN   2  1.0   9
     2     9.0   5  NaN   7
b    1     1.0   7  2.0   8
     2     3.0   6  0.0   2
c    1     9.0   6  6.0   6

Then fillna to the unstacked df does not work.

print(df.unstack().fillna(0))

       c1        c2        c3        c4     
ind2    1    2    1    2    1    2    1    2
ind1                                        
a     0.0  9.0  2.0  5.0  1.0  0.0  9.0  7.0
b     1.0  3.0  7.0  6.0  2.0  0.0  8.0  2.0
c     9.0  0.0  6.0  NaN  6.0  0.0  6.0  NaN

I raised this issue at stack overflow, and I suspect this is a bug.

I would appreciate to fix this problem.

Thanks,

Note
The pandas version is 1.1.3 (current version in Anaconda), and there is no issue with the version ==1.2.1.

[fujiaxiang]

hi @YuSeungHyun can you show us your pandas version by doing pd.__version__ inside python? I tried your code but I can't reproduce your result in pandas 1.2.3. The NaN values are filled by 0 on 1.2.3. I also tried it on latest master which doesn't seem to have this problem.

However I did manage to get this result in pandas 1.1.0. Did you accidentally "swap" your environment with different versions?

[fujiaxiang]

also see #37115 and #36495, possibly fixed by the corresponding PRs.

[SeunghyeonTonyYu]

Sorry, I have a mistake.

The issue version is pandas 1.1.3

[fujiaxiang]

Thanks for clarifying. I will close this for now. You can re-open if you find further issues.