Skip to content

Panel.fillna with method='ffill' ignores the axis parameter and only fills along axis=1 #8251

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
0zeroth opened this issue Sep 12, 2014 · 4 comments
Labels
Bug Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate

Comments

@0zeroth
Copy link

0zeroth commented Sep 12, 2014

This is also a documentation error, as the docs say the default axis is axis = 0

import pandas as pd
from pandas.util.testing import makePanel

#
# Demo A: We want to fill along axis 0
# 
p = makePanel().transpose(1,0,2)
p.iloc[10:20,0,0] = np.nan

# Demonstrate we have a gap
p.iloc[:,0,0].plot()

# Demonstrate that nothing seems to fill the gap
p.fillna(method='ffill').iloc[:,0,0].plot()
p.fillna(method='ffill', axis=0).iloc[:,0,0].plot()
p.fillna(method='ffill', axis=1).iloc[:,0,0].plot()
p.fillna(method='ffill', axis=2).iloc[:,0,0].plot()

# At least one of the above should be the same as:
p.apply(lambda series: series.fillna(method='ffill'), axis=0).iloc[:,0,0].plot()

#
# Demo B: We want to fill along axis 1
#
p = makePanel()
p.iloc[0, 10:20,0] = np.nan

# Demonstrate we have a gap
p.iloc[0,:,0].plot()

# Demonstrate the gap is filled regardless of the axis
p.fillna(method='ffill').iloc[0,:,0].plot()
p.fillna(method='ffill', axis=0).iloc[0,:,0].plot()
p.fillna(method='ffill', axis=1).iloc[0,:,0].plot()
p.fillna(method='ffill', axis=2).iloc[0,:,0].plot()
@jreback jreback added Bug Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate labels Sep 12, 2014
@jreback jreback added this to the 0.15.1 milestone Sep 12, 2014
@jreback
Copy link
Contributor

jreback commented Sep 12, 2014

looks like a bug. Panel.fillna not tested very well. I can see that the axis does not appear to be passed thru correctly.

pull-request welcome. Would like to have comprehensive tests for this (don't use plot though), construct a test panel and its expected output and check for equality.

@staple
Copy link
Contributor

staple commented Sep 27, 2014

Hi I started working on this yesterday as well, looks like there was a collision. @stahlous, feel free to pull in anything from my patch (including test cases). I'm going to close my PR now in deference to @stahlous who submitted the first PR.
#8401

@jreback jreback modified the milestones: 0.16.0, Next Major Release Mar 6, 2015
@thusson
Copy link

thusson commented Mar 24, 2015

Note too that the limit parameter is also ignored. Change the last few lines of the above example to:

p.fillna(method='ffill', axis=0, limit=5).iloc[0,:,0].plot()
p.fillna(method='ffill', axis=1, limit=5).iloc[0,:,0].plot()
p.fillna(method='ffill', axis=2, limit=5).iloc[0,:,0].plot()

And you can see it fills beyond 5 items.

@jreback
Copy link
Contributor

jreback commented Jul 11, 2017

closing as Panel deprecated

@jreback jreback closed this as completed Jul 11, 2017
@TomAugspurger TomAugspurger modified the milestones: won't fix, No action Jul 6, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate
Projects
None yet
6 participants