Skip to content

reindex(method="backfill") and periods within the last date #34

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tgefell opened this issue Jan 24, 2011 · 2 comments
Closed

reindex(method="backfill") and periods within the last date #34

tgefell opened this issue Jan 24, 2011 · 2 comments

Comments

@tgefell
Copy link

tgefell commented Jan 24, 2011

I have a DataFrame with 1 minute timestamps, pw.

I have a monthly average of the values:
ma_pw=pw.groupby(BMonthEnd(+1).rollforward).aggregate(np.mean)

When I do a reindex to push that average back into all of the periods for the month,
the last day's values are NaN. Here's my reindex attempt:

reindexed = ma_pw.reindex(pw.index, method="backfill").

I have worked around this by forcing the hour in my monthly data to something past the last minute that I care about in my by-minute data as follows, but this seems incredibly crude:

from dateutil.relativedelta import *
newi = [x + relativedelta(hour=20) for x in ma_pw.index]
ma_pw.index = newi[:]

Then my original reindex works.

Is there a clean way to instruct the reindex method to include the datetime objects having the same datetime.date on that last day of the month?

@wesm
Copy link
Member

wesm commented Jan 25, 2011

You are back-propagating values? hmm. There are two options: pad and backfill. Pad propagates forward, backfill the opposite.

You can do ma_pw.fill(method='pad') method-- please note that fill is going to be renamed to something else in the 0.4 release (which is coming after the yet-unreleased 0.3 release) but should be easy to fix your code ;)

@wesm
Copy link
Member

wesm commented Feb 20, 2011

Closing this issue, I don't think it's a bug

This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants