Skip to content

Make interpolate_2d handle datetime64 correctly #27628

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 31, 2019

Conversation

jbrockmendel
Copy link
Member

Broken off from #27626 because I decided that is poorly scoped.

@gfyoung gfyoung added Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Datetime Datetime data dtype labels Jul 28, 2019
Copy link
Member

@gfyoung gfyoung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix looks relatively straightforward, just need tests and whatsnew

@jbrockmendel
Copy link
Member Author

Test added, unnecessary piece removed. No release note needed since no user-facing changes.

Copy link
Contributor

@TomAugspurger TomAugspurger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall. I noticed the following while testing locally

In [1]: import pandas as pd

In [2]: dti = pd.date_range("2015-04-05", periods=5, tz="US/Central")
   ...: ser = pd.Series(dti)
   ...: ser.iloc[1:4] = pd.NaT
   ...: ser.astype('datetime64[ns]').to_frame().interpolate(method="pad", limit=2)
Out[2]:
                    0
0 2015-04-05 05:00:00
1                 NaT
2                 NaT
3                 NaT
4 2015-04-09 05:00:00

That happens on both master and this PR. Do you know if we have an issue for it? The important bits are

  1. datetime64[ns]-dtype. datetime with TZ seems fine
  2. DataFrame.interpolate. Series[datetime64[ns]].interpolate seems fine

@jbrockmendel
Copy link
Member Author

datetime64[ns]-dtype. datetime with TZ seems fine
DataFrame.interpolate. Series[datetime64[ns]].interpolate seems fine

I'm not sure I understand your example. Is it relevant that you passed tz to date_range? You subsequently astype-d it away.

Best guess: DatetimeTZBlock.fillna dispatches to DatetimeArray.fillna, so doesn't go directly through here.

@jbrockmendel
Copy link
Member Author

@jreback gentle ping. Between this and the quantile PR, we're about to be about to be able to rip out a bunch of _try_coerce_result code

Copy link
Contributor

@TomAugspurger TomAugspurger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts @jreback?

@jreback jreback added this to the 1.0 milestone Jul 31, 2019
@jreback jreback merged commit c7a1321 into pandas-dev:master Jul 31, 2019
@jreback
Copy link
Contributor

jreback commented Jul 31, 2019

thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Datetime Datetime data dtype
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants