Skip to content

BUG: resample with datetime64 data on series looses dtype #13119

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jorisvandenbossche opened this issue May 9, 2016 · 2 comments · Fixed by #14117 or #14118
Closed

BUG: resample with datetime64 data on series looses dtype #13119

jorisvandenbossche opened this issue May 9, 2016 · 2 comments · Fixed by #14117 or #14118
Labels
Bug Resample resample method Testing pandas testing functions or related to the test suite
Milestone

Comments

@jorisvandenbossche
Copy link
Member

From SO: http://stackoverflow.com/questions/37106810/bug-in-python-pandas-pandas-series-resample-vs-pandas-dataframe-resample

In [7]: import datetime as dt

In [8]: df = pd.DataFrame([[dt.datetime(2016,1,31,15)], [dt.datetime(2016,2,7,17)]], index=[dt.datetime(2016,1,31,15), dt.datetime(2016,2,7,17)], columns=['timestamp'])

In [10]: df
Out[10]:
                              timestamp
2016-01-31 15:00:00 2016-01-31 15:00:00
2016-02-07 17:00:00 2016-02-07 17:00:00

In [11]: df.resample('D').first().timestamp
Out[11]:
2016-01-31   2016-01-31 15:00:00
2016-02-01                   NaT
2016-02-02                   NaT
2016-02-03                   NaT
2016-02-04                   NaT
2016-02-05                   NaT
2016-02-06                   NaT
2016-02-07   2016-02-07 17:00:00
Freq: D, Name: timestamp, dtype: datetime64[ns]

In [12]: df.timestamp.resample('D').first()
Out[12]:
2016-01-31    1.454252e+18
2016-02-01             NaN
2016-02-02             NaN
2016-02-03             NaN
2016-02-04             NaN
2016-02-05             NaN
2016-02-06             NaN
2016-02-07    1.454864e+18
Freq: D, Name: timestamp, dtype: float64

Tested it with latest master

@jorisvandenbossche jorisvandenbossche added Resample resample method Bug labels May 9, 2016
@jorisvandenbossche jorisvandenbossche added this to the 0.18.2 milestone May 9, 2016
@jreback
Copy link
Contributor

jreback commented May 9, 2016

related to #3087 (though that's pretty old).

@jorisvandenbossche jorisvandenbossche added Difficulty Novice Testing pandas testing functions or related to the test suite and removed Difficulty Intermediate labels Aug 21, 2016
@wcwagner
Copy link
Contributor

wcwagner commented Aug 23, 2016

This is still broken with Timedeltas
Gonna add some tests, and try real quick for a fix.
(gonna ignore the incorrect bins for now?)
repro:

In [21]: df = DataFrame({'time':timedelta_range('1 day', '4 day', freq='D')},
                index=timedelta_range('1 day', '4 day', freq='D'))

In [22]: df
Out[22]: 
         time
1 days 1 days
2 days 2 days
3 days 3 days
4 days 4 days

In [23]: df.resample('2D').first()
Out[23]: 
         time
1 days 1 days
3 days 3 days
5 days    NaT

In [24]: df.time.resample('2D').first()
Out[24]: 
1 days    8.640000e+13
3 days    2.592000e+14
5 days             NaN
Freq: 2D, Name: time, dtype: float64

output of pd.show_versions

INSTALLED VERSIONS
------------------
commit: 6645b2b11a82343e5f07b15a25a250f411067819
python: 3.5.2.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-21-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.18.1+391.g6645b2b.dirty
nose: 1.3.7
pip: 8.1.2
setuptools: 23.0.0
Cython: 0.24
numpy: 1.11.1
scipy: 0.17.1
statsmodels: 0.6.1
xarray: None
IPython: 4.2.0
sphinx: 1.4.1
patsy: 0.4.1
dateutil: 2.5.3
pytz: 2016.4
blosc: None
bottleneck: 1.1.0
tables: 3.2.2
numexpr: 2.6.0
matplotlib: 1.5.1
openpyxl: 2.3.2
xlrd: 1.0.0
xlwt: 1.1.2
xlsxwriter: 0.9.2
lxml: 3.6.0
bs4: 4.4.1
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.13
pymysql: None
psycopg2: None
jinja2: 2.8
boto: 2.40.0
pandas_datareader: None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Resample resample method Testing pandas testing functions or related to the test suite
Projects
None yet
3 participants