Skip to content

Adding TimedeltaIndex to tz-aware DatetimeIndex: TypeError: data type not understood #18947

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
acowlikeobject opened this issue Dec 26, 2017 · 2 comments
Labels
Datetime Datetime data dtype Duplicate Report Duplicate issue or pull request Timezones Timezone data dtype

Comments

@acowlikeobject
Copy link

Code Sample, a copy-pastable example if possible

import pandas as pd

dates = pd.date_range('2017-01-01', '2017-01-31', tz='UTC')
deltas = pd.TimedeltaIndex(pd.np.arange(0, len(dates)), unit='m')
dates + deltas

# Removing the tz='UTC' (making dates tz naive) works

throws:

TypeError                                 Traceback (most recent call last)
<ipython-input-28-708df3869f57> in <module>()
      3 dates = pd.date_range('2017-01-01', '2017-01-31', tz='UTC')
      4 deltas = pd.TimedeltaIndex(pd.np.arange(0, len(dates)), unit='m')
----> 5 dates + deltas

~/miniconda/lib/python3.6/site-packages/pandas/core/indexes/datetimelike.py in __add__(self, other)
    643             from pandas.tseries.offsets import DateOffset
    644             if isinstance(other, TimedeltaIndex):
--> 645                 return self._add_delta(other)
    646             elif isinstance(self, TimedeltaIndex) and isinstance(other, Index):
    647                 if hasattr(other, '_add_delta'):

~/miniconda/lib/python3.6/site-packages/pandas/core/indexes/datetimes.py in _add_delta(self, delta)
    814             new_values = self._add_delta_td(delta)
    815         elif isinstance(delta, TimedeltaIndex):
--> 816             new_values = self._add_delta_tdi(delta)
    817             # update name when delta is Index
    818             name = com._maybe_match_name(self, delta)

~/miniconda/lib/python3.6/site-packages/pandas/core/indexes/datetimelike.py in _add_delta_tdi(self, other)
    727             mask = (self._isnan) | (other._isnan)
    728             new_values[mask] = iNaT
--> 729         return new_values.view(self.dtype)
    730 
    731     def isin(self, values):

TypeError: data type not understood

Problem description

Adding TimedeltaIndex to timezone-aware DatetimeIndex throws a TypeError. The error goes away if the DatetimeIndex does not have a timezone.

(I vaguely remember seeing an issue on this, but can't seem to find it now when I search.)

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.2.final.0
python-bits: 64
OS: Linux
OS-release: 4.10.0-33-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: C.UTF-8
LANG: C.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.21.0
pytest: None
pip: 9.0.1
setuptools: 36.5.0.post20170921
Cython: None
numpy: 1.13.3
scipy: 0.19.1
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: None
patsy: None
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: None
tables: 3.4.2
numexpr: 2.6.4
feather: None
matplotlib: 2.1.0
openpyxl: None
xlrd: 1.0.0
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 0.9999999
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@jschendel
Copy link
Member

This should be fixed on master via #18653

In [1]: import pandas as pd
   ...:
   ...: dates = pd.date_range('2017-01-01', '2017-01-31', tz='UTC')
   ...: deltas = pd.TimedeltaIndex(pd.np.arange(0, len(dates)), unit='m')
   ...: dates + deltas
   ...:
Out[1]:
DatetimeIndex(['2017-01-01 00:00:00+00:00', '2017-01-02 00:01:00+00:00',
               '2017-01-03 00:02:00+00:00', '2017-01-04 00:03:00+00:00',
               '2017-01-05 00:04:00+00:00', '2017-01-06 00:05:00+00:00',
               '2017-01-07 00:06:00+00:00', '2017-01-08 00:07:00+00:00',
               '2017-01-09 00:08:00+00:00', '2017-01-10 00:09:00+00:00',
               '2017-01-11 00:10:00+00:00', '2017-01-12 00:11:00+00:00',
               '2017-01-13 00:12:00+00:00', '2017-01-14 00:13:00+00:00',
               '2017-01-15 00:14:00+00:00', '2017-01-16 00:15:00+00:00',
               '2017-01-17 00:16:00+00:00', '2017-01-18 00:17:00+00:00',
               '2017-01-19 00:18:00+00:00', '2017-01-20 00:19:00+00:00',
               '2017-01-21 00:20:00+00:00', '2017-01-22 00:21:00+00:00',
               '2017-01-23 00:22:00+00:00', '2017-01-24 00:23:00+00:00',
               '2017-01-25 00:24:00+00:00', '2017-01-26 00:25:00+00:00',
               '2017-01-27 00:26:00+00:00', '2017-01-28 00:27:00+00:00',
               '2017-01-29 00:28:00+00:00', '2017-01-30 00:29:00+00:00',
               '2017-01-31 00:30:00+00:00'],
              dtype='datetime64[ns, UTC]', freq='1441T')

In [2]: pd.__version__
Out[2]: '0.22.0.dev0+427.gee9c7e9'

@jreback
Copy link
Contributor

jreback commented Dec 26, 2017

yep

closing as duplicate of #17558

@jreback jreback closed this as completed Dec 26, 2017
@jreback jreback added Duplicate Report Duplicate issue or pull request Datetime Datetime data dtype Timezones Timezone data dtype labels Dec 26, 2017
@jreback jreback added this to the No action milestone Dec 26, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Datetime Datetime data dtype Duplicate Report Duplicate issue or pull request Timezones Timezone data dtype
Projects
None yet
Development

No branches or pull requests

3 participants