Skip to content

datetime like series .dt properties fail after ts.dt.tz_localize(tz1).dt.tz_convert(tz2) sequence #11295

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
witosx opened this issue Oct 12, 2015 · 2 comments · Fixed by #11301
Labels
Bug Datetime Datetime data dtype Timezones Timezone data dtype
Milestone

Comments

@witosx
Copy link

witosx commented Oct 12, 2015

Second tz_convert fails with AmbiguousTimeError ('UTC' in the example, but other timezones fail too).

EDIT: Actually it's not only second tz_convert that fails, it looks like most (if not all) of the .dt accessed properties are affected

t = pd.Series(pd.date_range('2015-01-01', '2016-01-01', freq='T'))
t = t.dt.tz_localize('UTC').dt.tz_convert('America/Chicago')
t.head(10)
0   2014-12-31 18:00:00-06:00
1   2014-12-31 18:01:00-06:00
2   2014-12-31 18:02:00-06:00
3   2014-12-31 18:03:00-06:00
4   2014-12-31 18:04:00-06:00
5   2014-12-31 18:05:00-06:00
6   2014-12-31 18:06:00-06:00
7   2014-12-31 18:07:00-06:00
8   2014-12-31 18:08:00-06:00
9   2014-12-31 18:09:00-06:00
dtype: datetime64[ns, America/Chicago]
t.dt.tz_convert('UTC')
---------------------------------------------------------------------------

AmbiguousTimeError                        Traceback (most recent call last)

/usr/lib/python3.5/site-packages/pandas/core/series.py in _make_dt_accessor(self)
   2639         try:
-> 2640             return maybe_to_datetimelike(self)
   2641         except Exception:


/usr/lib/python3.5/site-packages/pandas/tseries/common.py in maybe_to_datetimelike(data, copy)
     48     if is_datetime64_dtype(data.dtype) or is_datetime64tz_dtype(data.dtype):
---> 49         return DatetimeProperties(DatetimeIndex(data, copy=copy, freq='infer'), index, name=data.name)
     50     elif is_timedelta64_dtype(data.dtype):


/usr/lib/python3.5/site-packages/pandas/util/decorators.py in wrapper(*args, **kwargs)
     88                     kwargs[new_arg_name] = new_arg_value
---> 89             return func(*args, **kwargs)
     90         return wrapper


/usr/lib/python3.5/site-packages/pandas/tseries/index.py in __new__(cls, data, freq, start, end, periods, copy, name, tz, verify_integrity, normalize, closed, ambiguous, dtype, **kwargs)
    343                     subarr = tslib.tz_localize_to_utc(ints, tz,
--> 344                                                       ambiguous=ambiguous)
    345 


pandas/tslib.pyx in pandas.tslib.tz_localize_to_utc (pandas/tslib.c:64516)()


AmbiguousTimeError: Cannot infer dst time from Timestamp('2015-11-01 01:00:00'), try using the 'ambiguous' argument


During handling of the above exception, another exception occurred:


AttributeError                            Traceback (most recent call last)

<ipython-input-2-eb75730cd6fc> in <module>()
----> 1 t.dt.tz_convert('UTC')


/usr/lib/python3.5/site-packages/pandas/core/generic.py in __getattr__(self, name)
   2239                 or name in self._metadata
   2240                 or name in self._accessors):
-> 2241             return object.__getattribute__(self, name)
   2242         else:
   2243             if name in self._info_axis:


/usr/lib/python3.5/site-packages/pandas/core/base.py in __get__(self, instance, owner)
    186             # this ensures that Series.str.<method> is well defined
    187             return self.accessor_cls
--> 188         return self.construct_accessor(instance)
    189 
    190     def __set__(self, instance, value):


/usr/lib/python3.5/site-packages/pandas/core/series.py in _make_dt_accessor(self)
   2640             return maybe_to_datetimelike(self)
   2641         except Exception:
-> 2642             raise AttributeError("Can only use .dt accessor with datetimelike "
   2643                                  "values")
   2644 


AttributeError: Can only use .dt accessor with datetimelike values
@witosx witosx changed the title datetime like series second tz_convert fails datetime like series .dt properties fail after ts.dt.tz_localize(tz1).dt.tz_convert(tz2) sequence Oct 12, 2015
@jorisvandenbossche jorisvandenbossche added Bug Datetime Datetime data dtype Timezones Timezone data dtype labels Oct 12, 2015
@jorisvandenbossche jorisvandenbossche added this to the 0.17.1 milestone Oct 12, 2015
@jorisvandenbossche
Copy link
Member

This seems to come down to the failing of DatetimeIndex(t) (which is used within the dt accessor):

In [37]: t.head()
Out[37]:
0   2014-12-31 18:00:00-06:00
1   2014-12-31 18:01:00-06:00
2   2014-12-31 18:02:00-06:00
3   2014-12-31 18:03:00-06:00
4   2014-12-31 18:04:00-06:00
dtype: datetime64[ns, America/Chicago]

In [38]: pd.DatetimeIndex(t)
---------------------------------------------------------------------------
AmbiguousTimeError                        Traceback (most recent call last)
<ipython-input-38-3e6430dea298> in <module>()
----> 1 pd.DatetimeIndex(t)

c:\users\vdbosscj\scipy\pandas-joris\pandas\util\decorators.pyc in wrapper(*args
, **kwargs)
     87                 else:
     88                     kwargs[new_arg_name] = new_arg_value
---> 89             return func(*args, **kwargs)
     90         return wrapper
     91     return _deprecate_kwarg

c:\users\vdbosscj\scipy\pandas-joris\pandas\tseries\index.pyc in __new__(cls, da
ta, freq, start, end, periods, copy, name, tz, verify_integrity, normalize, clos
ed, ambiguous, dtype, **kwargs)
    342                     ints = subarr.view('i8')
    343                     subarr = tslib.tz_localize_to_utc(ints, tz,
--> 344                                                       ambiguous=ambiguou
s)
    345
    346                 subarr = subarr.view(_NS_DTYPE)

tslib.pyx in pandas.tslib.tz_localize_to_utc (pandas\tslib.c:55271)()

AmbiguousTimeError: Cannot infer dst time from Timestamp('2015-11-01 01:00:00'),
 try using the 'ambiguous' argument

When providing tz data to DatetimeIndex, it first makes it naive and then converts back to the tz, which fails without supplying ambiguous='infer'.

@jreback
Copy link
Contributor

jreback commented Oct 12, 2015

@witosx give #11301 a try. This was passing thru the ambiguous kw incorrectly.

jreback added a commit that referenced this issue Oct 13, 2015
BUG: Bug in tz-conversions with an ambiguous time and .dt accessors, #11295
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Datetime Datetime data dtype Timezones Timezone data dtype
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants