Skip to content

BUG: Using DatetimeIndex.date with timezone returns incorrect date #2… #21281

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jun 7, 2018
Merged
2 changes: 2 additions & 0 deletions doc/source/whatsnew/v0.23.1.txt
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,10 @@ Indexing
- Bug in :meth:`Series.reset_index` where appropriate error was not raised with an invalid level name (:issue:`20925`)
- Bug in :func:`interval_range` when ``start``/``periods`` or ``end``/``periods`` are specified with float ``start`` or ``end`` (:issue:`21161`)
- Bug in :meth:`MultiIndex.set_names` where error raised for a ``MultiIndex`` with ``nlevels == 1`` (:issue:`21149`)
- Bug in :attr:`DatetimeIndex.date` where an incorrect date is returned when the input date has a non-UTC timezone (:issue:`21230`)
- Bug in :class:`IntervalIndex` constructors where creating an ``IntervalIndex`` from categorical data was not fully supported (:issue:`21243`, issue:`21253`)
- Bug in :meth:`MultiIndex.sort_index` which was not guaranteed to sort correctly with ``level=1``; this was also causing data misalignment in particular :meth:`DataFrame.stack` operations (:issue:`20994`, :issue:`20945`, :issue:`21052`)
- Bug in :attr:`DatetimeIndex.time` where given a tz-aware Timestamp, a tz-aware Time is returned instead of tz-naive (:issue:`21267`)
-

I/O
Expand Down
2 changes: 1 addition & 1 deletion pandas/_libs/tslib.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ cdef inline object create_time_from_ts(
int64_t value, pandas_datetimestruct dts,
object tz, object freq):
""" convenience routine to construct a datetime.time from its parts """
return time(dts.hour, dts.min, dts.sec, dts.us, tz)
return time(dts.hour, dts.min, dts.sec, dts.us)


def ints_to_pydatetime(ndarray[int64_t] arr, tz=None, freq=None,
Expand Down
22 changes: 20 additions & 2 deletions pandas/core/indexes/datetimes.py
Original file line number Diff line number Diff line change
Expand Up @@ -2032,15 +2032,33 @@ def time(self):
"""
Returns numpy array of datetime.time. The time part of the Timestamps.
"""
return libts.ints_to_pydatetime(self.asi8, self.tz, box="time")

# If the Timestamps have a timezone that is not UTC,
# convert them into their i8 representation while
# keeping their timezone and not using UTC
if (self.tz is not None and self.tz is not utc):
timestamps = self._local_timestamps()
else:
timestamps = self.asi8

return libts.ints_to_pydatetime(timestamps, box="time")

@property
def date(self):
"""
Returns numpy array of python datetime.date objects (namely, the date
part of Timestamps without timezone information).
"""
return libts.ints_to_pydatetime(self.normalize().asi8, box="date")

# If the Timestamps have a timezone that is not UTC,
# convert them into their i8 representation while
# keeping their timezone and not using UTC
if (self.tz is not None and self.tz is not utc):
timestamps = self._local_timestamps()
else:
timestamps = self.asi8

return libts.ints_to_pydatetime(timestamps, box="date")

def normalize(self):
"""
Expand Down
28 changes: 27 additions & 1 deletion pandas/tests/indexes/datetimes/test_timezones.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"""
Tests for DatetimeIndex timezone-related methods
"""
from datetime import datetime, timedelta, tzinfo
from datetime import datetime, timedelta, tzinfo, date, time
from distutils.version import LooseVersion

import pytest
Expand Down Expand Up @@ -706,6 +706,32 @@ def test_join_utc_convert(self, join_type):
assert isinstance(result, DatetimeIndex)
assert result.tz.zone == 'UTC'

@pytest.mark.parametrize("dtype", [
None, 'datetime64[ns, CET]',
'datetime64[ns, EST]', 'datetime64[ns, UTC]'
])
def test_date_accessor(self, dtype):
# Regression test for GH#21230
expected = np.array([date(2018, 6, 4), pd.NaT])

index = DatetimeIndex(['2018-06-04 10:00:00', pd.NaT], dtype=dtype)
result = index.date

tm.assert_numpy_array_equal(result, expected)

@pytest.mark.parametrize("dtype", [
None, 'datetime64[ns, CET]',
'datetime64[ns, EST]', 'datetime64[ns, UTC]'
])
def test_time_accessor(self, dtype):
# Regression test for GH#21267
expected = np.array([time(10, 20, 30), pd.NaT])

index = DatetimeIndex(['2018-06-04 10:20:30', pd.NaT], dtype=dtype)
result = index.time

tm.assert_numpy_array_equal(result, expected)

def test_dti_drop_dont_lose_tz(self):
# GH#2621
ind = date_range("2012-12-01", periods=10, tz="utc")
Expand Down