BUG: DatetimeIndex.time incorrect in case of timezones

[ssikdar1]

An offshoot of #21230

http://pandas.pydata.org/pandas-docs/stable/timeseries.html?highlight=weekofmonth#time-date-components

From the docs:
There are several time/date properties that one can access from Timestamp or a collection of timestamps like a DatetimeIndex.
...
time | Returns datetime.time (does not contain timezone information)

But take this code here:

import pandas as pd

df1 = pd.DataFrame(data=[24, 25],
                   index=pd.DatetimeIndex(['2013-01-24 15:01:00+01:00',
                                           '2013-01-25 15:01:00+01:00'],
                                          dtype='datetime64[ns, CET]',
                                          name='Date', freq=None))

print(type(df1.index))
print("df1.index.time")
print(df1.index.time)
                 

returns:

<class 'pandas.core.indexes.datetimes.DatetimeIndex'>
df1.index.time
[datetime.time(14, 1, tzinfo=<DstTzInfo 'CET' CET+1:00:00 STD>)
 datetime.time(14, 1, tzinfo=<DstTzInfo 'CET' CET+1:00:00 STD>)]

• This is showing timezone information.

This may not be a bad thing, but if tz information is expected the docs should probably be changed?

[paste the output of pd.show_versions() here below this line]

pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.4.final.0
python-bits: 64
OS: Linux
OS-release: 4.13.0-43-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.23.0
pytest: None
pip: 9.0.1
setuptools: 28.8.0
Cython: None
numpy: 1.14.3
scipy: None
pyarrow: None
xarray: None
IPython: None
sphinx: None
patsy: None
dateutil: 2.7.3
pytz: 2018.4
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

[mroeschke]

Agreed, the docs should be changed to indicate that .time results may hold timezone information. Additionally we should ensure that tests exist for this behavior (didn't find any after a quick skim). PR's welcome!

[uds5501]

@ssikdar1 @mroeschke I am sorry but I couldn't recreate the output that your code showed. Is it possible that it is an OS based feature? (mine is Windows)

Any other method by which that output could be achieved?

[mroeschke]

What version of pandas are you using @uds5501? I don't believe this should be OS specific, and I can confirm this on the latest release.

In [11]: pd.__version__
Out[11]: u'0.23.0'

In [12]: d = pd.DatetimeIndex(['2013-01-24 14:01:00+01:00'], dtype='datetime64[ns, CET]', name='Date')

In [20]: d.time
Out[20]:
array([datetime.time(14, 1, tzinfo=<DstTzInfo 'CET' CET+1:00:00 STD>)],
      dtype=object)

[ssikdar1]

So it works for pandas 0.22.0, is it possible then its a bug from 22->23?

Python 3.6.5 (default, Apr 25 2018, 14:23:58) 
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas as pd
>>> df1 = pd.DataFrame(data=[24, 25],
...                    index=pd.DatetimeIndex(['2013-01-24 15:01:00+01:00',
...                                            '2013-01-25 15:01:00+01:00'],
...                                           dtype='datetime64[ns, CET]',
...                                           name='Date', freq=None))
>>> 
>>> print(type(df1.index))
<class 'pandas.core.indexes.datetimes.DatetimeIndex'>
>>> print("df1.index.time")
df1.index.time
>>> print(df1.index.time)
[datetime.time(14, 1) datetime.time(14, 1)]
>>> 
>>> 
>>> 
>>> pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 3.6.5.final.0
python-bits: 64
OS: Darwin
OS-release: 17.5.0
machine: x86_64
processor: i386
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.22.0

[mroeschke]

So it appears this behavior changed after #18461. However IMO, I would consider this an coincidental enhancement instead of a regression (the time is still correct and now gained timezone information).

Thoughts @jorisvandenbossche?

[jorisvandenbossche]

I actuallly get a different output as you: with a timezone (so changed from 0.22), but also the wrong hour (similar actually as in the top post):

In [50]: d = pd.DatetimeIndex(['2013-01-24 14:01:00+01:00'], dtype='datetime64[ns, CET]', name='Date')

In [51]: d.time
Out[51]: array([datetime.time(13, 1, tzinfo=<DstTzInfo 'CET' CET+1:00:00 STD>)], dtype=object)

so this is clearly a regression.

The fact that it is working correctly for you and not for me is strange .. What is the locale on your system? (in case it depends on that)

[mroeschke]

Oops sorry. You're absolutely correct @jorisvandenbossche. Not sure what I was looking at but I am getting the same results as you now.

[jorisvandenbossche]

It's from a similar PR #18461 as the related issue #21230 (which was from PR #18163)

[uds5501]

@mroeschke I am using '0.21.1'

[mroeschke]

@jorisvandenbossche so this is actually correct

In [2]: pd.__version__
Out[2]: u'0.23.0'

In [3]: d = pd.DatetimeIndex(['2013-01-24 14:01:00+01:00'], dtype='datetime64[ns, CET]', name='Date')

In [4]: d
Out[4]: DatetimeIndex(['2013-01-24 13:01:00+01:00'], dtype='datetime64[ns, CET]', name='Date', freq=None)

In [5]: d.time
Out[5]:
array([datetime.time(13, 1, tzinfo=<DstTzInfo 'CET' CET+1:00:00 STD>)],
      dtype=object)

But this confusion stems from this construction bug in #15938 (I believe)

[jorisvandenbossche]

@jorisvandenbossche so this is actually correct

Ah, yes, sorry, missed this comment. It's indeed the construction that is confusing here!