Skip to content

describe() does not report first and last on tz-aware datetime Series #21328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
louispotok opened this issue Jun 5, 2018 · 2 comments · Fixed by #21332
Closed

describe() does not report first and last on tz-aware datetime Series #21328

louispotok opened this issue Jun 5, 2018 · 2 comments · Fixed by #21332
Labels
Bug Timezones Timezone data dtype
Milestone

Comments

@louispotok
Copy link
Contributor

Code Sample, a copy-pastable example if possible

s = pd.Series(pd.date_range("2018-01-01","2018-01-05"))
s.describe() # includes first and last
s.dt.tz_localize('US/Eastern').describe() # does not include first and last

Problem description

The describe method on a timezone-naive datetime Series reports the "first" (min) and "last" (max) values. I would expect the same to be true for a timezone-aware datetime Series, since min and max are defined there as well.

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.4.final.0
python-bits: 64
OS: Linux
OS-release: 4.13.0-43-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.23.0
pytest: 3.4.2
pip: 9.0.1
setuptools: 38.4.0
Cython: None
numpy: 1.14.3
scipy: 1.1.0
pyarrow: 0.9.0
xarray: None
IPython: 6.2.1
sphinx: 1.7.3
patsy: 0.5.0
dateutil: 2.7.3
pytz: 2018.4
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: 0.4.0
matplotlib: 2.1.2
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 1.0.1
sqlalchemy: 1.2.7
pymysql: None
psycopg2: 2.7.4 (dt dec pq3 ext lo64)
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@mroeschke
Copy link
Member

Thanks for the report; that definitely sounds reasonable. This block below needs some additional logic to handle timezone aware datetimes:

pandas/pandas/core/generic.py

Lines 8533 to 8538 in 7dc6f70

if is_datetime64_dtype(data):
asint = data.dropna().values.view('i8')
names += ['top', 'freq', 'first', 'last']
result += [tslib.Timestamp(top), freq,
tslib.Timestamp(asint.min()),
tslib.Timestamp(asint.max())]

Pull requests are welcome!

@mroeschke mroeschke added Timezones Timezone data dtype Bug labels Jun 5, 2018
@mroeschke mroeschke added this to the Next Major Release milestone Jun 5, 2018
@louispotok
Copy link
Contributor Author

Right, looks like replacing that check with is_datetime64_any_dtype would catch the tz-aware ones.

Thanks, I'll try to get a PR in soon.

@jreback jreback modified the milestones: Next Major Release, 0.24.0 Jun 19, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Timezones Timezone data dtype
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants