Skip to content

get localized Timestamp using ".at" cannot get the time zone info #15822

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
AndsonYe opened this issue Mar 28, 2017 · 2 comments
Closed

get localized Timestamp using ".at" cannot get the time zone info #15822

AndsonYe opened this issue Mar 28, 2017 · 2 comments
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves Regression Functionality that used to work in a prior pandas version Timezones Timezone data dtype
Milestone

Comments

@AndsonYe
Copy link

AndsonYe commented Mar 28, 2017

Code Sample, a copy-pastable example if possible

data = {'name':['John', 'Anderson'], 'date':[pd.Timestamp(2017,3,13,13,32,56), pd.Timestamp(2017,2,16,12,10,3)]}
test_df = pd.DataFrame(data=data)
test_df['date'] = test_df['date'].map(lambda x:x.tz_localize('Asia/Shanghai'))

In [18]: test_df.at[0,'date']
Out[18]: Timestamp('2017-03-13 05:32:56')

In [19]: test_df.loc[0,'date']
Out[19]: Timestamp('2017-03-13 13:32:56+0800', tz='Asia/Shanghai')

Problem description

The sample code is as above.
I have a column in DataFrame that contains the datetime values which are Timestamp object in pandas.
After I localise the timestamp, while using .at getter and .loc getter, I get different results.
One is without time zone info, the other is with time zone info.

Is this a bug? How can I get the timestamp with time zone info using ".at" getter?

Expected Output

test_df.at[0,'date']
Timestamp('2017-03-13 13:32:56+0800', tz='Asia/Shanghai')

Output of pd.show_versions()

# Paste the output here pd.show_versions() here

In [20]: pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.6.final.0
python-bits: 64
OS: Linux
OS-release: 3.19.0-66-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_HK.UTF-8
LOCALE: None.None

pandas: 0.19.2
nose: 1.3.7
pip: 9.0.1
setuptools: 25.1.1
Cython: 0.24.1
numpy: 1.12.1
scipy: 0.18.1
statsmodels: None
xarray: None
IPython: 5.0.0
sphinx: None
patsy: None
dateutil: 2.6.0
pytz: 2016.10
blosc: None
bottleneck: None
tables: None
numexpr: None
matplotlib: 1.5.1
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 0.999
httplib2: 0.8
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: None
boto: None
pandas_datareader: None

@jorisvandenbossche
Copy link
Member

I can confirm this bug on 0.19.2.
And actually, there is also a regression on master, as now both at and loc return the timestamp without timezone information:

In [2]: test_df.dtypes
Out[2]: 
date    datetime64[ns, Asia/Shanghai]
name                           object
dtype: object

In [3]: test_df.at[0,'date']
Out[3]: Timestamp('2017-03-13 05:32:56')

In [4]: test_df.loc[0,'date']
Out[4]: Timestamp('2017-03-13 05:32:56')

In [5]: pd.__version__
Out[5]: '0.19.0+686.g71f621f'

@AndsonYe As a temporary workaround, you can do the indexing in two steps: first access the column, then the value from that column:

In [6]: test_df['date'][0]
Out[6]: Timestamp('2017-03-13 13:32:56+0800', tz='Asia/Shanghai')

@jorisvandenbossche jorisvandenbossche added Bug Indexing Related to indexing on series/frames, not to indexes themselves Regression Functionality that used to work in a prior pandas version Timezones Timezone data dtype labels Mar 28, 2017
@jorisvandenbossche jorisvandenbossche added this to the 0.20.0 milestone Mar 28, 2017
@jreback
Copy link
Contributor

jreback commented Mar 28, 2017

this is a duplicate: #12938

jreback added a commit to jreback/pandas that referenced this issue Mar 28, 2017
mattip pushed a commit to mattip/pandas that referenced this issue Apr 3, 2017
closes pandas-dev#15822

Author: Jeff Reback <[email protected]>

Closes pandas-dev#15827 from jreback/at and squashes the following commits:

4fcd2c6 [Jeff Reback] BUG: bug in .at/.loc indexing with a tz-aware columns
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves Regression Functionality that used to work in a prior pandas version Timezones Timezone data dtype
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants