Skip to content

TST: .timestamp() uses local timezone #18037

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ghost opened this issue Oct 30, 2017 · 9 comments · Fixed by #18232
Closed

TST: .timestamp() uses local timezone #18037

ghost opened this issue Oct 30, 2017 · 9 comments · Fixed by #18232
Labels
Testing pandas testing functions or related to the test suite Timezones Timezone data dtype
Milestone

Comments

@ghost
Copy link

ghost commented Oct 30, 2017

Code Sample, a copy-pastable example if possible

~$ ./test_fast.sh 

=========== FAILURES ============
____ TestTimestamp.test_timestamp ____
[gw0] linux -- Python 3.6.3 /home/kris/.virtualenvs/pandas3/bin/python3.6

self = <pandas.tests.scalar.test_timestamp.TestTimestamp object at 0x7f4a725d2908>

    def test_timestamp(self):
        # GH#17329
        # tz-naive --> treat it as if it were UTC for purposes of timestamp()
        ts = Timestamp.now()
        uts = ts.replace(tzinfo=utc)
        assert ts.timestamp() == uts.timestamp()
    
        tsc = Timestamp('2014-10-11 11:00:01.12345678', tz='US/Central')
        utsc = tsc.tz_convert('UTC')
        # utsc is a different representation of the same time
        assert tsc.timestamp() == utsc.timestamp()
    
        if PY3:
            # should agree with datetime.timestamp method
            dt = ts.to_pydatetime()
>           assert dt.timestamp() == ts.timestamp()
E           AssertionError: assert 1509380391.670673 == 1509383991.670673
E            +  where 1509380391.670673 = <built-in method timestamp of datetime.datetime object at 0x7f4a728bee90>()
E            +    where <built-in method timestamp of datetime.datetime object at 0x7f4a728bee90> = datetime.datetime(2017, 10, 30, 17, 19, 51, 670673).timestamp
E            +  and   1509383991.670673 = <built-in method timestamp of Timestamp object at 0x7f4a7288fde0>()
E            +    where <built-in method timestamp of Timestamp object at 0x7f4a7288fde0> = Timestamp('2017-10-30 17:19:51.670673').timestamp

pandas/tests/scalar/test_timestamp.py:1086: AssertionError
____ TestTimeZones.test_replace_tzinfo ____
[gw0] linux -- Python 3.6.3 /home/kris/.virtualenvs/pandas3/bin/python3.6

self = <pandas.tests.tseries.test_timezones.TestTimeZones object at 0x7f4a71ab5400>

    def test_replace_tzinfo(self):
        # GH 15683
        dt = datetime(2016, 3, 27, 1)
        tzinfo = pytz.timezone('CET').localize(dt, is_dst=False).tzinfo
    
        result_dt = dt.replace(tzinfo=tzinfo)
        result_pd = Timestamp(dt).replace(tzinfo=tzinfo)
    
        if hasattr(result_dt, 'timestamp'):  # New method in Py 3.3
            assert result_dt.timestamp() == result_pd.timestamp()
        assert result_dt == result_pd
        assert result_dt == result_pd.to_pydatetime()
    
        result_dt = dt.replace(tzinfo=tzinfo).replace(tzinfo=None)
        result_pd = Timestamp(dt).replace(tzinfo=tzinfo).replace(tzinfo=None)
    
        if hasattr(result_dt, 'timestamp'):  # New method in Py 3.3
>           assert result_dt.timestamp() == result_pd.timestamp()
E           AssertionError: assert 1459036800.0 == 1459040400.0
E            +  where 1459036800.0 = <built-in method timestamp of datetime.datetime object at 0x7f4a71aa9d50>()
E            +    where <built-in method timestamp of datetime.datetime object at 0x7f4a71aa9d50> = datetime.datetime(2016, 3, 27, 1, 0).timestamp
E            +  and   1459040400.0 = <built-in method timestamp of Timestamp object at 0x7f4a71fd9840>()
E            +    where <built-in method timestamp of Timestamp object at 0x7f4a71fd9840> = Timestamp('2016-03-27 01:00:00').timestamp

pandas/tests/tseries/test_timezones.py:1290: AssertionError
===== 2 failed, 13366 passed, 1777 skipped, 10 xfailed, 1 xpassed in 111.18 seconds =====

Problem description

On python3.5 and python3.6 unittests fail as shown above.
Same results via pytest pandas and ./test_fast.sh.

Expected Output

Passing unittests.

Output of pd.show_versions()

INSTALLED VERSIONS

commit: 8449ffd
python: 3.6.3.final.0
python-bits: 64
OS: Linux
OS-release: 4.10.0-37-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.21.0rc1+67.g8449ffd
pytest: 3.2.3
pip: 9.0.1
setuptools: 36.6.0
Cython: None
numpy: 1.13.3
scipy: None
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: 1.6.5
patsy: None
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 1.0b10
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@TomAugspurger
Copy link
Contributor

I'm able to reproduce this locally. Strange that it isn't failing on Travis.

Probably from #17906. cc @jbrockmendel, are you able to reproduce locally? If not I can start debugging

@TomAugspurger
Copy link
Contributor

TomAugspurger commented Oct 30, 2017

FWIW, here's the difference:

(Pdb++) dt.timestamp() - ts.timestamp()
18000.0

That's 5 hours, and I'm in central time zone.

@jbrockmendel
Copy link
Member

jbrockmendel commented Oct 30, 2017 via email

@TomAugspurger
Copy link
Contributor

TomAugspurger commented Oct 30, 2017 via email

@jreback
Copy link
Contributor

jreback commented Oct 30, 2017

.timestamp() is really dumb here it uses the local tz if it’s not specified : https://docs.python.org/3/library/datetime.html#datetime-objects

so need to specify this in a context manager

@jreback
Copy link
Contributor

jreback commented Oct 31, 2017

Using the local timezone is about the worse possible behavior, but I guess this follows the c libraries :<

In [2]: ts = pd.Timestamp('2016-03-27 01:00:00')

In [4]: with set_timezone('EST'):
   ...:     print(ts.timestamp())
1459058400.0

In [6]: with set_timezone('CST'):
   ...:     print(ts.timestamp())
1459040400.0

In [9]: with set_timezone('CST'):
   ...:     print(ts.to_pydatetime().timestamp())
1459040400.0

@jreback jreback added Difficulty Intermediate Testing pandas testing functions or related to the test suite Timezones Timezone data dtype labels Oct 31, 2017
@jreback jreback added this to the 0.21.1 milestone Oct 31, 2017
@jreback
Copy link
Contributor

jreback commented Oct 31, 2017

@kchomski-reef if you'd like to do a PR to fix would be great!

@ghost
Copy link
Author

ghost commented Nov 1, 2017

@jreback I will definitely try if I find some time to look closer into it.

@jreback
Copy link
Contributor

jreback commented Nov 2, 2017

@kchomski-reef thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Testing pandas testing functions or related to the test suite Timezones Timezone data dtype
Projects
None yet
3 participants