TST: .timestamp() uses local timezone

[ghost]

Code Sample, a copy-pastable example if possible

~$ ./test_fast.sh 

=========== FAILURES ============
____ TestTimestamp.test_timestamp ____
[gw0] linux -- Python 3.6.3 /home/kris/.virtualenvs/pandas3/bin/python3.6

self = <pandas.tests.scalar.test_timestamp.TestTimestamp object at 0x7f4a725d2908>

    def test_timestamp(self):
        # GH#17329
        # tz-naive --> treat it as if it were UTC for purposes of timestamp()
        ts = Timestamp.now()
        uts = ts.replace(tzinfo=utc)
        assert ts.timestamp() == uts.timestamp()
    
        tsc = Timestamp('2014-10-11 11:00:01.12345678', tz='US/Central')
        utsc = tsc.tz_convert('UTC')
        # utsc is a different representation of the same time
        assert tsc.timestamp() == utsc.timestamp()
    
        if PY3:
            # should agree with datetime.timestamp method
            dt = ts.to_pydatetime()
>           assert dt.timestamp() == ts.timestamp()
E           AssertionError: assert 1509380391.670673 == 1509383991.670673
E            +  where 1509380391.670673 = <built-in method timestamp of datetime.datetime object at 0x7f4a728bee90>()
E            +    where <built-in method timestamp of datetime.datetime object at 0x7f4a728bee90> = datetime.datetime(2017, 10, 30, 17, 19, 51, 670673).timestamp
E            +  and   1509383991.670673 = <built-in method timestamp of Timestamp object at 0x7f4a7288fde0>()
E            +    where <built-in method timestamp of Timestamp object at 0x7f4a7288fde0> = Timestamp('2017-10-30 17:19:51.670673').timestamp

pandas/tests/scalar/test_timestamp.py:1086: AssertionError
____ TestTimeZones.test_replace_tzinfo ____
[gw0] linux -- Python 3.6.3 /home/kris/.virtualenvs/pandas3/bin/python3.6

self = <pandas.tests.tseries.test_timezones.TestTimeZones object at 0x7f4a71ab5400>

    def test_replace_tzinfo(self):
        # GH 15683
        dt = datetime(2016, 3, 27, 1)
        tzinfo = pytz.timezone('CET').localize(dt, is_dst=False).tzinfo
    
        result_dt = dt.replace(tzinfo=tzinfo)
        result_pd = Timestamp(dt).replace(tzinfo=tzinfo)
    
        if hasattr(result_dt, 'timestamp'):  # New method in Py 3.3
            assert result_dt.timestamp() == result_pd.timestamp()
        assert result_dt == result_pd
        assert result_dt == result_pd.to_pydatetime()
    
        result_dt = dt.replace(tzinfo=tzinfo).replace(tzinfo=None)
        result_pd = Timestamp(dt).replace(tzinfo=tzinfo).replace(tzinfo=None)
    
        if hasattr(result_dt, 'timestamp'):  # New method in Py 3.3
>           assert result_dt.timestamp() == result_pd.timestamp()
E           AssertionError: assert 1459036800.0 == 1459040400.0
E            +  where 1459036800.0 = <built-in method timestamp of datetime.datetime object at 0x7f4a71aa9d50>()
E            +    where <built-in method timestamp of datetime.datetime object at 0x7f4a71aa9d50> = datetime.datetime(2016, 3, 27, 1, 0).timestamp
E            +  and   1459040400.0 = <built-in method timestamp of Timestamp object at 0x7f4a71fd9840>()
E            +    where <built-in method timestamp of Timestamp object at 0x7f4a71fd9840> = Timestamp('2016-03-27 01:00:00').timestamp

pandas/tests/tseries/test_timezones.py:1290: AssertionError
===== 2 failed, 13366 passed, 1777 skipped, 10 xfailed, 1 xpassed in 111.18 seconds =====

Problem description

On python3.5 and python3.6 unittests fail as shown above.
Same results via pytest pandas and ./test_fast.sh.

Expected Output

Passing unittests.

Output of pd.show_versions()

INSTALLED VERSIONS

commit: 8449ffd
python: 3.6.3.final.0
python-bits: 64
OS: Linux
OS-release: 4.10.0-37-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.21.0rc1+67.g8449ffd
pytest: 3.2.3
pip: 9.0.1
setuptools: 36.6.0
Cython: None
numpy: 1.13.3
scipy: None
pyarrow: None
xarray: None
IPython: 6.2.1
sphinx: 1.6.5
patsy: None
dateutil: 2.6.1
pytz: 2017.3
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: None
html5lib: 1.0b10
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.9.6
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

[TomAugspurger]

I'm able to reproduce this locally. Strange that it isn't failing on Travis.

Probably from #17906. cc @jbrockmendel, are you able to reproduce locally? If not I can start debugging

[TomAugspurger]

FWIW, here's the difference:

(Pdb++) dt.timestamp() - ts.timestamp()
18000.0

That's 5 hours, and I'm in central time zone.

[jbrockmendel]

What TZ are Travis envs locale?

…

Sent from my iPhone

On Oct 30, 2017, at 11:29 AM, Tom Augspurger ***@***.***> wrote: I'm able to reproduce this locally. Strange that it isn't failing on Travis. Probably from #17906. cc @jbrockmendel, are you able to reproduce locally? If not I can start debugging — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[TomAugspurger]

Not sure, we do override one at https://github.com/pandas-dev/pandas/blob/master/.travis.yml#L40 On Mon, Oct 30, 2017 at 3:03 PM, jbrockmendel <[email protected]> wrote:

…

What TZ are Travis envs locale? Sent from my iPhone > On Oct 30, 2017, at 11:29 AM, Tom Augspurger ***@***.***> wrote: > > I'm able to reproduce this locally. Strange that it isn't failing on Travis. > > Probably from #17906. cc @jbrockmendel, are you able to reproduce locally? If not I can start debugging > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub, or mute the thread. > — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#18037 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABQHIgxsmmOWePP-QB-jtoEPajKNww5oks5sxiubgaJpZM4QLn7y> .

[jreback]

.timestamp() is really dumb here it uses the local tz if it’s not specified : https://docs.python.org/3/library/datetime.html#datetime-objects

so need to specify this in a context manager

[jreback]

Using the local timezone is about the worse possible behavior, but I guess this follows the c libraries :<

In [2]: ts = pd.Timestamp('2016-03-27 01:00:00')

In [4]: with set_timezone('EST'):
   ...:     print(ts.timestamp())
1459058400.0

In [6]: with set_timezone('CST'):
   ...:     print(ts.timestamp())
1459040400.0

In [9]: with set_timezone('CST'):
   ...:     print(ts.to_pydatetime().timestamp())
1459040400.0

[jreback]

@kchomski-reef if you'd like to do a PR to fix would be great!

[ghost]

@jreback I will definitely try if I find some time to look closer into it.

[jreback]

@kchomski-reef thanks!