pd.Timestamp.timestamp() doesn't match stdlib

[arsturges]

edit: See #29779 (comment)

I searched for this among open issues but couldn't find it. Apologies in advance if I overlooked an existing issue, resolved or otherwise.

Python 3.7.4 (default, Sep  7 2019, 18:29:04)
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import datetime
>>> import pandas as pd
>>> dt = datetime.datetime(1970, 1, 1)
>>> pdt = pd.Timestamp(1970, 1, 1)
>>> dt.timestamp() == pdt.timestamp()  # I expected this to be True
False
>>> dt.timestamp()  # I am in PST, hence the 8-hour offset from 0
28800.0
>>> pdt.timestamp()
0.0

Python3 docs explain that datetime.datetime.timestamp() assume local time if no tzinfo is specified: "Naive datetime instances are assumed to represent local time and this method relies on the platform C mktime() function to perform the conversion."

Problem description

I expected the behavior of pd.Timestamp.timestamp to match that of the stdlib (datetime.datetime.stdlib). Instead, pd.Timestamp.timestamp appears to treat naive timestamps differently.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : None
python : 3.7.4.final.0
python-bits : 64
OS : Darwin
OS-release : 18.7.0
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 0.25.1
numpy : 1.17.2
pytz : 2019.2
dateutil : 2.8.0
pip : 19.3.1
setuptools : 40.8.0
Cython : 0.29.13
pytest : 5.2.2
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : 2.8.3 (dt dec pq3 ext lo64)
jinja2 : 2.10.1
IPython : 7.8.0
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : 3.1.1
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
s3fs : None
scipy : None
sqlalchemy : 1.3.8
tables : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None

[WillAyd]

We diverge from the stdlib timestamp in quite a few ways so hard to judge if that is really problematic or not. What is the larger problem you are trying to solve?

[jbrockmendel]

Yah, for tz-naive we implement timestamp as if it were UTC. Among other things, this ensures that we get the same behavior regardless of where the code is running.

[arsturges]

this ensures that we get the same behavior regardless of where the code is running.

The pandas behavior is what I was expecting from datetime.datetime().timestamp(). I was surprised to find that datetime.datetime.timestamp() assumes local time, so it changes depending on what timezone it's run in. I like the pd.Timestamp.timestamp() implementation better, but I got burned when it didn't match the stdlib implementation.

What is the larger problem you are trying to solve?

One issue is that in an app where datetimes are coming a database via pd.from_sql() they are stored as pd.Timestamp, and other datetimes might be manually constructed from datetime.datetime. Both kinds of datetime are passed around the app, but their behaviors are different (with respect to .timestamp()). The solution is to force all timestamps to be of one class or the other, or force all timestamps to have timezones.

I don't have an answer for whether this should be changed. My first choice is for datetime.datetime.timestamp() to assume UTC instead of silently assuming local time.

[jbrockmendel]

I don't have an answer for whether this should be changed.

At the moment we have a comment in the Timestamp.timestamp method noting that it will behave differently from stdlib; putting that in the docstring would be worthwhile.

[TomAugspurger]

@arsturges does your thumbs-up indicate that you're interested in working on this?

[jbrockmendel]

Closing as duplicate of #22450.