Timestamp.timestamp() inconsistent with Datetime.timestamp()

[patrickspry]

Code Sample, a copy-pastable example if possible

import datetime as dt
import pandas as pd

t = dt.datetime.now()
ts1 = t.timestamp()
ts2 = pd.Timestamp(t).timestamp()

print((ts1 - ts2) / 3600)

Problem description

Pandas and Datetime evidently have different defaults when handling time objects for which tzinfo=None. This changed somewhere in between pandas 0.19 and 0.23 (upgrading broke a bunch of my tests). I think keeping consistency between these two libraries would remove some of the confusion in the bizarrely complicated area that is python time-handling.

In the example, the two timestamp floats should match. Currently, my understanding is the datetime library defaults to the current timezone whereas Pandas assumes GMT. There isn't information in the Pandas API documentation regarding how Timestamp.timestamp() handles naive Timestamps (link).

[chris-b1]

ref #17906 (@jbrockmendel)

pandas/pandas/_libs/tslibs/timestamps.pyx

Line 417 in 68273a7

def timestamp(self):

What python does here seems strange? But I suppose should be consistent since we only have that method for compat.

import datetime as dt
import pandas as pd

t = dt.datetime.now()
ts1 = t.timestamp()
ts2 = pd.Timestamp(t).tz_localize('US/Central').timestamp()

print((ts1 - ts2) / 3600)
# 0.0

[jbrockmendel]

IIRC the thought process in the current implementation revolved around the stdlib version being nonsensical; "unix timestamp if if were local" isn't a concept I've seen elsewhere. The decision was that pd.Timestamp.now().timestamp should match time.time().

That said, differing from the stdlib is certainly not ideal. Personally I prefer the More Correct implementation we have, but understand if others prefer the More Compatible implementation.

[jbrockmendel]

Discussion in #29779 included idea of putting a note in the Timestamp.timestamp docstring (currently we have a note in a comment)

[torbiak]

pandas.Timestamp.timestamp() behaviour is at odds with pretty much everything else I use, AFAICT. Any place that I specify times, if I don't add a timezone it's assumed that that it's in the local timezone. And if I want timezone-naive timestamps to be interpreted as UTC I can configure my OS's timezone (or on POSIX change the TZ env var).

@jbrockmendel, you say "The decision was that pd.Timestamp.now().timestamp should match time.time().", but it doesn't, at least not for Pandas 1.2.3 and Python 3.9.2 and at a -0600 UTC offset:

print((time.time()), pd.Timestamp.now().timestamp())
1616724259.7153366 1616702659.715341

I think the idea in stdlib and elsewhere is less "unix timestamp if if were local", which I agree makes no sense, and more "interpret this timestamp as being in my local timezone".

[jbrockmendel]

At this point I think a) the stdlib behavior is decidedly unhelpful, b) since the method is so little used, there's not much harm in deprecating then changing ours to match

[mroeschke]

Appears based on the discussion that the Timestamp method is unlikely to change so closing