POC/ENH: Timedelta min/max/resolution support non-nano

[jbrockmendel]

No description provided.

[mroeschke]

Thanks @jbrockmendel

[mroeschke]

I dont this this is directly related, but I think this hypothesis failure may be a near neighbor failure to the non-nano code you've been in the weeds with

022-07-08T19:09:27.0107944Z ================================== FAILURES ===================================
2022-07-08T19:09:27.0108768Z _______________ TestTimestampUnaryOps.test_round_sanity[round] ________________
2022-07-08T19:09:27.0188497Z [gw0] win32 -- Python 3.8.13 C:\Users\runneradmin\micromamba\envs\test\python.exe
2022-07-08T19:09:27.0188803Z 
2022-07-08T19:09:27.0192971Z >   return Timedelta._from_value_and_reso(res_value, self._reso)
2022-07-08T19:09:27.0193292Z 
2022-07-08T19:09:27.0193473Z pandas\_libs\tslibs\timestamps.pyx:521: 
2022-07-08T19:09:27.0193820Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2022-07-08T19:09:27.0194006Z 
2022-07-08T19:09:27.0194394Z >   def _from_value_and_reso(cls, int64_t value, NPY_DATETIMEUNIT reso):
2022-07-08T19:09:27.0198386Z E   OverflowError: int too big to convert
2022-07-08T19:09:27.0198647Z 
2022-07-08T19:09:27.0198846Z pandas\_libs\tslibs\timedeltas.pyx:1521: OverflowError
2022-07-08T19:09:27.0199075Z 
2022-07-08T19:09:27.0199272Z The above exception was the direct cause of the following exception:
2022-07-08T19:09:27.0199508Z 
2022-07-08T19:09:27.0199966Z self = <pandas.tests.scalar.timestamp.test_unary_ops.TestTimestampUnaryOps object at 0x000001C0E0CFFFA0>
2022-07-08T19:09:27.0204415Z method = <cyfunction Timestamp.round at 0x000001C0AE3BA520>
2022-07-08T19:09:27.0204681Z 
2022-07-08T19:09:27.0205195Z     @given(val=st.integers(iNaT + 1, lib.i8max))
2022-07-08T19:09:27.0205516Z >   @pytest.mark.parametrize(
2022-07-08T19:09:27.0206044Z         "method", [Timestamp.round, Timestamp.floor, Timestamp.ceil]
2022-07-08T19:09:27.0210300Z     )
2022-07-08T19:09:27.0210514Z 
2022-07-08T19:09:27.0210884Z pandas\tests\scalar\timestamp\test_unary_ops.py:301: 
2022-07-08T19:09:27.0211247Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2022-07-08T19:09:27.0211998Z pandas\tests\scalar\timestamp\test_unary_ops.py:351: in test_round_sanity
2022-07-08T19:09:27.0216046Z     assert np.abs((res - ts).value) < nanos
2022-07-08T19:09:27.0216452Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
2022-07-08T19:09:27.0216633Z 
2022-07-08T19:09:27.0216772Z >   raise OutOfBoundsDatetime(
2022-07-08T19:09:27.0221599Z E   pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: Result is too large for pandas.Timedelta. Convert inputs to datetime.datetime with 'Timestamp.to_pydatetime()' before subtracting.
2022-07-08T19:09:27.0222080Z 
2022-07-08T19:09:27.0227140Z pandas\_libs\tslibs\timestamps.pyx:525: OutOfBoundsDatetime
2022-07-08T19:09:27.0227891Z --------------------------------- Hypothesis ----------------------------------
2022-07-08T19:09:27.0228323Z Falsifying example: test_round_sanity(
2022-07-08T19:09:27.0228807Z     val=9223329600000000000,
2022-07-08T19:09:27.0229343Z     self=<pandas.tests.scalar.timestamp.test_unary_ops.TestTimestampUnaryOps at 0x1c0e0cfffa0>,
2022-07-08T19:09:27.0233440Z     method=<cyfunction Timestamp.round at 0x000001C0AE3BA520>,
2022-07-08T19:09:27.0233738Z )
2022-07-08T19:09:27.0233865Z 
2022-07-08T19:09:27.0238472Z You can reproduce this example by temporarily adding @reproduce_failure('6.47.1', b'AAAAf//ZZ2ZYgAA=') as a decorator on your test case

[GiorgioBalestrieri]

Hi @jbrockmendel, can I ask what was the context to this PR? This seems to be creating serious side-effects, so I'm wondering if it's even in the cards to drop support for Timedeltas larger than the datetime.timedelta range?

#53643 (comment)

[jbrockmendel]

datetimes in the distant past and future are needed for scientific use cases, mainly geology and astronomy. No, dropping this support is not in the cards. I advise you to avoid mixing-and-matching stdlib objects with pandas objects if this is causing trouble.

[GiorgioBalestrieri]

datetimes in the distant past and future are needed for scientific use cases, mainly geology and astronomy. No, dropping this support is not in the cards. I advise you to avoid mixing-and-matching stdlib objects with pandas objects if this is causing trouble.

thanks for the answer. I appreciate there are real-world use cases for this. I'd be very happy to avoid mixing and matching, the problem is that pd.Timedelta claims to be a subclass of datetime.timedelta, which it is not since it can lead to incorrect arithmetic, so even if a function's signature is (dt.datetime, dt.timedelta), someone/something might pass a pd.Timedelta object as its second argument, and no type checker would complain. This is particularly tricky because it does not lead to an exception being raised, it simply returns the wrong result, which can cause a lot of issues and confusion.

Perhaps the most logical thing is to make pd.Timedelta not a subclass of dt.timedelta, but that'd be definitely a breaking change, so it doesn't sound like a short-term fix.