BUG: overflow in Timedelta arithmetic

[jreback]

xref #17318 (comment)

This is not behaving on master, started after: b59f107

In [1]: date_range(start='1/1/1700', freq='B', periods=100000)
---------------------------------------------------------------------------
OverflowError                             Traceback (most recent call last)
OverflowError: Python int too large to convert to C long
Exception ignored in: 'pandas._libs.tslib._delta_to_nanoseconds'
OverflowError: Python int too large to convert to C long
Out[1]: DatetimeIndex(['1700-01-01', '1700-01-04', '1700-01-05', '1700-01-06', '1700-01-07'], dtype='datetime64[ns]', freq='B')

[jbrockmendel]

A sufficient condition to fix this particular error:

1. Revert cpdef int64_t _delta_to_nanoseconds(delta): --> def _delta_to_nanoseconds(delta): (with or without cpdef, shouldn't matter)
2. In _Timestamp.__add__ don't declare a type for nanos
3. Open an issue for a longer-term solution.

[jreback]

@jbrockmendel its actually much simpler than that.

3. there isn't really a good long term soln. well there is, multi-unit timedelta types, but that is quite a lot of work and will have to wait for pandas2.

[jreback]

2. In _Timestamp.add don't declare a type for nanos

not realistic, you want to declare types for things as you actually want the overflow

[jbrockmendel]

there isn't really a good long term soln. well there is, multi-unit timedelta types, but that is quite a lot of work and will have to wait for pandas2.

Yah. I've spent enough time with np_datetime.c to know that numpy has implemented multi-unit datetime64, haven't looked at their timedelta64 implementation that closely. I'll check if they've got anything that can be adapted.

[jreback]

There are actually 2 issues:

1. support multi-units (e.g. [s]) based Timedeltas.
2. properly cast when overflowing (then cast back)

these are both useful but non-trivial.