BUG: In 2.1.0 .resample extrapolates indexes to any 'origin' defined with Timestamp

[maread99]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
assert pd.__version__ == '2.1.0'

index = pd.date_range(
    start=pd.Timestamp("2023-09-08 10:00"),
    end=pd.Timestamp("2023-09-08 10:20"),
    freq="10T"
)
srs = pd.Series([0, 1, 2], index=index)

# srs.index
# DatetimeIndex(['2023-09-08 10:00:00', '2023-09-08 10:10:00',
#                '2023-09-08 10:20:00'],
#               dtype='datetime64[ns]', freq='10T')

origin = pd.Timestamp("2023-09-08 09:10")
srs_rs = srs.resample("19T", origin=origin).sum()
srs_rs.index

# indexes are extrapolated back to the origin...
# DatetimeIndex(['2023-09-08 09:10:00', '2023-09-08 09:29:00',
#                '2023-09-08 09:48:00', '2023-09-08 10:07:00'],
#               dtype='datetime64[ns]', freq='19T')

Issue Description

In Pandas 2.1.0 passing 'origin' to Series.resample or DataFrame.resample as a Timestamp results in indexes being extrapolated all the way back to the 'origin', whereas prior to 2.1.0 the first resampled index would always fall within one 'rule' of the first index being resampled.

I suspect it's a bug as there is no change in behaviour with origin="start_day" and I couldn't find anything in the 2.1.0 release notes that suggests the change was intended.

Thanks in advance for looking at this.

Expected Behavior

Would have expected behaviour to be unchanged from 2.0.3:

import pandas as pd
assert pd.__version__ == '2.0.3'

index = pd.date_range(
    start=pd.Timestamp("2023-09-08 10:00"),
    end=pd.Timestamp("2023-09-08 10:20"),
    freq="10T"
)
srs = pd.Series([0, 1, 2], index=index)
srs.index

DatetimeIndex(['2023-09-08 10:00:00', '2023-09-08 10:10:00',
               '2023-09-08 10:20:00'],
              dtype='datetime64[ns]', freq='10T')

origin = pd.Timestamp("2023-09-08 09:10")
srs_rs = srs.resample("19T", origin=origin).sum()
srs_rs.index  # first index is within 1 'rule' of first index being resampled...

DatetimeIndex(['2023-09-08 09:48:00', '2023-09-08 10:07:00'], dtype='datetime64[ns]', freq='19T')

origin="start_day" exhibits no change in behaviour (no extrapolation beyond 1 'rule'). The following is the same for Pandas 2.0.3 and 2.1.0:

srs_rs = srs.resample("19T", origin="start_day").sum()
srs_rs.index

DatetimeIndex(['2023-09-08 09:49:00', '2023-09-08 10:08:00'], dtype='datetime64[ns]', freq='19T')

Installed Versions

INSTALLED VERSIONS

commit : ba1cccd
python : 3.9.13.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.22621
machine : AMD64
processor : Intel64
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United Kingdom.1252

pandas : 2.1.0
numpy : 1.25.2
pytz : 2023.3.post1
dateutil : 2.8.2
setuptools : 58.1.0
pip : 23.2.1
Cython : 3.0.0
pytest : 7.4.0
hypothesis : 6.82.0
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.9.3
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : 8.14.0
pandas_datareader : None
bs4 : None
bottleneck : None
dataframe-api-compat: None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : None
numexpr : 2.8.5
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : 3.8.0
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2023.3
qtpy : None
pyqt5 : None

[MarcoGorelli]

thanks @maread99 for the report, this looks intended but I will take a closer look

this is probably due to #53938

[maread99]

Hi @MarcoGorelli

Thanks for looking at this.

I wouldn't have thought that extrapolation should fall within the ambit of a resampling operation? Aren't the indexes that 2.1.0 is adding all superfluous? By definition there can be no data corresponding to them(?).

If the behavior is intended then it would raise the questions:

• Why not extrapolate back to the start of the first day when origin="start_day"?
• How would it be possible to define an origin earlier than 1 'rule` prior to the first index but not extrapolate the data to that origin? (i.e. as prior to 2.1.0 and IMHO a reasonable use case).

Thanks again.

[MarcoGorelli]

True, although pandas already does some extrapolation for indexes in the middle of the Series

In [4]: df = pd.DataFrame({'ts': [datetime(2020, 1, 1), datetime(2020, 1, 3)], 'val': [1, 3]})

In [5]: df.resample('1d', on='ts').sum()
Out[5]:
            val
ts
2020-01-01    1
2020-01-02    0
2020-01-03    3

Agree that there's an inconsistency with origin='start_day' though, thanks

[maread99]

Although aren't the added indexes in your example what's being explicitly requested when undertaking a resampling operation? Depending on the 'rule' such indexes can have values, for example...

srs = pd.Series(
    [1, 2, 3],
    index=[datetime(2020, 1, 1), datetime(2020, 1, 2), datetime(2020, 1, 4)]
)
srs

2020-01-01    1
2020-01-02    2
2020-01-04    3
dtype: int64

srs.resample('2d').sum()

2020-01-01    3
2020-01-03    3
Freq: 2D, dtype: int64

I'm not sure these new indexes are the same as introducing indexes outside of the original time series which can never have corresponding data.

IMHO the pre 2.1.0 behavior better reflects the idea of an 'origin', i.e. a timestamp from which to evaluate the first index of the resampled data, rather than the timestamp of the first resampled index (for which an option name 'start' would seem more appropriate).

Thanks

[MarcoGorelli]

having looked at this further, I think you're right, and that the inconsistency we've introduced is worse than the original bug

I'd suggest reverting #53938 and noting that origin only takes effect for tick-like frequencies

[maread99]

Great, thank you for resolving this and reverting!