BUG: Inconsistent result from centered datetime-like rolling window aggregates

[dicristina]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Code Sample, a copy-pastable example

import pandas as pd

def rolling_test(window, df=None, center=True):
    if df is None:
        df = pd.DataFrame(
            {"x": 1},
            index=[pd.Timestamp("20130101 09:00:01"),
                   pd.Timestamp("20130101 09:00:02")
            ] 
        )
    
    df["right"] = df.rolling(window, closed="right", center=center).x.sum()
    df["both"] = df.rolling(window, closed="both", center=center).x.sum()
    df["left"] = df.rolling(window, closed="left", center=center).x.sum()
    df["neither"] = df.rolling(window, closed="neither", center=center).x.sum()
    
    return df

rolling_test('1T')

Problem description

When center=True and the offset is large enough to contain all of the rows, the first output row of rolling aggregates is wrong most of the time. This can happen for all possible values of the closed parameter. For the sample code the observed wrong results are 1 when closed is right or both and NaN when closed is left or neither. Here are some examples:

                     x  right  both  left  neither
2013-01-01 09:00:01  1    1.0   1.0   NaN      NaN
2013-01-01 09:00:02  1    2.0   2.0   2.0      2.0

                     x  right  both  left  neither
2013-01-01 09:00:01  1    2.0   2.0   2.0      NaN
2013-01-01 09:00:02  1    2.0   2.0   2.0      2.0

                     x  right  both  left  neither
2013-01-01 09:00:01  1    1.0   2.0   2.0      2.0
2013-01-01 09:00:02  1    2.0   2.0   2.0      2.0

I have tested with sum and count aggregates and the problem seems to be that the first row is taken by itself. The same behavior is present on 1.3.1 and on master (b8d750f).

Expected Output

The code sample uses an offset of 1T and the data only has two seconds worth of data points so any aggregate should include all the rows. Sometimes I get the correct result:

                     x  right  both  left  neither
2013-01-01 09:00:01  1    2.0   2.0   2.0      2.0
2013-01-01 09:00:02  1    2.0   2.0   2.0      2.0

Even if I were wrong about the expected result, there is the fact that the result vary from one moment to the next.

Output of pd.show_versions()

INSTALLED VERSIONS

commit : b8d750f
python : 3.9.4.final.0
python-bits : 64
OS : Darwin
OS-release : 19.6.0
Version : Darwin Kernel Version 19.6.0: Thu May 6 00:48:39 PDT 2021; root:xnu-6153.141.33~1/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.4.0.dev0+305.gb8d750fb90
numpy : 1.21.1
pytz : 2021.1
dateutil : 2.8.2
pip : 21.0.1
setuptools : 54.2.0
Cython : 0.29.24
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None

[jbrockmendel]

cc @mroeschke

[dicristina]

I'm working on a pull request for this. The reason for the inconsistency, as opposed to always getting the wrong result, is an out of bounds read of the index ndarray in the function calculate_variable_window_bounds. The problem itself is caused by the way the end of the window is calculated for the first row.

[simonjayhawkins]

the centered window capability for a datetime-like index was added #38780 (i.e. new in 1.3.0) so adding 1.3.2 milestone here. i.e. if we get a PR to fix, we should consider backporting.