BUG: rolling groupby does not respect min_periods when center=True

[wfvining]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

Code Sample, a copy-pastable example

s = pd.Series(1, index=list(range(0, 5)) * 4)
s.groupby(s.index).rolling(window=3, center=True, min_periods=1).sum()

Outputs:

0  0    2.0
   0    3.0
   0    3.0
   0    1.0
1  1    2.0
   1    3.0
   1    3.0
   1    1.0
2  2    2.0
   2    3.0
   2    3.0
   2    1.0
3  3    2.0
   3    3.0
   3    3.0
   3    1.0
4  4    2.0
   4    3.0
   4    3.0
   4    NaN
dtype: float64

Problem description

For the given input s, where every element is 1, the snippet above should return the same as counting the number of values in each window:

s.groupby(s.index).rolling(window=3, center=True, min_periods=1).count()

The output above is inconsistent with versions of pandas prior to 1.1 and is inconsistent in the semantics of min_periods; the first element of each group has the same number of periods as the last, but its value is computed correctly. The last element of each group is also computed incorrectly (should be 2). Finally, the last element of the final group (4) should not be NA, but should again be 2.

Some similar issues have been reported since the changes to .groupby().rolling() made in pandas 1.1.0.

• BUG: Rolling min_periods not working on groupby object #36040
• BUG: groupby.rolling.mean seems to roll over different groups when center=True #37141 might cover the same bug, but it is not immediately clear (I would also very much prefer to have this fixed in the 1.1.x line rather than wait for 1.2 and need to exclude 1.1.x in my dependencies).

Expected Output

0  0    2.0
   0    3.0
   0    3.0
   0    2.0
1  1    2.0
   1    3.0
   1    3.0
   1    2.0
2  2    2.0
   2    3.0
   2    3.0
   2    2.0
3  3    2.0
   3    3.0
   3    3.0
   3    2.0
4  4    2.0
   4    3.0
   4    3.0
   4    2.0
dtype: float64

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 67a3d42
python : 3.8.5.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.18362
machine : AMD64
processor : Intel64 Family 6 Model 142 Stepping 12, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United States.1252

pandas : 1.1.4
numpy : 1.19.1
pytz : 2020.1
dateutil : 2.8.1
pip : 20.2.2
setuptools : 49.6.0.post20200814
Cython : None
pytest : 6.0.1
hypothesis : None
sphinx : 2.2.0
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : None
pandas_datareader: None
bs4 : 4.9.1
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.3.1
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
pyxlsb : None
s3fs : None
scipy : 1.5.0
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : None

[ukarroum]

Thank you for reporting the issue.

I checked with your code example, and the issue seems solved on the master version (1.2.0.dev0+1080.g288bd7054.dirty).

>>> s = pd.Series(1, index=list(range(0, 5)) * 4)
>>> s.groupby(s.index).rolling(window=3, center=True, min_periods=1).sum()
0  0    2.0
   0    3.0
   0    3.0
   0    2.0
1  1    2.0
   1    3.0
   1    3.0
   1    2.0
2  2    2.0
   2    3.0
   2    3.0
   2    2.0
3  3    2.0
   3    3.0
   3    3.0
   3    2.0
4  4    2.0
   4    3.0
   4    3.0
   4    2.0
dtype: float64
>>> pd.__version__
'1.2.0.dev0+1080.g288bd7054.dirty'
>>> 

So my guess the issue has already been solved and will soon (if not yet) be available on 1.2 version.

[wfvining]

@ukarroum Thanks for checking that, I very much appreciate it. I would guess this was actually the same bug as #37141.