Skip to content

DOC: Remove mention that on with MultiIndex is supported with Rolling Groupby #39686

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
GeorgievF opened this issue Feb 8, 2021 · 3 comments · Fixed by #39747
Closed

DOC: Remove mention that on with MultiIndex is supported with Rolling Groupby #39686

GeorgievF opened this issue Feb 8, 2021 · 3 comments · Fixed by #39747
Labels
Docs Groupby Window rolling, ewma, expanding
Milestone

Comments

@GeorgievF
Copy link

GeorgievF commented Feb 8, 2021

  • [.] I have checked that this issue has not already been reported.

  • [.] I have confirmed this bug exists on the latest version of pandas.


Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example


dates = pd.date_range(start='2016-01-01 09:30:00', periods=20, freq='s')
df = pd.DataFrame({'A': [1] * 20 + [2] * 12 + [3] * 8,
                   'B': np.concatenate((dates, dates)),
                   'C': np.arange(40),})
df.set_index(['A', 'B'], inplace=True)
df.groupby('A').rolling('4s', on='B').C.mean()

Problem description

Multi-index dataframe gives the following error message:

ValueError: invalid on specified as B, must be a column (of DataFrame), an Index or None

when one layer of the multi index is used for groupby, while the other is used as a rolling datetime level.

Similar to #13966

Expected Output

A  B                  
1  2016-01-01 09:30:00     0.0
   2016-01-01 09:30:01     0.5
   2016-01-01 09:30:02     1.0
   2016-01-01 09:30:03     1.5
   2016-01-01 09:30:04     2.5
   2016-01-01 09:30:05     3.5
   2016-01-01 09:30:06     4.5
   2016-01-01 09:30:07     5.5
   2016-01-01 09:30:08     6.5
   2016-01-01 09:30:09     7.5
   2016-01-01 09:30:10     8.5
   2016-01-01 09:30:11     9.5
   2016-01-01 09:30:12    10.5
   2016-01-01 09:30:13    11.5
   2016-01-01 09:30:14    12.5
   2016-01-01 09:30:15    13.5
   2016-01-01 09:30:16    14.5
   2016-01-01 09:30:17    15.5
   2016-01-01 09:30:18    16.5
   2016-01-01 09:30:19    17.5
2  2016-01-01 09:30:00    20.0
   2016-01-01 09:30:01    20.5
   2016-01-01 09:30:02    21.0
   2016-01-01 09:30:03    21.5
   2016-01-01 09:30:04    22.5
   2016-01-01 09:30:05    23.5
   2016-01-01 09:30:06    24.5
   2016-01-01 09:30:07    25.5
   2016-01-01 09:30:08    26.5
   2016-01-01 09:30:09    27.5
   2016-01-01 09:30:10    28.5
   2016-01-01 09:30:11    29.5
3  2016-01-01 09:30:12    32.0
   2016-01-01 09:30:13    32.5
   2016-01-01 09:30:14    33.0
   2016-01-01 09:30:15    33.5
   2016-01-01 09:30:16    34.5
   2016-01-01 09:30:17    35.5
   2016-01-01 09:30:18    36.5
   2016-01-01 09:30:19    37.5
Name: C, dtype: float64

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 9d598a5
python : 3.8.5.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19041
machine : AMD64
processor : Intel64 Family 6 Model 42 Stepping 7, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United Kingdom.1252

pandas : 1.2.1
numpy : 1.19.2
pytz : 2021.1
dateutil : 2.8.1
pip : 20.3.3
setuptools : 52.0.0.post20210125
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.3
IPython : 7.20.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : 3.3.2
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : 1.6.0
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : 1.2.0
xlwt : None
numba : None

@GeorgievF GeorgievF added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Feb 8, 2021
@GeorgievF GeorgievF changed the title BUG: BUG: Rolling Groupby on both levels of a Multi-Index Fails Feb 8, 2021
@jreback jreback added Groupby Window rolling, ewma, expanding and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Feb 8, 2021
@mroeschke
Copy link
Member

Thanks for the report. Currently rolling on a multiindex level is unsupported: #34642

Going to close this issue as a duplicate of the linked issue.

@GeorgievF
Copy link
Author

It is described in the documents as a possibility. If the documentation is changed that wouldn't be a issue :)
By the way, I was asked in the linked issue to open a new issue for this particular situation.

@mroeschke
Copy link
Member

Good point; it is still mentioned in the documentation as a possibility. I will reopen this issue to note to correct the documentation.

@mroeschke mroeschke reopened this Feb 10, 2021
@mroeschke mroeschke changed the title BUG: Rolling Groupby on both levels of a Multi-Index Fails DOC: Remove mention that on with MultiIndex is supported with Rolling Groupby Feb 10, 2021
@mroeschke mroeschke added Docs and removed Bug labels Feb 10, 2021
jeet-parekh added a commit to jeet-parekh/pandas that referenced this issue Feb 11, 2021
@jreback jreback added this to the 1.3 milestone Feb 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Docs Groupby Window rolling, ewma, expanding
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants