BUG: RollingGroupby when groupby key is in the index

[mroeschke]

• closes BUG: RollingGroupby duplicates columns in index even with group_keys=False #37641
• tests added / passed
• passes black pandas
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

[rhshadrach]

This will remove an index label in the case of partial overlap between the index labels and the groupby labels; I am wondering if a more natural behavior here would be "add any groupby labels to the index if they aren't already there".

[mroeschke]

Not sure if I 100% follow, but this is the resulting index behavior I am trying to mimic here:

In [1]: from pandas import *

In [2]:         arrays = [["val1", "val1", "val2"], ["val1", "val1", "val2"]]
   ...:         index = MultiIndex.from_arrays(arrays, names=("idx1", "idx2"))
   ...:
   ...:         df = DataFrame({"A": [1, 1, 2], "B": range(3)}, index=index)

In [3]: df
Out[3]:
           A  B
idx1 idx2
val1 val1  1  0
     val1  1  1
val2 val2  2  2

# idx2 is dropped
In [4]: df.groupby(["idx1", "A"]).mean()
Out[4]:
          B
idx1 A
val1 1  0.5
val2 2  2.0

[rhshadrach]

Shouldn't groupby(...).rolling behave more like a transform than a reduction? In any case, the behavior here does not mimic a reduction in the case when the group labels are entirely not in the index (please correct me if this is wrong!). This means we are mimicking the behavior of a reduction in one case but not another.

I almost never use rolling in my work so I don't have much of an opinion as to what the right behavior is. Just noticed what I think is an inconsistency.

[mroeschke]

Yes groupby.rolling should essentially act as a transform, and I think in both test cases I added the resulting shape (number of rows) is maintained with consideration that not all columns should be maintained since one of the columns was a grouping key.

In [7]:         arrays = [["val1", "val1", "val2"], ["val1", "val1", "val2"]]
   ...:         index = MultiIndex.from_arrays(arrays, names=("idx1", "idx2"))
   ...:
   ...:         df = DataFrame({"A": [1, 1, 2], "B": range(3)}, index=index)

In [8]: result = df.groupby(["idx1", "A"]).rolling(1).mean()

In [9]: result
Out[9]:
          B
idx1 A
val1 1  0.0
     1  1.0
val2 2  2.0

In [10]: df
Out[10]:
           A  B
idx1 idx2
val1 val1  1  0
     val1  1  1
val2 val2  2  2

[jreback]

i think this is ok to backport ton 1.1.5 if possible

[simonjayhawkins]

this fixes a regression from 1.0.5 -> 1.1.0, so backport is preferable.

[jreback]

thanks @mroeschke

[simonjayhawkins]

@meeseeksdev backport 1.1.x