REF: de-duplicate iterate_slices

[jbrockmendel]

# From DataFrameGroupBy
    def _iterate_slices(self) -> Iterable[Series]:
        obj = self._selected_obj
        if self.axis == 1:
            obj = obj.T

        if isinstance(obj, Series) and obj.name not in self.exclusions:
            # Occurs when doing DataFrameGroupBy(...)["X"]
            yield obj
        else:
            for label, values in obj.items():
                if label in self.exclusions:
                    continue

                yield values

Iterating over self._selected_obj but then excluding self.exclusons seems a lot like just iterating over self._obj_with_exclusions. I checked that using obj_with_exclusions does break things, but didn't figure out why.

it'd be nice if we could simplify this, or at least document why we need a third thing. cc @rhshadrach

If it were just obj_with_exclusions, then in DataFrameGroupBy._indexed_output_to_ndframe we could just use self._obj_with_exclusions.columns instead of re-constructing an Index (i think this re-construction is largely leftover from a time when we dropped nuisance columns)

Expected Behavior

NA

Installed Versions

Replace this line with the output of pd.show_versions()

[rhshadrach]

I'm seeing 1 test in groupby fail due to this behavior

df = pd.DataFrame({'a': [1, 1, 2], 'b': [3, 3, 4], 'c': [5, 6, 7]})
gb = df.groupby(['a', 'b'])
result = gb.sum()
print(result)
#       c
# a b    
# 1 3  11
# 2 4   7

gb2 = gb[['a', 'b', 'c']]
result2 = gb2.sum()
print(result2)
#      a  b   c
# a b          
# 1 3  2  6  11
# 2 4  2  4   7

In the 2nd example, 'a' and 'b' are in the exclusions but this is ignored since _obj_with_exclusions is determined via _selection in this case:

pandas/pandas/core/base.py

Lines 214 to 228 in 74d5c19

	def _obj_with_exclusions(self):
	if isinstance(self.obj, ABCSeries):
	return self.obj
	if self._selection is not None:
	return self.obj._getitem_nocopy(self._selection_list)
	if len(self.exclusions) > 0:
	# equivalent to `self.obj.drop(self.exclusions, axis=1)
	# but this avoids consolidating and making a copy
	# TODO: following GH#45287 can we now use .drop directly without
	# making a copy?
	return self.obj._drop_axis(self.exclusions, axis=1, only_slice=True)
	else:
	return self.obj

When using .sum(), we never hit _iterate_slices. Within agg, _iterate_slices causes inconsistent behavior:

result3 = gb2.agg(lambda x: x.sum())
print(result3)
#       c
# a b
# 1 3  11
# 2 4   7

To get consistent behavior, I think we can just replace _selected_obj with _obj_with_exclusions and remove any use of exclusions.

[jbrockmendel]

nice! thanks for tracking this down