Skip to content

TST: Add test for groupby rename categories with multiIndexes #51555

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed

TST: Add test for groupby rename categories with multiIndexes #51555

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b8a3dc7
Add test
Feb 22, 2023
894fdb8
Merge branch 'main' into operation-with-multipleindex
luke396 Mar 4, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions pandas/tests/groupby/test_categorical.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2013,3 +2013,48 @@ def test_many_categories(as_index, sort, index_kind, ordered):
expected = DataFrame({"a": Series(index), "b": data})

tm.assert_frame_equal(result, expected)


@pytest.mark.parametrize("op", ["add", "sub", "mul", "div"])
def test_groupyby_rename_categories_operation_with_multiindex(op):
# 51500
data = DataFrame(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is too complicated. First step is to figure out what changed and fixed the bug. Afterwards, we want a dedicated test for this operation

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have to admit that finding bugs that have been fixed is difficult.

However, I searched the entire test folder and there doesn't seem to be a test written for groupyby along with rename_categories. I would like to add some simpler test cases to test that the behavior between their two methods is as expected. @phofl any suggestions?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think that testing this is necessary. If rename_categories provides a value Categorical, this should work as expected. I'd suggest going back to a version where the test was failing

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BUG: groupby with CategoricalIndex doesn't include unobserved categories by rhshadrach · Pull Request #49373 · pandas-dev/pandas fix the bug.

After comparing the buggy version and the main branch, I noticed that the results from the groupby may differ between the two versions, which in turn led to a series of subsequent issues.

In my opinion, this pull request can be closed because the test cases added to fix the previous bug appear to be complete.

[["C", "B", "B"], ["B", "A", "A"], ["B", "A", "B"]], columns=["0", "1", "2"]
)

def astype_rename_categories(data):
return data.astype("category").cat.rename_categories({"C": "B", "B": "C"})

data["0"] = astype_rename_categories(data["0"])

gb = data.groupby(["0", "1"])

s1 = gb["2"].value_counts()
s2 = gb.size()
result = getattr(s1, op)(s2)

s1_expected_idx = DataFrame(
[
("B", "A", "A"),
("B", "A", "B"),
("B", "B", "A"),
("B", "B", "B"),
("C", "A", "A"),
("C", "A", "B"),
("C", "B", "B"),
("C", "B", "A"),
],
columns=["0", "1", "2"],
)
s1_expected_idx["0"] = astype_rename_categories(s1_expected_idx["0"])
s1_expected = Series(
[1, 1, 0, 0, 0, 0, 1, 0], index=MultiIndex.from_frame(s1_expected_idx)
)
s2_expected_idx = DataFrame(
[("B", "A"), ("B", "B"), ("C", "A"), ("C", "B")], columns=["0", "1"]
)
s2_expected_idx["0"] = astype_rename_categories(s2_expected_idx["0"])
s2_expected = Series([2, 0, 0, 1], index=MultiIndex.from_frame(s2_expected_idx))

expected = getattr(s1_expected, op)(s2_expected)
tm.assert_series_equal(result, expected)