added test to indexing on groupby, #32464

[gabrieldi95]

• closes Grouping by all columns of an empty DataFrame should produce MultiIndex, but doesn't #32464
• tests added / passed
• Ensure all linting tests pass, see here for how to run them

This makes sure an index of a groupby with more then one column is always a MultiIndex.

[mroeschke]

Instead could you construct the full DataFrame result of a.groupby(["a", "b", "c"]).sum() and a.groupby(["a", "b"]).sum() and use tm.assert_frame_equal?

Also could you move this test to a more appropriate groupby testing file?

[gabrieldi95]

Hi @mroeschke, I changed it so it uses tm.assert_frame_equal, but I'm not sure what the appropriate testing file would be.

[mroeschke]

Probably pandas/tests/groupby/test_function.py

[gabrieldi95]

I tried using tm.assert_frame_equal to test if a.groupby(["a", "b", "c"]).sum() has a MultiIndex like this:

    a = DataFrame({"a": [], "b": [], "c": []})
    result = a.groupby(["a", "b", "c"]).sum()

    expected_index = MultiIndex.from_arrays([[], [], []], names=("a", "b", "c"))
    expected = DataFrame(
        np.ndarray((0, 0)), dtype="float64", index=expected_index, columns=Index([])
    )

    # Tests if groupby with all columns has a multiindex
    tm.assert_frame_equal(result, expected)

But it didn't work, so I'm keeping just the other assertion, that tests if both groupby's are equal.

[mroeschke]

Okay could you change the assertion back to testing the MultiIndex result if the new assertion can't test the whole frame?

[gabrieldi95]

Sure! Like this?

# GH 32464
# Test if index after groupby with more then one column is always MultiIndex
a = DataFrame({"a": [], "b": [], "c": []})

agg_1 = a.groupby(["a", "b"]).sum()
agg_2 = a.groupby(["a", "b", "c"]).sum()

# Tests if group by with all columns has a MultiIndex
assert isinstance(agg_2.index, pd.core.indexes.multi.MultiIndex)

result = agg_1.iloc[:, :0]
expected = agg_2.droplevel("c")

# Tests if both agreggations have multiindex
tm.assert_frame_equal(result, expected)

[mroeschke]

You can use tm.assert_index_equal

[mroeschke]

assert_index_equal already checks this

[mroeschke]

Instead, assert the index of these two results independently.

[gabrieldi95]

Sorry, I don't get what you mean...
You mean something like this?

tm.assert_index_equal(a.groupby(["a", "b", "c"]).sum().index, pd.MultiIndex.from_arrays([[], [], []], names=("a", "b", "c")))
tm.assert_index_equal(a.groupby(["a", "b"]).sum().index, pd.MultiIndex.from_arrays([[], []], names=("a", "b")))

I tried but it doesn't work, it says AssertionError: MultiIndex level [0] are different

[gabrieldi95]

But I tried this and it passed:

def test_if_is_multiindex():
    # GH 32464
    # Test if index after groupby with more then one column is always MultiIndex
    a = DataFrame({"a": [1, 2], "b": [5, 6], "c": [8, 9]})

    result = a.groupby(["a", "b"]).sum().index
    expected = pd.MultiIndex.from_arrays([[1,2],[5,6]], names=("a", "b"))
    
    tm.assert_index_equal(result, expected)

    result = a.groupby(["a", "b", "c"]).sum().index
    expected = pd.MultiIndex.from_arrays([[1,2],[5,6],[8,9]], names=("a", "b", "c"))

    tm.assert_index_equal(result, expected)

[mroeschke]

You can build the index using the pd.MultiIndex constructor directly.

[rhshadrach]

Can you just test in the case of an empty frame here - nonempty is covered elsewhere. Indeed it is tricky to construct an empty MultiIndex of the expected type; this works:

df = DataFrame({"a": [1], "b": [2], "c": [3]}).set_index(['a', 'b', 'c'])
result = df.groupby(["a", "b", "c"]).sum().index[:0]
expected = df.index[:0]
tm.assert_index_equal(result, expected)

[jreback]

@gabrieldi95 can you replace the test with basically this example

[jreback]

can you merge master and update to comments

[gabrieldi95]

Yes, I was working on other stuff, will work on it now!

[gabrieldi95]

Hm I don't really know why the check fail, can someone help me?

[jreback]

can you merge master again and ping on green

[gabrieldi95]

@jreback can you help me with the error?

[jreback]

is worthwhile but needs to be passing (and pls merge master)

[rhshadrach]

@gabrieldi95 - The CI failure appears unrelated to me; can you try merging master again

[jreback]

@gabrieldi95 can you merge master and address comments

[rhshadrach]

Closing to take this off the queue. @gabrieldi95 please let me know if you'd like to continue this!

[gabrieldi95]

@rhshadrach sorry for taking so long! Yes, I would like to continue

[rhshadrach]

Can you rename to something like test_empty_multiindex

[jreback]

can you merge master

[jreback]

can you merge master and update to comments

[mroeschke]

Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in main and address the review and we can reopen.