Skip to content

BUG: Join did not work correctly when one MultiIndex had only one level #37208

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jan 3, 2021

Conversation

phofl
Copy link
Member

@phofl phofl commented Oct 17, 2020

droplevel for MultiIndex returns an Index when the resulting MultiIndex had only one level. But if the input had only one level and no level should be dropped, the return was a MultiIndex. This did not seem consistent, so I changed it, that in this case an Index would be returned too. If this is not the desired behavior, we could fix the join problems after calling this functions.

If this is not desired, we should add a note to the docstring, that the return for droplevel is a MultiIndex, if no level is dropped.

@phofl phofl added MultiIndex Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Oct 17, 2020
@github-actions
Copy link
Contributor

This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.

@github-actions github-actions bot added the Stale label Nov 17, 2020
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@phofl if you can merge master will have a look

@@ -474,6 +474,7 @@ Reshaping
- Bug in func :meth:`crosstab` when using multiple columns with ``margins=True`` and ``normalize=True`` (:issue:`35144`)
- Bug in :meth:`DataFrame.agg` with ``func={'name':<FUNC>}`` incorrectly raising ``TypeError`` when ``DataFrame.columns==['Name']`` (:issue:`36212`)
- Bug in :meth:`Series.transform` would give incorrect results or raise when the argument ``func`` was dictionary (:issue:`35811`)
- Bug in :func:`join` over :class:`MultiIndex` returned wrong result, when one of both indexes had only one level (:issue:`36909`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to 1.3

phofl added 2 commits January 2, 2021 00:42
� Conflicts:
�	doc/source/whatsnew/v1.2.0.rst
�	pandas/core/indexes/base.py
�	pandas/tests/indexes/multi/test_drop.py
�	pandas/tests/reshape/merge/test_join.py
@phofl
Copy link
Member Author

phofl commented Jan 1, 2021

Done

@@ -815,3 +815,20 @@ def test_join_cross(input_col, output_cols):
result = left.join(right, how="cross", lsuffix="_x", rsuffix="_y")
expected = DataFrame({output_cols[0]: [1, 1, 3, 3], output_cols[1]: [3, 4, 3, 4]})
tm.assert_frame_equal(result, expected)


@pytest.mark.parametrize("how", ["left", "right", "inner", "outer"])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't we have a fixture for this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, yes. thx

@@ -1672,7 +1672,7 @@ def _drop_level_numbers(self, levnums: List[int]):
Drop MultiIndex levels by level _number_, not name.
"""

if not levnums:
if not levnums and (len(self) > 1 or not isinstance(self, ABCMultiIndex)):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this len(self) > 1 is a very odd check, what is the intent?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to be no longer necessary. Code changed quite a bit since I implemented this. Removed it.

@jreback jreback added this to the 1.3 milestone Jan 3, 2021
@phofl
Copy link
Member Author

phofl commented Jan 3, 2021

@jreback green

@jreback jreback merged commit 8c4d7cd into pandas-dev:master Jan 3, 2021
@jreback
Copy link
Contributor

jreback commented Jan 3, 2021

thanks

@phofl phofl deleted the 36909 branch January 3, 2021 22:26
luckyvs1 pushed a commit to luckyvs1/pandas that referenced this pull request Jan 20, 2021
…el (pandas-dev#37208)

* BUG: join did not work correctly when one MultiIndex had only one level

* Change pr number

* Move whatsnew

* Remove pd

* Usef fixture

* Remove len self
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
MultiIndex Reshaping Concat, Merge/Join, Stack/Unstack, Explode Stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: Left join is broken with a MultiIndex with only one level
2 participants