Skip to content

BUG: MultiIndex slicing with negative step #46156

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Mar 11, 2022

Conversation

lukemanley
Copy link
Member

MultiIndex label-based (.loc) slicing with a negative step size appears to contain a few bugs.

#38071 may have partially addressed negative step size issues, but a few issues seem to remain.

I believe this PR fixes the following:

Bug 1: slicing with non-null start/stop values returns an empty frame

    from pandas import DataFrame, IndexSlice, MultiIndex

    mi = MultiIndex.from_arrays([range(5)] * 2)
    df = DataFrame(1.0, index=mi, columns=['A'])

    target = IndexSlice[3:1:-1, :]
    df.loc[target, :]

Bug 2: negative step size other than -1 raises

    from pandas import DataFrame, IndexSlice, MultiIndex

    mi = MultiIndex.from_arrays([range(5)] * 2)
    df = DataFrame(1.0, index=mi, columns=['A'])

    target = IndexSlice[::-2, :]
    df.loc[target, :]

Bug 3: slicing a non-int labeled index level raises

    from pandas import DataFrame, IndexSlice, MultiIndex

    mi = MultiIndex.from_arrays([list('abcde')] * 2)
    df = DataFrame(1.0, index=mi, columns=['A'])

    target = IndexSlice['d':'b':-1, :]
    df.loc[target, :]

The tests have been expanded to cover these cases.

@lukemanley lukemanley changed the title Multiindex slice negative step BUG: MultiIndex slicing with negative step Feb 25, 2022
@lukemanley lukemanley added Bug Indexing Related to indexing on series/frames, not to indexes themselves MultiIndex and removed Bug labels Feb 26, 2022
@jreback jreback added this to the 1.5 milestone Feb 26, 2022
@jreback
Copy link
Contributor

jreback commented Feb 26, 2022

awesome. cc @phofl @jbrockmendel if any comments

@lukemanley
Copy link
Member Author

gentle ping, but no worries if you want to give it more time for review. have a follow-on after this one.

@jreback
Copy link
Contributor

jreback commented Mar 2, 2022

@lukemanley yeah just having @jbrockmendel and @phofl give it a quick look over

Copy link
Member

@phofl phofl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small comment, otherwise lgtm

@@ -393,6 +393,9 @@ Missing

MultiIndex
^^^^^^^^^^
- Bug in :meth:`DataFrame.loc` returning empty result when indexer contains a slice with a negative step size and non-null start/stop values (:issue:`46156`)
- Bug in :meth:`DataFrame.loc` raising when indexer contains a slice with a negative step size other than -1 (:issue:`46156`)
- Bug in :meth:`DataFrame.loc` raising when indexer contains a slice with a negative step size and slicing a non-int labeled index level (:issue:`46156`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you clarify that the df has a MultiIndex?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated, thanks

@jreback jreback merged commit 8561eb5 into pandas-dev:main Mar 11, 2022
@jreback
Copy link
Contributor

jreback commented Mar 11, 2022

thanks @lukemanley your other PR likely needs to be rebased

@lukemanley lukemanley deleted the multiindex-slice-negative-step branch March 20, 2022 23:18
yehoshuadimarsky pushed a commit to yehoshuadimarsky/pandas that referenced this pull request Jul 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves MultiIndex
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants