BUG: .get_indexer_non_unique() must return an array of ints (#44084) #44404
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ev#44084) GH#44084 boils down to the following. According to the docs `.get_indexer_non_unique()` is supposed to return "integers from 0 to n - 1 indicating that the index at these positions matches the corresponding target values". However, for an index that is non unique and non monotonic it returns a boolean mask. That is because it uses `.get_loc()` which for non unique, non monotonic indexes returns a boolean mask. This patch catches that case and converts the boolean mask from `.get_loc()` into the corresponding array of integers if the index is not unique and not monotonic.
phofl
requested changes
Nov 12, 2021
| @@ -373,6 +373,16 @@ def test_get_indexer_with_nans(self): | |||
| expected = np.array([0, 1], dtype=np.intp) | |||
| tm.assert_numpy_array_equal(result, expected) | |||
|
|
|||
| def test_get_index_non_unique_non_monotonic(self): | |||
| # GH#44084 | |||
| index = IntervalIndex.from_tuples( | |||
There was a problem hiding this comment.
Could you add the MultiIndex cases
On Windows `np.array([1, 3])` is obviously int32 and thus the comparison to the int64 array fails due to dtype mismatch.
Sometimes the world out there is a bit more complicated than what you have on your cozy desktop :)
phofl
approved these changes
Nov 12, 2021
phofl
added
Indexing
|
thanks @johannes-mueller do we have sufficient testing on this for other index types aside from those explicityly tested here? if not would take a PR for that |
4 tasks
nickleus27
pushed a commit
to nickleus27/pandas
that referenced
this pull request
Nov 28, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
GH#44084 boils down to the following.
According to the docs
.get_indexer_non_unique()is supposed to return"integers from 0 to n - 1 indicating that the index at these positions matches
the corresponding target values". However, for an index that is non unique and
non monotonic it returns a boolean mask. That is because it uses
.get_loc()which for non unique, non monotonic indexes returns a boolean mask.
This patch catches that case and converts the boolean mask from
.get_loc()into the corresponding array of integers if the index is not unique and not
monotonic.