-
-
Notifications
You must be signed in to change notification settings - Fork 18.5k
ENH: Implement fastpath for masked indexes in join #50310
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
functions. | ||
""" | ||
if isinstance(self._values, BaseMaskedArray): | ||
# This is only used if our array is monotonic, so no NAs present |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where is the monoticity of self._values._data
checked?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Using those functions is guarded by those checks. The actual join implementations here only work with monotonic data. As soon as we reach _inner_indexer
and others we have to be monotonic
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah okay cool
Thanks @phofl |
doc/source/whatsnew/vX.X.X.rst
file if fixing a bug or adding a new feature.As of right now, masked indexes go through object even though we only use the join functions when we are monotonic, e.g. no NA present. We can simply use the numpy array to perform the join.
This is related to #49420, will make it a bit easier to review since it solves the join changes