-
-
Notifications
You must be signed in to change notification settings - Fork 18.5k
ENH: Add fast array equal function for indexers #50592
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Scrolling through issues, it looks like this is basically #32339. |
Commented on this in the initial post. Both are running around 700micro seconds if the whole array is checked (2 million entries). Ours is actually a bit faster, but not by much. Numpy is doing some things that are more expensive, e.g. allocating a Boolean array and then calling all on it. + Edit: Looks like NumPy is faster with less elements, but not by that much:
and with 2 Million:
Edit2: I'd try using this only for indexers in CoW in the beginning. The advantage is, that if an indexer is equal, we avoid a copy which brings a big speedup. And if it's not equal, we should find different values early on. |
@cython.wraparound(False) | ||
@cython.boundscheck(False) | ||
def array_equal_fast( | ||
ndarray[int6432_t, ndim=1] left, ndarray[int6432_t, ndim=1] right, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I check locally a little bit, and using a memoryview (int[:]
) instead of ndarray doesn't seem to give a significant speed up here (I seemed to remember that those could sometimes be faster)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me. Also for me locally it more or less gives the exact same timing as np.array_equal (for fully equal arrays)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. Nice speedup!
doc/source/whatsnew/vX.X.X.rst
file if fixing a bug or adding a new feature.The all equal case is as fast as numpy on 2 million entries, the early exist is significantly faster obviously