Skip to content

PERF: increase the minimum number of elements to use numexpr for ops from 1e4 to 1e6 #40609

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

jorisvandenbossche
Copy link
Member

Closes #40500, see that issue for the analysis of speedup vs size.

Another take of #40502, but hopefully now without crashing the tests because of increased memory use.

@jorisvandenbossche jorisvandenbossche added Performance Memory or execution speed performance Numeric Operations Arithmetic, Comparison, and Logical operations labels Mar 24, 2021
@jorisvandenbossche jorisvandenbossche added this to the 1.3 milestone Mar 24, 2021
@jorisvandenbossche
Copy link
Member Author

This time the tests are passing!

@jreback
Copy link
Contributor

jreback commented Mar 24, 2021

cc @rhshadrach i believe you saw memory issues with the previous PR, can you test this out?

@jorisvandenbossche
Copy link
Member Author

@jreback the memory issues were actually present on CI (we just ignored the failing tests when merging ;)), and now not anymore

@jreback jreback merged commit a20c169 into pandas-dev:master Mar 24, 2021
@jreback
Copy link
Contributor

jreback commented Mar 24, 2021

ok great

@jorisvandenbossche jorisvandenbossche deleted the numexpr-min-elements-take2 branch March 24, 2021 14:10
vladu pushed a commit to vladu/pandas that referenced this pull request Apr 5, 2021
JulianWgs pushed a commit to JulianWgs/pandas that referenced this pull request Jul 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Numeric Operations Arithmetic, Comparison, and Logical operations Performance Memory or execution speed performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Re-evaluate the minimum number of elements to use numexpr for elementwise ops
2 participants