Skip to content

CLN: remove arrmap, closes #27251 #27530

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 23, 2019
Merged

Conversation

jbrockmendel
Copy link
Member

No description provided.

@jbrockmendel
Copy link
Member Author

jbrockmendel commented Jul 23, 2019

This removes the dependency of this module on non-cython parts of the code

@simonjayhawkins
Copy link
Member

#27251 (comment) was it not feasible to just use np.vectorize? is there is a perf impact?

@jbrockmendel
Copy link
Member Author

feasible to just use np.vectorize?

Probably. My understanding was that is just a fancy for-loop. Am I missing something?

is there is a perf impact?

Haven't measured. On the one hand we get rid of a call to lib.maybe_convert_objects (and a semi-expensive runtime import), and on the other we do a python-space list comprehension. My assumption is that this isn't called with particularly large arrays for the quantile arg.

One alternative would be to keep arrmap_float64 and specialize it given that we know the output will be float64. We'd still get to remove the maybe_convert_objects call, which is what I'm most excited about.

@simonjayhawkins
Copy link
Member

feasible to just use np.vectorize?

Probably. My understanding was that is just a fancy for-loop. Am I missing something?

so according the numpy docs, The vectorize function is provided primarily for convenience, not for performance. so I guess that using a list comprehension is a sensible approach.

Haven't measured. On the one hand we get rid of a call to lib.maybe_convert_objects (and a semi-expensive runtime import), and on the other we do a python-space list comprehension. My assumption is that this isn't called with particularly large arrays for the quantile arg.

One alternative would be to keep arrmap_float64 and specialize it given that we know the output will be float64. We'd still get to remove the maybe_convert_objects call, which is what I'm most excited about.

i'll leave that to you to decide if its worth further investigation.

@jreback jreback added the Clean label Jul 23, 2019
@jreback jreback added this to the 1.0 milestone Jul 23, 2019
@jreback
Copy link
Contributor

jreback commented Jul 23, 2019

@simonjayhawkins yeah we talked about this in a previous issue. even if this was a measureable slowdown, would still take it in reducing complexity.

@jreback jreback merged commit 49e6e53 into pandas-dev:master Jul 23, 2019
@jreback
Copy link
Contributor

jreback commented Jul 23, 2019

thanks @jbrockmendel

@jbrockmendel jbrockmendel deleted the arrmap branch July 23, 2019 21:39
quintusdias pushed a commit to quintusdias/pandas_dev that referenced this pull request Aug 16, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants