Skip to content

PERF/ENH: np.apply_along_axis -> reduce in moments.py/nanops.py #3185

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jreback opened this issue Mar 27, 2013 · 1 comment
Closed

PERF/ENH: np.apply_along_axis -> reduce in moments.py/nanops.py #3185

jreback opened this issue Mar 27, 2013 · 1 comment
Labels
Groupby Performance Memory or execution speed performance Reshaping Concat, Merge/Join, Stack/Unstack, Explode

Comments

@jreback
Copy link
Contributor

jreback commented Mar 27, 2013

http://stackoverflow.com/questions/15652343/how-to-speed-up-pandas-rolling-sum
http://stackoverflow.com/questions/21058333/compute-rolling-maximum-drawdown-of-pandas-series

this numpy recipe might be of iterest to create a strided dimension:

numpy/numpy#31

prob easiest in these cases to create a algos.roll_generic_2d

./pandas/core/frame.py: (this is OK, just a fallback if reduce doesn't work)
 4109 :             result = np.apply_along_axis(func, axis, self.values)
./pandas/core/nanops.py:
  201 :         return np.apply_along_axis(get_median, axis, values)
  235 :             result = np.apply_along_axis(__builtin__.min, apply_ax, values)
  260 :             result = np.apply_along_axis(__builtin__.max, apply_ax, values)
./pandas/core/panel.py:
 1246 :         result = np.apply_along_axis(func, i, self.values)
./pandas/stats/moments.py:
  281 :         result = np.apply_along_axis(calc, axis, values)
  363 :     output = np.apply_along_axis(_ewma, 0, values)
  651 :     result = np.apply_along_axis(f, axis, values)
   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  1000003    8.930    0.000    8.930    0.000 {method 'put' of 'numpy.ndarray' objects}
     1000    8.413    0.008   23.225    0.023 shape_base.py:11(apply_along_axis)
  1000000    3.612    0.000    3.612    0.000 {pandas.algos.roll_sum}
  1000000    0.833    0.000    5.277    0.000 moments.py:277(<lambda>)
  1000000    0.628    0.000    4.444    0.000 moments.py:499(call_cython)
  2000000    0.551    0.000    0.551    0.000 {method 'tolist' of 'numpy.ndarray' objects}
        2    0.517    0.259    0.517    0.259 {pandas.algos.take_2d_axis0_float64_float64}
@jreback jreback modified the milestones: 0.15.0, 0.14.0 Feb 15, 2014
@jreback jreback modified the milestones: 0.16.0, 0.17.0 Jan 26, 2015
@jreback jreback added the Reshaping Concat, Merge/Join, Stack/Unstack, Explode label Feb 5, 2015
@wesm wesm added the Won't Fix label Sep 29, 2016
@wesm
Copy link
Member

wesm commented Sep 29, 2016

I'm putting this in the "won't fix" category as this will be addressed in low level code during the pandas 2.0 refactor

@wesm wesm closed this as completed Sep 29, 2016
@jorisvandenbossche jorisvandenbossche modified the milestones: Next Major Release, No action Sep 30, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Groupby Performance Memory or execution speed performance Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

No branches or pull requests

3 participants