Skip to content

REF: remove fast_apply #32086

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

jbrockmendel
Copy link
Member

xref #32083, one more coming that i think finishes off libreduction

@WillAyd
Copy link
Member

WillAyd commented Feb 19, 2020

Any perf differences here?

@mroeschke
Copy link
Member

Progress on #31845 hits this code path you are removing. Happy to remove - would just like to see this PR move forward sooner rather than later.

@jbrockmendel
Copy link
Member Author

Any perf differences here?

Ran full asv run overnight and am shocked to see no differences. Do we just not have asvs that hit this?

@jreback
Copy link
Contributor

jreback commented Feb 19, 2020

it’s possible we only have test that hit this - maybe have to construct one?

@jbrockmendel
Copy link
Member Author

Tentatively looks like I ran the asvs on the wrong pair of commits, re-running now.

@jbrockmendel
Copy link
Member Author

OK, a run with just -b groupby gives:

       before           after         ratio
     <master>         <no-fast-apply>
+     5.73±0.05ms          277±4ms    48.30  groupby.Apply.time_scalar_function_single_col
+      19.0±0.1ms          690±5ms    36.37  groupby.Apply.time_scalar_function_multi_col
-      13.2±0.1ms       11.7±0.4ms     0.89  groupby.Nth.time_series_nth('float64')
-         754±2μs          667±3μs     0.88  groupby.GroupByMethods.time_dtype_as_group('object', 'unique', 'direct')
-         755±3μs          666±1μs     0.88  groupby.GroupByMethods.time_dtype_as_group('object', 'unique', 'transformation')
-         813±5ms         702±20ms     0.86  groupby.Apply.time_copy_overhead_single_col
-      2.05±0.06s       1.75±0.02s     0.85  groupby.Apply.time_copy_function_multi_col

so that's pretty rough

@jbrockmendel jbrockmendel added the Performance Memory or execution speed performance label Feb 19, 2020
@mroeschke
Copy link
Member

Ouch. Guessing ripping it out is not justifiable given the performance hit.

@jbrockmendel
Copy link
Member Author

Ouch. Guessing ripping it out is not justifiable given the performance hit.

Yah. On the plus side, profiling it has turned up some optimization opportunities, locally ive got the worst one down from 48x to 10x.

@jbrockmendel
Copy link
Member Author

closing to clear the queue, will revisit after 1.0.2

@jbrockmendel jbrockmendel added the Mothballed Temporarily-closed PR the author plans to return to label Apr 21, 2020
@jbrockmendel jbrockmendel deleted the no-fast-apply branch August 27, 2020 20:17
@jbrockmendel jbrockmendel removed the Mothballed Temporarily-closed PR the author plans to return to label Aug 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Clean Groupby Performance Memory or execution speed performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants