Skip to content

PERF: Rolling.cov/corr #39388

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 25, 2021
Merged

Conversation

mroeschke
Copy link
Member

  • tests added / passed
  • Ensure all linting tests pass, see here for how to run them
  • whatsnew entry

Will attempt to improve the performance of groupby.rolling next.

       before           after         ratio
     [37b5800a]       [ee9a56c1]
     <master>         <ref/rolling_cov_corr>
-        10.4±2ms       8.90±0.2ms     0.86  rolling.Pairwise.time_pairwise(10, 'cov', True)
-        10.2±1ms       8.52±0.1ms     0.83  rolling.Pairwise.time_pairwise(None, 'cov', True)
-      2.48±0.1ms      1.07±0.01ms     0.43  rolling.Pairwise.time_pairwise(1000, 'cov', False)
-      2.55±0.2ms      1.08±0.02ms     0.42  rolling.Pairwise.time_pairwise(10, 'cov', False)
-     2.33±0.04ms          918±6μs     0.39  rolling.Pairwise.time_pairwise(None, 'cov', False)
-      4.02±0.6ms      1.46±0.02ms     0.36  rolling.Pairwise.time_pairwise(10, 'corr', False)
-      4.08±0.4ms       1.42±0.1ms     0.35  rolling.Pairwise.time_pairwise(1000, 'corr', False)
-     3.69±0.07ms      1.17±0.03ms     0.32  rolling.Pairwise.time_pairwise(None, 'corr', False)

SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY.
PERFORMANCE INCREASED.

Added a ddof argument to corr which was implicitly accepted (and used) by kwargs before; making it an explicit kwargs in this PR. Will document it after #39219 is in.

@mroeschke mroeschke added Performance Memory or execution speed performance Window rolling, ewma, expanding labels Jan 25, 2021
@mroeschke mroeschke added this to the 1.3 milestone Jan 25, 2021
@jreback jreback merged commit b471c29 into pandas-dev:master Jan 25, 2021
@jreback
Copy link
Contributor

jreback commented Jan 25, 2021

thanks @mroeschke

@mroeschke mroeschke deleted the ref/rolling_cov_corr branch January 25, 2021 17:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Performance Memory or execution speed performance Window rolling, ewma, expanding
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants