Skip to content

REGR: window ewm slowdown #43052

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Aug 16, 2021
Merged

REGR: window ewm slowdown #43052

merged 4 commits into from
Aug 16, 2021

Conversation

mzeitlin11
Copy link
Member 

-     3.22±0.09ms       1.29±0.3ms     0.40  rolling.EWMMethods.time_ewm('DataFrame', 10, 'int', 'mean')
-      2.92±0.1ms       1.08±0.1ms     0.37  rolling.EWMMethods.time_ewm('DataFrame', 1000, 'int', 'mean')
-      2.68±0.1ms        959±100μs     0.36  rolling.EWMMethods.time_ewm('DataFrame', 1000, 'float', 'mean')
-      3.00±0.2ms      1.02±0.03ms     0.34  rolling.EWMMethods.time_ewm('DataFrame', 10, 'float', 'mean')
-      2.78±0.1ms        850±100μs     0.31  rolling.EWMMethods.time_ewm('Series', 10, 'int', 'mean')
-      2.34±0.2ms         683±30μs     0.29  rolling.EWMMethods.time_ewm('Series', 10, 'float', 'mean')
-     2.74±0.06ms         739±60μs     0.27  rolling.EWMMethods.time_ewm('Series', 1000, 'float', 'mean')
-      2.55±0.3ms         681±40μs     0.27  rolling.EWMMethods.time_ewm('Series', 1000, 'int', 'mean')

@mzeitlin11 mzeitlin11 added Performance Memory or execution speed performance Regression Functionality that used to work in a prior pandas version Window rolling, ewma, expanding labels Aug 15, 2021
@mzeitlin11 mzeitlin11 added this to the 1.3.3 milestone Aug 15, 2021
@jbrockmendel
Copy link
Member

nice speedup, cc @mroeschke

mroeschke
mroeschke approved these changes Aug 15, 2021
View reviewed changes
Copy link
Member

@mroeschke mroeschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! LGTM

jreback
jreback requested changes Aug 15, 2021
View reviewed changes
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep lgtm

doc/source/whatsnew/v1.3.2.rst Outdated
@@ -15,6 +15,7 @@ including other versions of pandas.
Fixed regressions
~~~~~~~~~~~~~~~~~
- Performance regression in :meth:`DataFrame.isin` and :meth:`Series.isin` for nullable data types (:issue:`42714`)
- Performance regression in :meth:`core.window.ewm.ExponentialMovingWindow.mean` (:issue:`42333`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move to 1.3.3

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

pandas/_libs/window/aggregations.pyx
@@ -1489,7 +1489,7 @@ def roll_weighted_var(const float64_t[:] values, const float64_t[:] weights,

def ewma(const float64_t[:] vals, const int64_t[:] start, const int64_t[:] end,
int minp, float64_t com, bint adjust, bint ignore_na,
const float64_t[:] deltas) -> np.ndarray:
const float64_t[:] deltas=None) -> np.ndarray:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can u update the doc string

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@jreback jreback merged commit 4d986c5 into pandas-dev:master Aug 16, 2021
@jreback
Copy link
Contributor

jreback commented Aug 16, 2021

thanks @mzeitlin11

@jreback
Copy link
Contributor

jreback commented Aug 16, 2021

@meeseeksdev backport 1.3.x

@meeseeksmachine meeseeksmachine mentioned this pull request Aug 16, 2021
Merged
meeseeksmachine pushed a commit to meeseeksmachine/pandas that referenced this pull request Aug 16, 2021
@mzeitlin11 @meeseeksmachine
Backport PR pandas-dev#43052: REGR: window ewm slowdown
60ed4bb
@lumberbot-app
Copy link

lumberbot-app bot commented Aug 16, 2021

Something went wrong ... Please have a look at my logs.

@mzeitlin11 mzeitlin11 deleted the regr_ewm branch August 16, 2021 21:01
simonjayhawkins pushed a commit that referenced this pull request Aug 17, 2021
@meeseeksmachine @mzeitlin11
Backport PR #43052: REGR: window ewm slowdown (#43069)
7f9074a 
Co-authored-by: Matthew Zeitlin <[email protected]>
feefladder pushed a commit to feefladder/pandas that referenced this pull request Sep 7, 2021
@mzeitlin11 @feefladder
REGR: window ewm slowdown (pandas-dev#43052)
e08d088
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jreback jreback jreback approved these changes

@mroeschke mroeschke mroeschke approved these changes

Assignees
No one assigned
Labels
Performance Memory or execution speed performance Regression Functionality that used to work in a prior pandas version Window rolling, ewma, expanding
Projects
None yet
Milestone
1.3.3
Development

Successfully merging this pull request may close these issues.

REGR: rolling.EWMMethods.time_ewm
4 participants
@mzeitlin11 @jbrockmendel @jreback @mroeschke