Skip to content

REF: use WrappedCythonOp for GroupBy.std, sem #52053

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Mar 20, 2023

Conversation

jbrockmendel
Copy link
Member

  • closes #xxxx (Replace xxxx with the GitHub issue number)
  • Tests added and passed if fixing a bug or adding a new feature
  • All code checks passed.
  • Added type annotations to new arguments/methods/functions.
  • Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

Slight slowdown (but there should be a decreased memory footprint) for std, pretty nice speedup for sem:

import pandas as pd
import numpy as np

arr = np.random.randn(10**5, 3)
df = pd.DataFrame(arr)
gpr = np.arange(2).repeat(len(arr) // 2)
gb = df.groupby(gpr)

%timeit gb.std()
790 µs ± 55.9 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)  # <- main
803 µs ± 34.4 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)  # <- PR

%timeit gb.sem()
2.73 ms ± 150 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)    # <- main
796 µs ± 25.4 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)  # <- PR

@mroeschke mroeschke added this to the 2.1 milestone Mar 20, 2023
@mroeschke mroeschke added Refactor Internal refactoring of code Groupby Reduction Operations sum, mean, min, max, etc. labels Mar 20, 2023
@mroeschke mroeschke merged commit 33a3fb1 into pandas-dev:main Mar 20, 2023
@mroeschke
Copy link
Member

Thanks @jbrockmendel

@jbrockmendel jbrockmendel deleted the ref-gb-std branch March 20, 2023 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Groupby Reduction Operations sum, mean, min, max, etc. Refactor Internal refactoring of code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants