Skip to content

PERF: groupby.pad/backfill follow-up #43518

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 12, 2021

Conversation

jbrockmendel
Copy link
Member

  • closes #xxxx
  • tests added / passed
  • Ensure all linting tests pass, see here for how to run them
  • whatsnew entry

Follow-ups include the big perf bump discussed in #43478

import pandas as pd
import numpy as np

np.random.seed(23446365)
arr = np.random.randn(10**5, 10)
mask = arr < -1
arr[mask] = np.nan

df = pd.DataFrame(arr)

gb = df.groupby(df.index % 7)

%timeit res = gb.pad()
29.8 ms ± 193 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)  # <- master
14.6 ms ± 2.41 ms per loop (mean ± std. dev. of 7 runs, 100 loops each)  # <- PR

@jreback jreback added Groupby Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Performance Memory or execution speed performance labels Sep 12, 2021
@jreback jreback added this to the 1.4 milestone Sep 12, 2021
@jreback
Copy link
Contributor

jreback commented Sep 12, 2021

prob should do some kind of whatsnew note for significant improvements like this (followon)

@jreback jreback merged commit ecff20e into pandas-dev:master Sep 12, 2021
@jbrockmendel jbrockmendel deleted the ref-gb-followup branch September 12, 2021 15:16
AlexeyGy pushed a commit to AlexeyGy/pandas that referenced this pull request Sep 13, 2021
AlexeyGy pushed a commit to AlexeyGy/pandas that referenced this pull request Sep 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Groupby Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Performance Memory or execution speed performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants