Skip to content

PERF: Improve performance with copy=True and different dtype #52054

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Mar 20, 2023
Merged

PERF: Improve performance with copy=True and different dtype #52054

merged 2 commits into from
Mar 20, 2023

Conversation

phofl
Copy link
Member

@phofl phofl commented Mar 17, 2023

  • closes #xxxx (Replace xxxx with the GitHub issue number)
  • Tests added and passed if fixing a bug or adding a new feature
  • All code checks passed.
  • Added type annotations to new arguments/methods/functions.
  • Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

This cuts runtime roughly in half when dtype is different than the dtype from values

# main
%timeit DataFrame(arr, dtype="int32", copy=True)
6.01 ms ± 37.9 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

# pr
%timeit DataFrame(arr, dtype="int32", copy=True)
3.54 ms ± 25.4 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

@phofl phofl added this to the 2.0 milestone Mar 18, 2023
@phofl phofl added Performance Memory or execution speed performance DataFrame DataFrame data structure labels Mar 18, 2023
@mroeschke mroeschke merged commit 7b93b06 into pandas-dev:main Mar 20, 2023
@mroeschke
Copy link
Member

Thanks @phofl

@meeseeksmachine meeseeksmachine mentioned this pull request Mar 20, 2023
Merged
meeseeksmachine pushed a commit to meeseeksmachine/pandas that referenced this pull request Mar 20, 2023
@phofl @meeseeksmachine
Backport PR pandas-dev#52054: PERF: Improve performance with copy=Tru…
a51c5a3 
…e and different dtype
mroeschke pushed a commit that referenced this pull request Mar 20, 2023
@meeseeksmachine @phofl
Backport PR #52054 on branch 2.0.x (PERF: Improve performance with co…
eccf9bd 
…py=True and different dtype) (#52088)

Backport PR #52054: PERF: Improve performance with copy=True and different dtype

Co-authored-by: Patrick Hoefler <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mroeschke mroeschke mroeschke approved these changes

Assignees
No one assigned
Labels
DataFrame DataFrame data structure Performance Memory or execution speed performance
Projects
None yet
Milestone
2.0
Development

Successfully merging this pull request may close these issues.
2 participants
@phofl @mroeschke