Skip to content

PERF: materialize less on slice in sparse __getitem__ #43777

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 7 commits into from
Sep 28, 2021
Merged

PERF: materialize less on slice in sparse __getitem__ #43777

merged 7 commits into from
Sep 28, 2021

Conversation

mzeitlin11
Copy link
Member

Could probably be pushed further to avoid densifying doe step != 1, but this seemed like a good starting point as the most common case

       before           after         ratio
     [21c29919]       [5622f636]
     <master>         <sparse_getitem_slice>
-        12.2±1ms         35.2±3μs     0.00  sparse.GetItem.time_slice

@mzeitlin11 mzeitlin11 added Indexing Related to indexing on series/frames, not to indexes themselves Performance Memory or execution speed performance Sparse Sparse Data Type labels Sep 28, 2021
@jreback jreback added this to the 1.4 milestone Sep 28, 2021
jreback
jreback approved these changes Sep 28, 2021
View reviewed changes
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm. @jbrockmendel if any comments.

jbrockmendel
jbrockmendel approved these changes Sep 28, 2021
View reviewed changes
Copy link
Member

@jbrockmendel jbrockmendel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jreback jreback merged commit 0844ca0 into pandas-dev:master Sep 28, 2021
@mzeitlin11 mzeitlin11 deleted the sparse_getitem_slice branch September 28, 2021 23:50
gasparitiago pushed a commit to gasparitiago/pandas that referenced this pull request Oct 9, 2021
@mzeitlin11 @gasparitiago
PERF: materialize less on slice in sparse __getitem__ (pandas-dev#43777)
d6d8e3b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jreback jreback jreback approved these changes

@jbrockmendel jbrockmendel jbrockmendel approved these changes

Assignees
No one assigned
Labels
Indexing Related to indexing on series/frames, not to indexes themselves Performance Memory or execution speed performance Sparse Sparse Data Type
Projects
None yet
Milestone
1.4
Development

Successfully merging this pull request may close these issues.
3 participants
@mzeitlin11 @jreback @jbrockmendel