Skip to content

BUG/PERF: MultiIndex.difference losing EA dtype #48722

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

lukemanley
Copy link
Member

@lukemanley lukemanley commented Sep 22, 2022

Maintains EA dtypes and improves perf:

       before           after         ratio
     [71fc89cd]       [19e9ae76]
     <main>           <mi-difference>
-      29.0±0.8ms       16.7±0.9ms     0.58  multiindex_object.SetOperations.time_operation('monotonic', 'string', 'difference')
-      30.4±0.4ms       15.7±0.1ms     0.52  multiindex_object.SetOperations.time_operation('non_monotonic', 'string', 'difference')
-      26.7±0.7ms       13.0±0.5ms     0.49  multiindex_object.SetOperations.time_operation('monotonic', 'ea_int', 'difference')
-      30.1±0.3ms       13.3±0.1ms     0.44  multiindex_object.SetOperations.time_operation('non_monotonic', 'ea_int', 'difference')
-      25.1±0.5ms       8.87±0.2ms     0.35  multiindex_object.SetOperations.time_operation('monotonic', 'int', 'difference')
-      27.7±0.3ms       9.58±0.3ms     0.35  multiindex_object.SetOperations.time_operation('monotonic', 'datetime', 'difference')
-      27.5±0.4ms       9.13±0.3ms     0.33  multiindex_object.SetOperations.time_operation('non_monotonic', 'int', 'difference')
-      33.4±0.2ms       10.4±0.4ms     0.31  multiindex_object.SetOperations.time_operation('non_monotonic', 'datetime', 'difference')

@lukemanley lukemanley added Performance Memory or execution speed performance MultiIndex ExtensionArray Extending pandas with custom dtypes or arrays. labels Sep 22, 2022
@phofl
Copy link
Member

phofl commented Sep 23, 2022

I’ve already got a pr about this, #48606

@lukemanley
Copy link
Member Author

Ah, sorry. I saw the other setop PRs merged and thought difference might have been missed. I’ll close this.

@lukemanley lukemanley closed this Sep 23, 2022
@lukemanley lukemanley deleted the mi-difference branch September 24, 2022 00:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ExtensionArray Extending pandas with custom dtypes or arrays. MultiIndex Performance Memory or execution speed performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants