Skip to content

PERF: DataFrame.astype where dtype is an ExtensionDtype #54299

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 30, 2023

Conversation

lukemanley
Copy link
Member

       before           after         ratio
     [c6353c49]       [e84e1676]
     <main>           <perf-astype-ea-dtype>
-      37.0±0.9ms         24.4±2ms     0.66  frame_methods.AsType.time_astype(('float64', 'float64[pyarrow]'), False)
-        36.0±2ms         23.0±2ms     0.64  frame_methods.AsType.time_astype(('float64', 'Float64'), False)
-      37.9±0.4ms         23.7±2ms     0.63  frame_methods.AsType.time_astype(('int64[pyarrow]', 'float64[pyarrow]'), False)
-        32.4±1ms         19.3±2ms     0.59  frame_methods.AsType.time_astype(('int64[pyarrow]', 'float64[pyarrow]'), True)
-        33.0±1ms         19.2±2ms     0.58  frame_methods.AsType.time_astype(('float64', 'Float64'), True)
-      32.0±0.5ms       17.8±0.9ms     0.56  frame_methods.AsType.time_astype(('Int64', 'Float64'), False)
-        30.0±1ms         16.5±2ms     0.55  frame_methods.AsType.time_astype(('Int64', 'Float64'), True)
-        32.7±2ms         17.8±2ms     0.54  frame_methods.AsType.time_astype(('float64', 'float64[pyarrow]'), True)
-        24.4±2ms       1.31±0.3ms     0.05  frame_methods.AsType.time_astype(('Float64', 'Float64'), True)
-        24.2±2ms         973±50μs     0.04  frame_methods.AsType.time_astype(('float64[pyarrow]', 'float64[pyarrow]'), True)
-        21.0±1ms        500±100μs     0.02  frame_methods.AsType.time_astype(('Float64', 'Float64'), False)
-        20.9±2ms          378±5μs     0.02  frame_methods.AsType.time_astype(('float64[pyarrow]', 'float64[pyarrow]'), False)

@lukemanley lukemanley added Performance Memory or execution speed performance Dtype Conversions Unexpected or buggy dtype conversions labels Jul 29, 2023
@lukemanley lukemanley added this to the 2.1 milestone Jul 29, 2023
@phofl phofl merged commit 564b7d2 into pandas-dev:main Jul 30, 2023
@phofl
Copy link
Member

phofl commented Jul 30, 2023

thx @lukemanley

@lukemanley lukemanley deleted the perf-astype-ea-dtype branch August 1, 2023 21:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Dtype Conversions Unexpected or buggy dtype conversions Performance Memory or execution speed performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants