Skip to content

PERF: maybe_promote #51592

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 2, 2023
Merged

PERF: maybe_promote #51592

merged 5 commits into from
Mar 2, 2023

Conversation

jbrockmendel
Copy link
Member

Test runtime for pandas/tests/indexing/multiindex/test_indexing_slow.py decreases from 30s to 14.5s.

See discussion around #39692 (comment)

@mroeschke mroeschke added Performance Memory or execution speed performance Dtype Conversions Unexpected or buggy dtype conversions labels Feb 23, 2023
@@ -556,6 +557,13 @@ def ensure_dtype_can_hold_na(dtype: DtypeObj) -> DtypeObj:
return dtype


_canonical_nans = {
np.datetime64: np.datetime64("NaT", "ns"),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will it always be correct to assume "ns"?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in most contexts it wont matter, but we could probably cook up an example (maybe with object dtype) where it did

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated so as to retain identical fill_value in object-result cases

@jbrockmendel
Copy link
Member Author

our asv site appears to be done, but locally a lot of the regressions im seeing since 1.5.0 are for groupby.GroupByMethods.time_dtype_as_field('datetime', ...) and profiling points at maybe_promote so im guessing this will address a bunch of those

cc @phofl

@phofl phofl added this to the 2.0 milestone Mar 2, 2023
@phofl phofl merged commit 1151e3b into pandas-dev:main Mar 2, 2023
@phofl
Copy link
Member

phofl commented Mar 2, 2023

thx @jbrockmendel

meeseeksmachine pushed a commit to meeseeksmachine/pandas that referenced this pull request Mar 2, 2023
@jbrockmendel jbrockmendel deleted the perf-maybe_promote branch March 2, 2023 02:09
phofl pushed a commit that referenced this pull request Mar 2, 2023
Backport PR #51592: PERF: maybe_promote

Co-authored-by: jbrockmendel <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Dtype Conversions Unexpected or buggy dtype conversions Performance Memory or execution speed performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants