Skip to content

ENH/TST: Add quantile & mode tests for ArrowExtensionArray #47744

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Jul 28, 2022

Conversation

mroeschke
Copy link
Member

@mroeschke mroeschke commented Jul 15, 2022

@mroeschke mroeschke added Testing pandas testing functions or related to the test suite Arrow pyarrow functionality labels Jul 15, 2022
@mroeschke mroeschke added this to the 1.5 milestone Jul 15, 2022
msg = "Falling back on a non-pyarrow code path which may decrease performance."
msg = (
"Falling back on a non-pyarrow code path which may decrease performance or "
"not be fully compatible with pyarrow."
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a specific reason for adding this additional sentence?
(since it is so generic, it's also not that useful / potentially confusing I think)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also wanted to convey in this message that a user may get incorrect results if falling back.

e.g. A user's code could fall back to using ExtensionArray._mode/_quantile which may have some numpy-based assumptions that may fail when passing arrow (or arrow transformed) data.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we know (think?) that a certain fallback can give incorrect results, can we rather have it error in that case? (for that specific function)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah that's a good point. Okay I'll leave this message alone, and raise a NotImplementedError for the associated case (mode/quantile)

@jreback
Copy link
Contributor

jreback commented Jul 21, 2022

32-bit test failing?

@mroeschke
Copy link
Member Author

32-bit test failing?

Unrelated, but addressed at #47803

@jreback
Copy link
Contributor

jreback commented Jul 22, 2022

can u rebase

@mroeschke
Copy link
Member Author

Rebased

@jreback jreback merged commit ffc111c into pandas-dev:main Jul 28, 2022
@mroeschke mroeschke deleted the arrow/misc_methods branch July 28, 2022 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Arrow pyarrow functionality Testing pandas testing functions or related to the test suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants