Skip to content

ENH: Implement interpolation for arrow and masked dtypes #56757

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Jan 10, 2024
Merged

ENH: Implement interpolation for arrow and masked dtypes #56757

merged 4 commits into from
Jan 10, 2024

Conversation

phofl
Copy link
Member

@phofl phofl commented Jan 7, 2024

@phofl phofl requested a review from mroeschke January 7, 2024 00:21
@phofl phofl added Enhancement Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate NA - MaskedArrays Related to pd.NA and nullable extension arrays Arrow pyarrow functionality labels Jan 7, 2024
mroeschke
mroeschke reviewed Jan 9, 2024
View reviewed changes
pandas/tests/frame/methods/test_interpolate.py

@pytest.mark.parametrize(
"dtype",
["int64", "uint64", "int32", "int16", "int8", "uint32", "uint16", "uint8"],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any_int_numpy_dtype

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are also passing in int, which screws with the logic below, I think handling this specifically causes more complexity than simply keeping the explicit list?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah okay. Sure this is fine for now then

mroeschke
mroeschke reviewed Jan 9, 2024
View reviewed changes
pandas/tests/frame/methods/test_interpolate.py Outdated
def test_interpolate_ea_raise(self):
@pytest.mark.parametrize(
"dtype",
["Int64", "UInt64", "Int32", "Int16", "Int8", "UInt32", "UInt16", "UInt8"],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any_int_ea_dtype

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx added

mroeschke
mroeschke reviewed Jan 9, 2024
View reviewed changes
pandas/core/arrays/arrow/array.py
if self.dtype.kind == "f":
data = self._pa_array.to_numpy()
elif self.dtype.kind in "iu":
data = self.to_numpy(dtype="f8", na_value=0.0)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To confirm, the na_value doesn't matter here because of the mask right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes correct

@mroeschke mroeschke added this to the 2.2 milestone Jan 10, 2024
@mroeschke mroeschke merged commit 5fc2ed2 into pandas-dev:main Jan 10, 2024
@mroeschke
Copy link
Member

Thanks @phofl

@lumberbot-app Lumberbot (App)
Copy link

lumberbot-app bot commented Jan 10, 2024

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

  1. Checkout backport branch and update it.
git checkout 2.2.x
git pull
  1. Cherry pick the first parent branch of the this PR on top of the older branch:
git cherry-pick -x -m1 5fc2ed2703a1370207f4ebad834e665b6c2ad42f
  1. You will likely have some merge/cherry-pick conflict here, fix them and commit:
git commit -am 'Backport PR #56757: ENH: Implement interpolation for arrow and masked dtypes'
  1. Push to a named branch:
git push YOURFORK 2.2.x:auto-backport-of-pr-56757-on-2.2.x
  1. Create a PR against branch 2.2.x, I would have named this PR:

"Backport PR #56757 on branch 2.2.x (ENH: Implement interpolation for arrow and masked dtypes)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

phofl added a commit to phofl/pandas that referenced this pull request Jan 10, 2024
@phofl
ENH: Implement interpolation for arrow and masked dtypes (pandas-dev#…
6a2642f 
…56757)

* ENH: Implement interpolation for arrow and masked dtypes

* Fixup

* Fix typing

* Update

(cherry picked from commit 5fc2ed2)
@phofl phofl mentioned this pull request Jan 10, 2024
Merged
lithomas1 pushed a commit that referenced this pull request Jan 10, 2024
@phofl
Backport PR #56757 on branch 2.2.x (ENH: Implement interpolation for …
8757a3c 
…arrow and masked dtypes) (#56809)

ENH: Implement interpolation for arrow and masked dtypes (#56757)

* ENH: Implement interpolation for arrow and masked dtypes

* Fixup

* Fix typing

* Update

(cherry picked from commit 5fc2ed2)
pmhatre1 pushed a commit to pmhatre1/pandas-pmhatre1 that referenced this pull request May 7, 2024
@phofl @pmhatre1
ENH: Implement interpolation for arrow and masked dtypes (pandas-dev#…
85d50e8 
…56757)

* ENH: Implement interpolation for arrow and masked dtypes

* Fixup

* Fix typing

* Update
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mroeschke mroeschke mroeschke approved these changes

Assignees
No one assigned
Labels
Arrow pyarrow functionality Enhancement Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate NA - MaskedArrays Related to pd.NA and nullable extension arrays
Projects
None yet
Milestone
2.2
Development

Successfully merging this pull request may close these issues.

.interpolate() Method Incompatible with float[pyarrow] Dtype
3 participants
@phofl @mroeschke @lithomas1