Skip to content

BUG: pd.api.extensions.take does not accept NumpyExtensionArray #59177

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
3 tasks done
andrewgsavage opened this issue Jul 3, 2024 · 4 comments · Fixed by #59181
Closed
3 tasks done

BUG: pd.api.extensions.take does not accept NumpyExtensionArray #59177

andrewgsavage opened this issue Jul 3, 2024 · 4 comments · Fixed by #59181
Assignees
Labels
Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Bug

Comments

@andrewgsavage
Copy link

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd 
import numpy as np

arr = pd.arrays.NumpyExtensionArray(np.array([1,2,3]))
pd.api.extensions.take(arr,[2])

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\mambaforge\envs\test_py300\Lib\site-packages\pandas\core\algorithms.py", line 1166, in take
    raise TypeError(
TypeError: pd.api.extensions.take requires a numpy.ndarray, ExtensionArray, Index, or Series, got NumpyExtensionArray.

Issue Description

I'd expect NumpyExtensionArray to be considered an ExtensionArray, so compatable with pd.api.extensions.take.

Expected Behavior

return

<NumpyExtensionArray>
[2]
Length: 1, dtype: int32

Installed Versions

Replace this line with the output of pd.show_versions()

@andrewgsavage andrewgsavage added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Jul 3, 2024
@Aloqeely
Copy link
Member

Aloqeely commented Jul 4, 2024

Thanks for the report! I'm surprised that NumpyExtensionArray fails the isinstance check. cc @jbrockmendel

On a separate matter, is there a specific reason you're using NumpyExtensionArray? I don't think there are many benefits to using it.

@mutricyl
Copy link
Contributor

mutricyl commented Jul 4, 2024

For the record this issue is linked to pint-pandas development and particularly hgrecco/pint-pandas#239.

New version of pandas.core.algorithms.take function introduced in #52981 is very restrictive and I wonder why NumpyExtensionArray is not in the list.

NumpyExtensionArray is used as pd.array is used in PintArray constructor. Depending on provided dtype pd.array will result on a ExtensionArray subclass or will fall back on NumpyExtensionArray.

For example:

>>> pd.array([1+1j,2,3]).dtype  
NumpyEADtype('complex128') # NumpyExtensionArray
>>> pd.array([1,2,3]).dtype    
Int64Dtype() # ExtensionArray

There is no build in pandas ExtensionArray for complex so pd.array falls back to NumpyExtensionArray

@rhshadrach
Copy link
Member

If the check is disabled, the take operation appears to work without issue. Agreed this should work - PRs to fix are welcome.

@rhshadrach rhshadrach added Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Jul 4, 2024
@mutricyl
Copy link
Contributor

mutricyl commented Jul 4, 2024

take

mutricyl pushed a commit to mutricyl/pandas that referenced this issue Jul 4, 2024
mroeschke pushed a commit that referenced this issue Jul 6, 2024
* update algo.take to solve #59177

* forgot to update TestExtensionTake::test_take_coerces_list

* fixing pandas/tests/dtypes/test_generic.py::TestABCClasses::test_abc_hierarchy

* ABCExtensionArray set formatting

---------

Co-authored-by: Laurent Mutricy <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Algos Non-arithmetic algos: value_counts, factorize, sorting, isin, clip, shift, diff Bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants