-
-
Notifications
You must be signed in to change notification settings - Fork 18.4k
BUG: unwrap setitem indexer for ArrowExtensionArray #50085
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
BUG: unwrap setitem indexer for ArrowExtensionArray #50085
Conversation
pandas/core/internals/blocks.py
Outdated
@@ -1669,7 +1669,11 @@ def _unwrap_setitem_indexer(self, indexer): | |||
# Should never have length > 2. Caller is responsible for checking. | |||
# Length 1 is reached vis setitem_single_block and setitem_single_column | |||
# each of which pass indexer=(pi,) | |||
if len(indexer) == 2: | |||
if len(indexer) == 1: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure, but it might make more sense to handle this in setitem of the ArrowExtensionArray?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was 50/50 on whether to put it here or in ArrowExtensionArray.setitem. I ended up leaning towards this as it seems consistent with this method and may be helpful beyond ArrowExtensionArray (e.g. third-party EAs). But happy to move it if setitem is a better place for it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am more concerned with our setiteim for Arrow diverging from the numpy setitem call, but not sure if this is correct either.
cc @jbrockmendel Could you weigh in here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
id want to handle it correctly in the ArrowArray method. could add a test to the extension base tests for handling this case correctly
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, I will change this to be handled within ArrowExtensionArray.__setitem__
.
I guess the typing on ExtensionArray.__setitem__
and ArrowExtensionArray.__setitem__
should change as well?
pandas/pandas/core/arrays/base.py
Line 351 in 1488157
def __setitem__(self, key: int | slice | np.ndarray, value: Any) -> None: |
pandas/pandas/core/arrays/arrow/array.py
Line 878 in 1488157
def __setitem__(self, key: int | slice | np.ndarray, value: Any) -> None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah would make sense
Thanks @lukemanley |
doc/source/whatsnew/v2.0.0.rst
file if fixing a bug or adding a new feature.Setting values in a pyarrow-backed column with a non-scalar indexer was raising.
Fixes the following: