ENH: Allow ArrowDtype(pa.string()) to be compatable with str accessor #51207
Conversation
mroeschke
added
Strings
|
Looks like dask-related doctests are failing. do we need to pin a version somewhere? |
pandas/core/groupby/ops.py
Outdated
| @@ -405,7 +406,7 @@ def _reconstruct_ea_result( | |||
| """ | |||
| dtype: BaseMaskedDtype | StringDtype | |||
|
|
|||
| if isinstance(values.dtype, StringDtype): | |||
| if is_string_dtype(values.dtype): | |||
There was a problem hiding this comment.
doesnt make a difference here bc we know we have EA, but we should have a func like is_string_dtype that excludes object
pandas/core/groupby/ops.py
Outdated
| @@ -96,6 +96,7 @@ | |||
| ) | |||
|
|
|||
| if TYPE_CHECKING: | |||
| from pandas.core.arrays.string_ import StringDtype | |||
There was a problem hiding this comment.
the annotation that uses this may no longer be accurate?
There was a problem hiding this comment.
Good point. Added to | ArrowDtype to the annotation where this is used
|
I think the docbuild will fail until dask releases a new version. @jrbourbeau typically how often does dask release a new version? |
We release every two weeks. In fact, the latest release was pushed out a couple of hours ago |
|
Oh perfect looks like this PR picked up the dask update and the tests are passing now! |
|
Any other comments here? (Failures are unrelated) |
pandas/tests/extension/test_arrow.py
Outdated
| ) | ||
| def test_str_unsupported_methods(method, args): | ||
| ser = pd.Series(["abc", None], dtype=ArrowDtype(pa.string())) | ||
| with pytest.raises(NotImplementedError, match=f"str.{method} not supported."): |
There was a problem hiding this comment.
should the message be string[pyarrow]-specific?
There was a problem hiding this comment.
Sure, I noted it specifically with pd.ArrowDtype(pa.string())
| def _str_fullmatch( | ||
| self, pat, case: bool = True, flags: int = 0, na: Scalar | None = None | ||
| ): | ||
| if not pat.endswith("$") or pat.endswith("//$"): |
There was a problem hiding this comment.
i see this is already in the string_arrow code, but why "//$"?
There was a problem hiding this comment.
Not sure, as you mentioned I just copied it over from the string_arrow code
There was a problem hiding this comment.
OK. Best guess is it should be backslashes, meant to exclude escaped dollar signs. OK to consider out of scope.
|
Nice! thx |
Reboot of #50325
Notable change was that
pandas.core.strings.BaseStringArrayMethodsin the__init__.pywas causing a circular import when used inArrowExtensionArray, so needed to remove the import into__init__.py.This breaks one of our dask tests but it has been fixed downstream dask/dask#9907