fix: use fastpath for PyCapsule export when starting from pyarrow-backed Series, respect requested_schema #59683
Conversation
…ked Series, respect requested_schema
MarcoGorelli
changed the title
|
@jorisvandenbossche @WillAyd fancy taking a look? |
pandas/core/series.py
Outdated
| if isinstance(self.dtype, ArrowDtype): | ||
| # fastpath! | ||
| ca = self.values._pa_array | ||
| if type is not None: | ||
| ca = ca.cast(type) | ||
| else: | ||
| ca = pa.chunked_array([pa.Array.from_pandas(self, type=type)]) |
There was a problem hiding this comment.
This does not yet handle StringDtype (which is not an ArrowDtype).
Alternative option is something like (untested):
ca = pa.array(self, type=type)
if not isinstance(ca, pa.ChunkedArray):
ca = pa.chunked_array([ca])
Note: 1) it's a bit strange, but pa.array() can return a chunked array (in this case, it will check the __arrow_array__ attribute on the passed object, and for our dtypes backed by a chunked array, that method will return the chunked array, and this is directly passed through as result of pa.array()), and 2) pa.array() is essentially just the same as pa.Array.from_pandas but a bit simpler (and because we rely on it returning potentially a chunked array, it would read even more strangely ..)
There was a problem hiding this comment.
that seems to work, thanks @jorisvandenbossche !
|
Thanks for the follow up @MarcoGorelli |
…ked Series, respect requested_schema (pandas-dev#59683) * fix: use fastpath for PyCapsule export when starting from pyarrow-backed Series, respect requested_schema * simplify * stringdtype test
Follow-up to https://github.com/pandas-dev/pandas/pull/59587/files (so, I haven't added a new whatsnew)
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.