Skip to content

Speed up string inference in maybe_convert_objects #54498

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 12, 2023
Merged

Speed up string inference in maybe_convert_objects #54498

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
1aa9150
Speed up string inference in maybe_convert_objects
phofl Aug 11, 2023
81d9fb5
Speed up StringDtype arrow implementation
phofl Aug 12, 2023
4115867
Revert "Speed up StringDtype arrow implementation"
phofl Aug 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 5 additions & 6 deletions pandas/_libs/lib.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2680,14 +2680,13 @@ def maybe_convert_objects(ndarray[object] objects,
seen.object_ = True

elif seen.str_:
if is_string_array(objects, skipna=True):
if using_pyarrow_string_dtype():
import pyarrow as pa
if using_pyarrow_string_dtype() and is_string_array(objects, skipna=True):
import pyarrow as pa

from pandas.core.dtypes.dtypes import ArrowDtype
from pandas.core.dtypes.dtypes import ArrowDtype

dtype = ArrowDtype(pa.string())
return dtype.construct_array_type()._from_sequence(objects, dtype=dtype)
dtype = ArrowDtype(pa.string())
return dtype.construct_array_type()._from_sequence(objects, dtype=dtype)

seen.object_ = True
elif seen.interval_:
Expand Down