Skip to content

[ArrowStringArray] use pyarrow string trimming functions if available #41219

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Apr 29, 2021
Merged

[ArrowStringArray] use pyarrow string trimming functions if available #41219

merged 1 commit into from
Apr 29, 2021

Conversation

simonjayhawkins
Copy link
Member 

[ 27.78%] ··· strings.Methods.time_lstrip                                                                                                                 ok
[ 27.78%] ··· ============== ==========
                  dtype                
              -------------- ----------
                   str        22.7±0ms 
                  string      16.4±0ms 
               arrow_string   1.71±0ms 
              ============== ==========


[ 38.89%] ··· strings.Methods.time_rstrip                                                                                                                 ok
[ 38.89%] ··· ============== ==========
                  dtype                
              -------------- ----------
                   str        19.7±0ms 
                  string      13.9±0ms 
               arrow_string   1.71±0ms 
              ============== ==========


[ 43.06%] ··· strings.Methods.time_strip                                                                                                                  ok
[ 43.06%] ··· ============== ==========
                  dtype                
              -------------- ----------
                   str        18.9±0ms 
                  string      14.6±0ms 
               arrow_string   1.64±0ms 
              ============== ==========

@simonjayhawkins simonjayhawkins added Performance Memory or execution speed performance Strings String extension data type and string data labels Apr 29, 2021
@simonjayhawkins simonjayhawkins added this to the 1.3 milestone Apr 29, 2021
jreback
jreback reviewed Apr 29, 2021
View reviewed changes
pandas/core/arrays/string_arrow.py
@@ -831,3 +831,30 @@ def _str_lower(self):

def _str_upper(self):
return type(self)(pc.utf8_upper(self._data))

def _str_strip(self, to_strip=None):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ought to create an issue to remove the hasattr when we bump the required min pyarrow version

@jreback jreback merged commit 97fb40b into pandas-dev:master Apr 29, 2021
@simonjayhawkins simonjayhawkins deleted the String-trimming branch April 30, 2021 09:20
@simonjayhawkins simonjayhawkins mentioned this pull request Apr 30, 2021
Merged
yeshsurya pushed a commit to yeshsurya/pandas that referenced this pull request May 6, 2021
@simonjayhawkins @yeshsurya
[ArrowStringArray] use pyarrow string trimming functions if available (…
57052e5 
…pandas-dev#41219)
JulianWgs pushed a commit to JulianWgs/pandas that referenced this pull request Jul 3, 2021
@simonjayhawkins @JulianWgs
[ArrowStringArray] use pyarrow string trimming functions if available (…
61608b9 
…pandas-dev#41219)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jreback jreback jreback left review comments

Assignees
No one assigned
Labels
Performance Memory or execution speed performance Strings String extension data type and string data
Projects
None yet
Milestone
1.3
Development

Successfully merging this pull request may close these issues.
2 participants
@simonjayhawkins @jreback