Skip to content

[ArrowStringArray] use pyarrow.compute.match_substring_regex if available #41217

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
May 2, 2021
Merged

[ArrowStringArray] use pyarrow.compute.match_substring_regex if available #41217

merged 4 commits into from
May 2, 2021

Conversation

simonjayhawkins
Copy link
Member

follow-up to #41025

[ 50.00%] ··· strings.Contains.time_contains                                                                                                              ok
[ 50.00%] ··· ============== ========== ==========
              --                     regex        
              -------------- ---------------------
                  dtype         True      False   
              ============== ========== ==========
                   str        22.0±0ms   15.0±0ms 
                  string      17.2±0ms   10.9±0ms 
               arrow_string   6.84±0ms   2.37±0ms 
              ============== ========== ==========

@simonjayhawkins simonjayhawkins added Performance Memory or execution speed performance Strings String extension data type and string data labels Apr 29, 2021
@simonjayhawkins simonjayhawkins added this to the 1.3 milestone Apr 29, 2021
jorisvandenbossche
jorisvandenbossche reviewed Apr 30, 2021
View reviewed changes
pandas/core/arrays/string_arrow.py
return super()._str_contains(pat, case, flags, na, regex)

if regex:
if hasattr(pc, "match_substring_regex") and case:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it would be good to add a comment like "added in pyarrow x.x", so we can more easily clean up those hasattr checks if we later raise the minimum version

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes. I was thinking the same to address #41219 (comment)

@simonjayhawkins simonjayhawkins mentioned this pull request May 1, 2021
Merged
4 tasks
@jreback jreback merged commit 94f20fa into pandas-dev:master May 2, 2021
@simonjayhawkins simonjayhawkins deleted the match_substring_regex branch May 3, 2021 07:04
yeshsurya pushed a commit to yeshsurya/pandas that referenced this pull request May 6, 2021
@simonjayhawkins @yeshsurya
[ArrowStringArray] use pyarrow.compute.match_substring_regex if avail…
c4e3339 
…able (pandas-dev#41217)
@simonjayhawkins simonjayhawkins mentioned this pull request May 15, 2021
Merged
JulianWgs pushed a commit to JulianWgs/pandas that referenced this pull request Jul 3, 2021
@simonjayhawkins @JulianWgs
[ArrowStringArray] use pyarrow.compute.match_substring_regex if avail…
f30e571 
…able (pandas-dev#41217)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jorisvandenbossche jorisvandenbossche jorisvandenbossche approved these changes

Assignees
No one assigned
Labels
Performance Memory or execution speed performance Strings String extension data type and string data
Projects
None yet
Milestone
1.3
Development

Successfully merging this pull request may close these issues.
3 participants
@simonjayhawkins @jorisvandenbossche @jreback