ENH: add string method remove prefix and suffix, python 3.9

[erfannariman]

Opening this issue for discussion:

Since python 3.9 is out, and we have the new string methods removeprefix and removesuffix, it would be nice to add them to the pandas string methods as well

In [2]: import pandas as pd

In [3]: df = pd.DataFrame({'A': ['str_string1', 'str_string2', 'str_string3']})

In [4]: print(df)
             A
0  str_string1
1  str_string2
2  str_string3

In [5]: df['A'].str.removeprefix('str_')
Out[5]: 
0    string1
1    string2
2    string3
Name: A, dtype: object

An argument not to add this is that it's pretty easily to achieve with str.split:

df['A'].str.split('_').str[-1]

[rhshadrach]

+1

@erfannariman Your alternative to removeprefix will not work on any input, e.g. str_str_string1 or other_str_string1.

[erfannariman]

+1

@erfannariman Your alternative to removeprefix will not work on any input, e.g. str_str_string1 or other_str_string1.

True, but you could solve that with: str.split('_', n=1).str[-1]

[ghost]

@erfannariman, your updated version won't work if the prefix is not separated by a delimiter.

e.g. for prefix str: strFollowedByOtherThings, strWithDifferentOtherThingsFollowing.

The most robust method is the one documented in PEP 616.

e.g.

prefix = 'str'
df2 = df.copy()
df2.loc[df['A'].str.startswith(prefix), 'A'] = df2.loc[df['A'].str.startswith(prefix), 'A'].str[len(prefix):]
df2

I think this makes a pretty good case that it would be nice to add the new string methods!

[erfannariman]

@erfannariman, your updated version won't work if the prefix is not separated by a delimiter.

e.g. for prefix str: strFollowedByOtherThings, strWithDifferentOtherThingsFollowing.

The most robust method is the one documented in PEP 616.

e.g.

prefix = 'str'
df2 = df.copy()
df2.loc[df['A'].str.startswith(prefix), 'A'] = df2.loc[df['A'].str.startswith(prefix), 'A'].str[len(prefix):]
df2

I think this makes a pretty good case that it would be nice to add the new string methods!

Yes, I agree.

[jreback]

+1 in adding this

[janosh]

@simonjayhawkins Would this be a good first issue? Happy to help in that case.

[simonjayhawkins]

Thanks @janosh. sure. go for it!

[janosh]

Great. Anything to be aware of? Is it okay to use removeprefix or should the implementation be backwards compatible, i.e. work for people on Python <3.9?

[simonjayhawkins]

From the comments here and the stalled PR #39226, I believe that we would want to use the implementation in PEP 616 to order to provide support for Python <3.9.

I think there is an outstanding question on whether we use the python native functions when available #39226 (comment).

[janosh]

Cool, I opened a draft PR at #43328. Let's discuss details there.