BUG: df.str.match(pattern) fails with pd.options.future.infer_string = True with re.compile()

[ErichMarx]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
pd.options.future.infer_string = True
df = pd.DataFrame(
    {
        "ID": {0: 1, 1: 2, 2: 3},
        "R_test1": {0: 1, 1: 2, 2: 3},
        "R_test2": {0: 1, 1: 3, 2: 5},
        "R_test3": {0: 2, 1: 4, 2: 6},
        "D": {0: 1, 1: 3, 2: 5},
    }
)

pd.wide_to_long(df, stubnames="R", i='ID', j="UNPIVOTED", sep="_", suffix=".*")

Issue Description

During the execution of the example, wide_to_long calls the func get_var_names (see below) in which a patten is created using re.compile. This pattern however is not accepted by df.columns.str.match(pattern) when pd.options.future.infer_string = True.
Setting pd.options.future.infer_string = False does solve the issue.

def get_var_names(df, stub: str, sep: str, suffix: str):
    regex = rf"^{re.escape(stub)}{re.escape(sep)}{suffix}$"
    pattern = re.compile(regex)
    return df.columns[df.columns.str.match(pattern)]

Apologies if this is not a pandas issue, since the pd.options.future.infer_string is causing it, I thought it would belong here.

Expected Behavior

import pandas as pd
pd.options.future.infer_string = False
df = pd.DataFrame(
{
"ID": {0: 1, 1: 2, 2: 3},
"R_test1": {0: 1, 1: 2, 2: 3},
"R_test2": {0: 1, 1: 3, 2: 5},
"R_test3": {0: 2, 1: 4, 2: 6},
"D": {0: 1, 1: 3, 2: 5},
}
)

pd.wide_to_long(df, stubnames="R", i='ID', j="UNPIVOTED", sep="_", suffix=".*")

Installed Versions

INSTALLED VERSIONS

commit : f538741
python : 3.11.6.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19045
machine : AMD64
processor : AMD64 Family 25 Model 80 Stepping 0, AuthenticAMD
byteorder : little
LC_ALL : None
LANG : None
LOCALE : English_United Kingdom.1252

pandas : 2.2.0
numpy : 1.26.3
pytz : 2023.3.post1
dateutil : 2.8.2
setuptools : 65.5.0
pip : 22.3
Cython : None
pytest : None
hypothesis : None
...
zstandard : None
tzdata : 2023.4
qtpy : None
pyqt5 : None

[rhshadrach]

Thanks for the report; I'm seeing the implementation prior to #55976 work on

pandas/pandas/core/reshape/melt.py

Line 462 in d928a5c

return df.columns[df.columns.str.match(pattern)]

with [col for col in df.columns if pattern.match(col)] instead.

cc @mroeschke