PERF: DataFrame.all(axis="columns") orders of magnitude slower for bool[pyarrow]

[randolf-scholz]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this issue exists on the latest version of pandas.

• I have confirmed this issue exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import numpy as np
from functools import reduce
import operator

data = np.random.randn(10_000, 10) > 0.5  # larger won't even finish
df_numpy = pd.DataFrame(data, dtype=bool)
df_arrow = df_numpy.astype("bool[pyarrow]")

%timeit df_numpy.all(axis="index")     # 208 µs ± 813 µs
%timeit df_numpy.all(axis="columns")   # 294 µs ± 3.4 µs

%timeit df_arrow.all(axis="index")     # 987 µs ± 3.93 µs
%timeit df_arrow.all(axis="columns")   # 812 ms ± 7.5 ms  (!!!)

%timeit reduce(operator.__and__, (s for _, s in df_arrow.items()))  # 421 µs ± 7.65 µs

Larger examples won't even run, so presumably it falls back to some pure python implementation.

Installed Versions

INSTALLED VERSIONS
------------------
commit           : 232d84e9a5add831c753e67f1c88863442dc3c92
python           : 3.10.11.final.0
python-bits      : 64
OS               : Linux
OS-release       : 5.19.0-50-generic
Version          : #50-Ubuntu SMP PREEMPT_DYNAMIC Mon Jul 10 18:24:29 UTC 2023
machine          : x86_64
processor        : x86_64
byteorder        : little
LC_ALL           : None
LANG             : en_US.UTF-8
LOCALE           : en_US.UTF-8

pandas           : 2.1.0.dev0+1375.g232d84e9a5
numpy            : 1.24.4
pytz             : 2023.3
dateutil         : 2.8.2
setuptools       : 65.5.0
pip              : 23.2.1
Cython           : 0.29.33
pytest           : 7.4.0
hypothesis       : 6.82.0
sphinx           : 6.2.1
blosc            : 1.11.1
feather          : None
xlsxwriter       : 3.1.2
lxml.etree       : 4.9.3
html5lib         : 1.1
pymysql          : 1.4.6
psycopg2         : 2.9.6
jinja2           : 3.1.2
IPython          : 8.14.0
pandas_datareader: None
bs4              : 4.12.2
bottleneck       : 1.3.7
brotli           : 
fastparquet      : 2023.7.0
fsspec           : 2023.6.0
gcsfs            : 2023.6.0
matplotlib       : 3.7.2
numba            : 0.57.1
numexpr          : 2.8.4
odfpy            : None
openpyxl         : 3.1.2
pandas_gbq       : None
pyarrow          : 12.0.1
pyreadstat       : 1.2.2
pyxlsb           : 1.0.10
s3fs             : 2023.6.0
scipy            : 1.11.1
snappy           : 
sqlalchemy       : 2.0.19
tables           : 3.8.0
tabulate         : 0.9.0
xarray           : 2023.7.0
xlrd             : 2.0.1
zstandard        : 0.21.0
tzdata           : 2023.3
qtpy             : None
pyqt5            : None

Prior Performance

No response

[lukemanley]

xref #52016 which reports the same issue for BooleanDtype