BUG: pd.DataFrame.to_hdf does not support multiindex frames with string[pyarrow_numpy]

[nvasilevv]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import numpy as np
data = pd.DataFrame(
    np.random.randint(0, 50, size=(15, 4)), columns=list('ABCD')
).set_index(['A', 'B'])

# This action will be successful as it works on numpy objects
data.to_hdf('test.hdf', key='test', format='table')

# If we force the usage of pyarrow string, the same will fail

pd.options.future.infer_string = True

data_arrow = pd.DataFrame(
    np.random.randint(0, 50, size=(15, 4)), columns=list('ABCD')
).set_index(['A', 'B'])

data_arrow.to_hdf('test_arrow.hdf', key='test', format='table') # This will raise ValueError("cannot have non-object label DataIndexableCol")

Issue Description

Looks like using string[pyarrow_numpy] as dtype and to_hdf with a multiindex frame throws a value error. The error comes from pytables.py and more specifically the check below.

pandas/pandas/io/pytables.py

Lines 2647 to 2650 in 723feb9

	def validate_names(self) -> None:
	if not is_object_dtype(Index(self.values).dtype):
	# TODO: should the message here be more specifically non-str?
	raise ValueError("cannot have non-object label DataIndexableCol")

The is_object_dtype always checks against np.object_.

pandas/pandas/core/dtypes/common.py

Lines 137 to 165 in 723feb9

	def is_object_dtype(arr_or_dtype) -> bool:
	"""
	Check whether an array-like or dtype is of the object dtype.
	Parameters
	----------
	arr_or_dtype : array-like or dtype
	The array-like or dtype to check.
	Returns
	-------
	boolean
	Whether or not the array-like or dtype is of the object dtype.
	Examples
	--------
	>>> from pandas.api.types import is_object_dtype
	>>> is_object_dtype(object)
	True
	>>> is_object_dtype(int)
	False
	>>> is_object_dtype(np.array([], dtype=object))
	True
	>>> is_object_dtype(np.array([], dtype=int))
	False
	>>> is_object_dtype([1, 2, 3])
	False
	"""
	return _is_dtype_type(arr_or_dtype, classes(np.object_))

Expected Behavior

I would expect that is_object_dtype allows for string[pyrrow_numpy] (if and only if downstream behavior is supported by that dtype). If that action is not supported with string[pyarrow_numpy] I would expected a more informative error.

Installed Versions

INSTALLED VERSIONS

commit : 723feb9
python : 3.9.13.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19044
machine : AMD64
processor : Intel64 Family 6 Model 165 Stepping 2, GenuineIntel
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : English_United States.1252

pandas : 2.2.0dev0+208.g723feb984e
numpy : 1.22.4
pytz : 2022.7.1
dateutil : 2.8.2
setuptools : 68.0.0
pip : 23.2.1
Cython : 0.29.36
pytest : 6.2.5
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.1.2
IPython : None
pandas_datareader : None
bs4 : None
bottleneck : None
dataframe-api-compat: None
fastparquet : None
fsspec : None
gcsfs : None
matplotlib : None
numba : 0.56.4
numexpr : 2.8.5
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 10.0.1
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.10.1
sqlalchemy : None
tables : 3.7.0
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2023.3
qtpy : None
pyqt5 : None

[phofl]

Thx for the report. Your assumption is correct, this should work

[nvasilevv]

I think that I could try to propose a PR fixing this. It would be my first one in this project, so if you think that the topic is too hard to approach, I am fine with waiting for the solution.

[phofl]

I'd appreciate the help if you have time. The first thing I would do is figuring out where "pyarrow[string]" is treated differently than the new dtype.

[nvasilevv]

After undertaking a short investigation, I believe that the different treatment in that case comes from here

https://github.com/pandas-dev/pandas/blob/705d4312cf1d94ef2497bcd8091e0eabd1085f4a/pandas/_libs/lib.pyx#L2700C12-L2706

Essentially, this part always uses pyarrow_numpy as a string type if pd.options.future.infer_string = True. However, if that is not set and instead pd.options.mode.string_storage = 'pyarrow' nothing changes.

I see that this part was changed here - 1ecfa67 and was released in v2.1.0. Previously, the array was not casted at this point.