MultiIndex.from_product could accept MultiIndex

[ghislainp]

Is your feature request related to a problem?

The problem is in xarray.DataArray.to_dataframe() which uses MultiIndex.from_product() to build the index based on the array coordinates.
When an xarray has a MultiIndex as one of the coordinates, from_product is called with a Multiindex as one of the iterable. This raises an exception, as discussed here pydata/xarray#3008

Describe the solution you'd like

I've implemented a solution in xarray [PR ##4442], this close the issue for xarray.
However, the solution could be ported to from_product to allow this method to accept MultiIndex. I'm not sure it is of great interest for the users, but at least the solution exists and is relatively simple.

API breaking implications

none breaking

Describe alternatives you've considered

none

[jreback]

can u show an example

[FRidh]

One of the symptoms of this issue is #34019, which could be fixed with #38558, however, that leads to

ValueError: Names should be list-like for a MultiIndex

--------------------------------------------------------------------------- ValueError Traceback (most recent call last) in ----> 1 xr.DataArray(np.ones([3, 3, 3]), dims=["x", "y", "z"], name="foo").stack(a=("x", "y")).to_dataframe()

/nix/store/db2v0kg0x1gsxc34g5qn6s8hzy1v9wrz-python3-3.8.6-env/lib/python3.8/site-packages/xarray/core/dataarray.py in to_dataframe(self, name, dim_order)
2457 ordered_dims = ds._normalize_dim_order(dim_order=dim_order)
2458
-> 2459 df = ds._to_dataframe(ordered_dims)
2460 df.columns = [name if c == unique_name else c for c in df.columns]
2461 return df

/nix/store/db2v0kg0x1gsxc34g5qn6s8hzy1v9wrz-python3-3.8.6-env/lib/python3.8/site-packages/xarray/core/dataset.py in _to_dataframe(self, ordered_dims)
4587 for k in columns
4588 ]
-> 4589 index = self.coords.to_index([*ordered_dims])
4590 return pd.DataFrame(dict(zip(columns, data)), index=index)
4591

/nix/store/db2v0kg0x1gsxc34g5qn6s8hzy1v9wrz-python3-3.8.6-env/lib/python3.8/site-packages/xarray/core/coordinates.py in to_index(self, ordered_dims)
109 indexes = [self._data.get_index(k) for k in ordered_dims] # type: ignore
110 names = list(ordered_dims)
--> 111 return pd.MultiIndex.from_product(indexes, names=names)
112
113 def update(self, other: Mapping[Hashable, Any]) -> None:

/nix/store/db2v0kg0x1gsxc34g5qn6s8hzy1v9wrz-python3-3.8.6-env/lib/python3.8/site-packages/pandas/core/indexes/multi.py in from_product(cls, iterables, sortorder, names)
564 # codes are all ndarrays, so cartesian_product is lossless
565 codes = cartesian_product(codes)
--> 566 return MultiIndex(levels, codes, sortorder=sortorder, names=names)
567
568 @classmethod

/nix/store/db2v0kg0x1gsxc34g5qn6s8hzy1v9wrz-python3-3.8.6-env/lib/python3.8/site-packages/pandas/core/indexes/multi.py in new(cls, levels, codes, sortorder, names, dtype, copy, name, verify_integrity, _set_identity)
290
291 if verify_integrity:
--> 292 new_codes = result._verify_integrity()
293 result._codes = new_codes
294

/nix/store/db2v0kg0x1gsxc34g5qn6s8hzy1v9wrz-python3-3.8.6-env/lib/python3.8/site-packages/pandas/core/indexes/multi.py in _verify_integrity(self, codes, levels)
344 # nlevels matches nor that sortorder matches actually sortorder.
345 codes = codes or self.codes
--> 346 levels = levels or self.levels
347
348 if len(levels) != len(codes):

pandas/_libs/properties.pyx in pandas._libs.properties.CachedProperty.get()

/nix/store/db2v0kg0x1gsxc34g5qn6s8hzy1v9wrz-python3-3.8.6-env/lib/python3.8/site-packages/pandas/core/indexes/multi.py in levels(self)
695 # create new IndexEngine
696 # #31648
--> 697 result = [
698 x._shallow_copy(name=name) for x, name in zip(self._levels, self._names)
699 ]

/nix/store/db2v0kg0x1gsxc34g5qn6s8hzy1v9wrz-python3-3.8.6-env/lib/python3.8/site-packages/pandas/core/indexes/multi.py in (.0)
696 # #31648
697 result = [
--> 698 x._shallow_copy(name=name) for x, name in zip(self._levels, self._names)
699 ]
700 for level in result:

/nix/store/db2v0kg0x1gsxc34g5qn6s8hzy1v9wrz-python3-3.8.6-env/lib/python3.8/site-packages/pandas/core/indexes/multi.py in _shallow_copy(self, values, name, levels, codes, dtype, sortorder, names, _set_identity)
1031 codes = codes if codes is not None else self.codes
1032
-> 1033 result = MultiIndex(
1034 levels=levels,
1035 codes=codes,

/nix/store/db2v0kg0x1gsxc34g5qn6s8hzy1v9wrz-python3-3.8.6-env/lib/python3.8/site-packages/pandas/core/indexes/multi.py in new(cls, levels, codes, sortorder, names, dtype, copy, name, verify_integrity, _set_identity)
282 if names is not None:
283 # handles name validation
--> 284 result._set_names(names)
285
286 if sortorder is not None:

/nix/store/db2v0kg0x1gsxc34g5qn6s8hzy1v9wrz-python3-3.8.6-env/lib/python3.8/site-packages/pandas/core/indexes/multi.py in _set_names(self, names, level, validate)
1340 # Don't allow a single string for names in a MultiIndex
1341 if names is not None and not is_list_like(names):
-> 1342 raise ValueError("Names should be list-like for a MultiIndex")
1343 names = list(names)
1344

ValueError: Names should be list-like for a MultiIndex

[mroeschke]

Thanks for the issue, but it appears this hasn't gotten traction in a while so closing