Skip to content

Regression: concatenating MultiIndex with empty RangeIndex raises #41234

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 of 3 tasks
mlondschien opened this issue Apr 30, 2021 · 1 comment · Fixed by #41275
Closed
2 of 3 tasks

Regression: concatenating MultiIndex with empty RangeIndex raises #41234

mlondschien opened this issue Apr 30, 2021 · 1 comment · Fixed by #41275
Labels
Index Related to the Index class or subclasses Regression Functionality that used to work in a prior pandas version
Milestone

Comments

@mlondschien
Copy link
Contributor

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


As also reported here: dask/dask#7610, PR #38671 introduced the following behaviour:

A reproducer with just pandas:

df1 = pd.DataFrame([[1, 2]], columns=pd.MultiIndex.from_tuples([('B', 1), ('C', 1)]))
df2 = pd.DataFrame(index=[0], columns=pd.RangeIndex(0))

pd.concat([df1, df2])

What triggers the error here is the columns=pd.RangeIndex(0) for the empty df2 (by default pandas creates a zero-length object dtype Index, which works fine, but if it's an empty RangeIndex, we now get this error).

This is a regression in itself in pandas. But I am also wondering a bit where the empty RangeIndex is coming from (it seems that the groupby operation in dask results in some empty partitions)

Originally posted by @jorisvandenbossche in dask/dask#7610 (comment)

IIUC the fix for this could possibly be as simple as replacing

if isinstance(self, ABCMultiIndex) and not is_object_dtype(
unpack_nested_dtype(other)
):

with

            if isinstance(self, ABCMultiIndex) and not is_object_dtype(
                unpack_nested_dtype(other)
            ) and len(other) > 0:
@mlondschien mlondschien added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Apr 30, 2021
@jorisvandenbossche jorisvandenbossche added Index Related to the Index class or subclasses Regression Functionality that used to work in a prior pandas version and removed Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Apr 30, 2021
@jorisvandenbossche jorisvandenbossche added this to the 1.3 milestone Apr 30, 2021
@jorisvandenbossche
Copy link
Member

cc @jbrockmendel

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Index Related to the Index class or subclasses Regression Functionality that used to work in a prior pandas version
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants