Skip to content

BUG: MultiIndex.append now drops names if not the same #54303

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
2 of 3 tasks
attack68 opened this issue Jul 29, 2023 · 5 comments
Open
2 of 3 tasks

BUG: MultiIndex.append now drops names if not the same #54303

attack68 opened this issue Jul 29, 2023 · 5 comments
Labels
Index Related to the Index class or subclasses MultiIndex

Comments

@attack68
Copy link
Contributor

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

m1 = pd.MultiIndex.from_tuples([("a", 1)], names=["str", "int"])
m2 = pd.MultiIndex.from_tuples([("b", 2)])

m1.append(m2)

Issue Description

On previous pandas versions the names arg from m1 was maintained when m2 was appended that had no names.

On 2.0.3 the names are dropped from the return. Caught this in tests of a package upgrade.

Is this a bug or was this intentional - I can see arguments for both cases?

Expected Behavior

?

Installed Versions

Replace this line with the output of pd.show_versions()

@attack68 attack68 added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Jul 29, 2023
@lukemanley
Copy link
Member

I think this was introduced in #53697. This wasn't intentional, but it is now consistent with a regular index:

In [1]: import pandas as pd

In [2]: idx1 = pd.Index([1], name="foo")

In [3]: idx2 = pd.Index([2])

In [4]: idx1, idx2
Out[4]: (Index([1], dtype='int64', name='foo'), Index([2], dtype='int64'))

In [5]: idx1.append(idx2)
Out[5]: Index([1, 2], dtype='int64')

I'd be in favor of keeping Index and MultiIndex consistent and would probably lean towards the new behavior (but can see an argument for keeping the names)

@lukemanley lukemanley added MultiIndex and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Jul 29, 2023
@phofl
Copy link
Member

phofl commented Jul 30, 2023

Agree with @lukemanley

@phofl phofl added Closing Candidate May be closeable, needs more eyeballs and removed Bug labels Jul 30, 2023
@attack68
Copy link
Contributor Author

I agree both Index types should be consistent.

Is it the right behaviour that when an index which has no meta data is appended to an index with some meta data that that meta data gets destroyed?

In general, I consider an append as an operation which usually adds a smaller, possibly singular item, to a larger, more significant object in memory.

Im typing on a phone now and cant see what happens if we append an index with meta data to one without, or the outcome when both have meta data that conflict with other.

Personally I would prefer that meta data is preserved where possible and would raise on conflict.

@phofl
Copy link
Member

phofl commented Jul 31, 2023

append was the odd one out, all other methods already behaved like append now (union, intersection, ...).

Names are generally only propagated if they match.

Having None as names is not equivalent to no meta information, users can set names to None. This could be implemented properly through using no_default as the default, but I am personally not interested enough.

@attack68
Copy link
Contributor Author

Having None as names is not equivalent to no meta information, users can set names to None. This could be implemented properly through using no_default as the default, but I am personally not interested enough.

Fair enough, I can see this as being a bit of nuisance to implement and maintain.

Prefer to leave this issue open if anyone else likes to add user perspectives.

@attack68 attack68 added Index Related to the Index class or subclasses and removed Closing Candidate May be closeable, needs more eyeballs labels Jul 31, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Index Related to the Index class or subclasses MultiIndex
Projects
None yet
Development

No branches or pull requests

3 participants