Skip to content

DataFrame.__setitem__ with MultiIndex fails when expanding with new key #27248

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jottbele opened this issue Jul 5, 2019 · 2 comments
Open
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves MultiIndex setitem-with-expansion

Comments

@jottbele
Copy link

jottbele commented Jul 5, 2019

import pandas as pd
df = pd.DataFrame({'ID': [1, 2, 3, 4],
                   'DT': [2018, 2018, 2017, 2018],
                   'F1': [0, 1, 0, 0],
                   'F2': [0, 0, 1, 0]  })

df.loc[5]= [5, 2019, 1, 0] # this works
indexed= df.set_index(['ID', 'DT'], inplace=False)
indexed.loc[(2, 2018)] # this also works
indexed.loc[(1, 2019)]= [3, 4]  # this fails

Problem description

I know there is a workaround for this by adding , : but I feel the behavior is not intuitive as it is implemented. I think the arguments to getitem and setitem should be treated consistently and also an assignment over setitem should work the same way for simple indexes (with one level) and for multilevel indexes.

A formatted description of this can be viewed on stackoverflow here:
Stackoverflow description

Expected Output

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit: None python: 3.6.8.final.0 python-bits: 64 OS: Linux OS-release: 4.15.0-52-generic machine: x86_64 processor: x86_64 byteorder: little LC_ALL: None LANG: de_DE.UTF-8 LOCALE: de_DE.UTF-8 pandas: 0.24.1 pytest: 3.3.2 pip: 19.0.3 setuptools: 40.8.0 Cython: 0.29.6 numpy: 1.16.2 scipy: 1.2.1 pyarrow: None xarray: None IPython: 7.3.0 sphinx: 1.8.4 patsy: None dateutil: 2.8.0 pytz: 2018.9 blosc: None bottleneck: None tables: 3.4.4 numexpr: 2.6.9 feather: None matplotlib: 3.0.3 openpyxl: None xlrd: None xlwt: None xlsxwriter: None lxml.etree: 4.3.2 bs4: 4.7.1 html5lib: 1.0.1 sqlalchemy: None pymysql: None psycopg2: None jinja2: 2.10 s3fs: None fastparquet: None pandas_gbq: None pandas_datareader: None gcsfs: None
@TomAugspurger TomAugspurger added Indexing Related to indexing on series/frames, not to indexes themselves MultiIndex labels Jul 8, 2019
@TomAugspurger
Copy link
Contributor

Looks like a bug. Updated the title a bit (cc @toobaz didn't see any duplicates, but may have missed it).

@TomAugspurger TomAugspurger changed the title Different handling of __getitem__ and __setitem__ in locindexer DataFrame.__setitem__ with MultiIndex fails when expanding with new key Jul 8, 2019
@toobaz
Copy link
Member

toobaz commented Jul 8, 2019

Indeed, related to #16018 and maybe #19110 and #22247, but seems like a separate bug.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Indexing Related to indexing on series/frames, not to indexes themselves MultiIndex setitem-with-expansion
Projects
None yet
Development

No branches or pull requests

5 participants