Skip to content

MultiIndex-Columns Integer and String: Column level 1 not accessable after a using append multiple times #29699

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Alrik-star opened this issue Nov 18, 2019 · 3 comments · Fixed by #41712
Assignees
Labels
good first issue Needs Tests Unit test(s) needed to prevent regressions
Milestone

Comments

@Alrik-star
Copy link

Alrik-star commented Nov 18, 2019

Code Sample, a copy-pastable example if possible

import pandas as pd
pd.show_versions()

# Create Multi-Index
df = pd.DataFrame({'col1':[1,2,3], 'col2':[11,12,13]})
df = pd.concat([df], keys=['multi'], names=['level1'], axis=1)


for i in range(100):
    df[i, 'colA'] = 10
    a = pd.DataFrame({'col1':[1,2,3], 'col2':[11,12,13]})
    a = pd.concat([a], keys=['multi'], names=['level1'], axis=1)
    df = df.append(a, ignore_index=True)
    print(df['multi'])

Problem description

Raises an exception "KeyError: 'multi' ", although this should perfectly work, as the key "Multi" exists, e.g when printing df.columns.

The most strange thing is:

  • The step in which this exception is raised depends on the Name of the key "Multi".. Sometimes 2nd loop, sometimes 6th,...
  • During the first loop it works perfectly and prints the df

However it has to do something with using "integer" and "string" column-names for level1 --- if i use an integer, e.g. -1 instead of "Multi", this seems to work.

Expected Output

It prints 100 times the dataframe

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit : None python : 3.7.1.final.0 python-bits : 64 OS : Windows OS-release : 10 machine : AMD64 processor : Intel64 Family 6 Model 142 Stepping 9, GenuineIntel byteorder : little LC_ALL : None LANG : None LOCALE : None.None

pandas : 0.25.3
numpy : 1.17.4
pytz : 2019.3
dateutil : 2.8.1
pip : 19.3.1
setuptools : 39.0.1
Cython : None
pytest : 5.2.2
hypothesis : None
sphinx : 2.2.1
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.10.3
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pytables : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None

@Alrik-star Alrik-star changed the title MultiIndex-Columns Integer and String: Column level 1 not accessable after a apllying append MultiIndex-Columns Integer and String: Column level 1 not accessable after a using append multiple times Nov 18, 2019
@Dr-Irv
Copy link
Contributor

Dr-Irv commented Sep 5, 2020

Works fine with 1.1.1

@Dr-Irv Dr-Irv closed this as completed Sep 5, 2020
@Dr-Irv Dr-Irv reopened this Sep 5, 2020
@Dr-Irv
Copy link
Contributor

Dr-Irv commented Sep 5, 2020

A PR that creates a test case would be welcomed.

@Dr-Irv Dr-Irv added good first issue Needs Tests Unit test(s) needed to prevent regressions labels Sep 5, 2020
@TAJD
Copy link
Contributor

TAJD commented Sep 7, 2020

take

@jreback jreback added this to the 1.2 milestone Sep 8, 2020
@jreback jreback modified the milestones: 1.2, Contributions Welcome Nov 4, 2020
@mroeschke mroeschke mentioned this issue May 29, 2021
15 tasks
@mroeschke mroeschke modified the milestones: Contributions Welcome, 1.3 May 30, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Needs Tests Unit test(s) needed to prevent regressions
Projects
None yet
5 participants