Skip to content

value assignment bug for multi-indexed dataframes in v1.0.0rc0 #30892

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
leo4183 opened this issue Jan 10, 2020 · 4 comments · Fixed by #31358
Closed

value assignment bug for multi-indexed dataframes in v1.0.0rc0 #30892

leo4183 opened this issue Jan 10, 2020 · 4 comments · Fixed by #31358
Labels
Indexing Related to indexing on series/frames, not to indexes themselves Regression Functionality that used to work in a prior pandas version
Milestone

Comments

@leo4183
Copy link

leo4183 commented Jan 10, 2020

Problem Description

in new pre-release 1.0.0.rc0, sliced value assignment for multi-indexed dataframe with duplicated datetime indices would fail

import pandas as pd
x = pd.DataFrame([[1,2,3],['a','a','c']],index=['c1','index2'],
                 columns=pd.to_datetime(['20190101','20190101','20190102'])).T
x['c2'] = np.nan
x.index.name = 'index1'
x.set_index(['index2'],append=True,inplace=True)


In [0]: x
Out[0]:
                 c1  c2
index1   index2       
20190101 a       1  NaN
         a       2  NaN
20190102 c       3  NaN


x.loc[(pd.to_datetime('20190101'),'a'),'c2'] = 1.0     # ok
x.loc[[(pd.to_datetime('20190101'),'a')],'c2'] = 1.0   # ValueError
@jorisvandenbossche
Copy link
Member

@leo4183 Thanks a lot for trying out 1.0.0rc, and for the report! That's indeed a regression.

cc @jbrockmendel from the traceback, this seems to be caused by the exception cleaning from #28361. Can you take a look?

@jorisvandenbossche jorisvandenbossche added Indexing Related to indexing on series/frames, not to indexes themselves Regression Functionality that used to work in a prior pandas version labels Jan 12, 2020
@jorisvandenbossche jorisvandenbossche added this to the 1.0.0 milestone Jan 12, 2020
@jbrockmendel
Copy link
Member

Will take a look

@TomAugspurger
Copy link
Contributor

@jbrockmendel did you have a chance to look into this?

@jbrockmendel
Copy link
Member

Looks like catching InvalidIndexError in the spot @jorisvandenbossche pointed out in #28361 fixes one of the two problems (on master both of the examples raise, not just the second). i expect that the other can be fixed by catching elsewhere fairly high-up. I'll get a PR for this in hopefully today.

Longer-term, the underlying problem looks like MultiIndex.get_loc is doing something weird that we need to dig into.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Indexing Related to indexing on series/frames, not to indexes themselves Regression Functionality that used to work in a prior pandas version
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants