ENH: allow for top and mid-level assignment to DataFrames with MultIndex columns #7475

[iamlemec]

• closes ENH/BUG: support multi-index slicing assignment expansions #7475 (related to QST: Any support for df["x"] = y where y.columns are MultiIndexed? #35727)
• tests added / passed
• passes black pandas
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

This allows for one to assign to the top and mid-levels of DataFrame columns. To do so, it adds another fallback execution path to __setitem__. The logic is:

• In __setitem__, if the other existing paths don't apply, there is a single key, and the columns are a MultiIndex, pass to _setitem_multilevel. Here, assign if the key exists in the columns. Otherwise, use the new function append_block to assign multiple columns at once.
• In the existing _setitem_array, as a fallback, if the length of the assigning key equals the length of the top level of the value columns, recurse these back onto __setitem__. This allows for higher level list assignments.

I've also added some tests of the functionality in various cases. Any comments much appreciated.

[arw2019]

I don't have a comment on the changes themselves.

However, looking through the code there's two places where you added lines that aren't currently hit by tests (according to Codecov). We probably want to expand the test cases to include those

[iamlemec]

Thanks for the comment @arw2019. Just added in some new tests for that exception path. I believe there is full coverage now. The only weird thing I found was that indexing/multiindex/test_insert.py expects assigning one column to a multiindex with insufficient key levels to work regardless, so I commented out what would have been a conflicting test on my side.

[github-actions]

This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.

[iamlemec]

Sorry for the delay here, finally brought this up to date again. Had to change a few things to work well with the changes in #38459. Basically this is now a sub-path of _set_item_frame_value that kicks in when the key is not in the existing columns and when it is not a fully specified column.

[jreback]

closing as stale, if you want to continue working, please ping.