Skip to content

BUG: Panel setitem with a multiindex #10360 (partial) #10838

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 18, 2015
Merged

BUG: Panel setitem with a multiindex #10360 (partial) #10838

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/source/whatsnew/v0.17.0.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -606,7 +606,7 @@ Bug Fixes
- Bug in ``offsets.generate_range`` where ``start`` and ``end`` have finer precision than ``offset`` (:issue:`9907`)
- Bug in ``pd.rolling_*`` where ``Series.name`` would be lost in the output (:issue:`10565`)
- Bug in ``stack`` when index or columns are not unique. (:issue:`10417`)

- Bug in setting a Panel when an axis has a multi-index (:issue:`10360`)



Expand Down
17 changes: 11 additions & 6 deletions pandas/core/indexing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,7 @@ def _setitem_with_indexer(self, indexer, value):

# also has the side effect of consolidating in-place
from pandas import Panel, DataFrame, Series
info_axis = self.obj._info_axis_number

# maybe partial set
take_split_path = self.obj._is_mixed_type
Expand All @@ -213,6 +214,16 @@ def _setitem_with_indexer(self, indexer, value):
val = list(value.values()) if isinstance(value,dict) else value
take_split_path = not blk._can_hold_element(val)

if isinstance(indexer, tuple) and len(indexer) == len(self.obj.axes):

for i, ax in zip(indexer, self.obj.axes):

# if we have any multi-indexes that have non-trivial slices (not null slices)
# then we must take the split path, xref GH 10360
if isinstance(ax, MultiIndex) and not (is_integer(i) or is_null_slice(i)):
take_split_path = True
break

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jreback looks like this is intended for Panel, but is still reached. Any chance it is unnecessary? Or more specifically, I'd like to rule out the possibility of take_split_path = True being set on 224 for Series. Is that unreachable for some reason?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

umm maybe, the whole take_split_path is based on the notion that we have a block of a single type and we are indexing into it for smaller number of columns than the whole block so we might need to split it before we index. fairly dense & complicated, but necessary in some way.

to answer your question though, I agree you might be able to remove this entire clause.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apparently there was still a use case: #27855

if isinstance(indexer, tuple):
nindexer = []
for i, idx in enumerate(indexer):
Expand Down Expand Up @@ -328,14 +339,8 @@ def _setitem_with_indexer(self, indexer, value):
return self.obj.__setitem__(indexer, value)

# set
info_axis = self.obj._info_axis_number
item_labels = self.obj._get_axis(info_axis)

# if we have a complicated setup, take the split path
if (isinstance(indexer, tuple) and
any([isinstance(ax, MultiIndex) for ax in self.obj.axes])):
take_split_path = True

# align and set the values
if take_split_path:

Expand Down
40 changes: 39 additions & 1 deletion pandas/tests/test_indexing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -411,7 +411,7 @@ def test_iloc_exceeds_bounds(self):
df.iloc[30]
self.assertRaises(IndexError, lambda : df.iloc[-30])

# GH10779
# GH10779
# single positive/negative indexer exceeding Series bounds should raise an IndexError
with tm.assertRaisesRegexp(IndexError, 'single positional indexer is out-of-bounds'):
s.iloc[30]
Expand Down Expand Up @@ -2652,6 +2652,44 @@ def test_panel_setitem(self):
tm.assert_panel_equal(p, expected)


def test_panel_setitem_with_multiindex(self):

# 10360
# failing with a multi-index
arr = np.array([[[1,2,3],[0,0,0]],[[0,0,0],[0,0,0]]],dtype=np.float64)

# reg index
axes = dict(items=['A', 'B'], major_axis=[0, 1], minor_axis=['X', 'Y' ,'Z'])
p1 = Panel(0., **axes)
p1.iloc[0, 0, :] = [1, 2, 3]
expected = Panel(arr, **axes)
tm.assert_panel_equal(p1, expected)

# multi-indexes
axes['items'] = pd.MultiIndex.from_tuples([('A','a'), ('B','b')])
p2 = Panel(0., **axes)
p2.iloc[0, 0, :] = [1, 2, 3]
expected = Panel(arr, **axes)
tm.assert_panel_equal(p2, expected)

axes['major_axis']=pd.MultiIndex.from_tuples([('A',1),('A',2)])
p3 = Panel(0., **axes)
p3.iloc[0, 0, :] = [1, 2, 3]
expected = Panel(arr, **axes)
tm.assert_panel_equal(p3, expected)

axes['minor_axis']=pd.MultiIndex.from_product([['X'],range(3)])
p4 = Panel(0., **axes)
p4.iloc[0, 0, :] = [1, 2, 3]
expected = Panel(arr, **axes)
tm.assert_panel_equal(p4, expected)

arr = np.array([[[1,0,0],[2,0,0]],[[0,0,0],[0,0,0]]],dtype=np.float64)
p5 = Panel(0., **axes)
p5.iloc[0, :, 0] = [1, 2]
expected = Panel(arr, **axes)
tm.assert_panel_equal(p5, expected)

def test_panel_assignment(self):

# GH3777
Expand Down