Skip to content

Fix MultiIndex .loc "Too Many Indexers" with None as return value #34450

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jun 3, 2020
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.1.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -811,6 +811,7 @@ Indexing
- Bug in :meth:`DataFrame.truncate` and :meth:`Series.truncate` where index was assumed to be monotone increasing (:issue:`33756`)
- Indexing with a list of strings representing datetimes failed on :class:`DatetimeIndex` or :class:`PeriodIndex`(:issue:`11278`)
- Bug in :meth:`Series.at` when used with a :class:`MultiIndex` would raise an exception on valid inputs (:issue:`26989`)
- Bug in :meth:`Series.loc` when used with a :class:`MultiIndex` would raise an IndexingError when accessing a None value (:issue:`34318`)

Missing
^^^^^^^
Expand Down
14 changes: 9 additions & 5 deletions pandas/core/indexing.py
Original file line number Diff line number Diff line change
Expand Up @@ -766,9 +766,11 @@ def _getitem_lowerdim(self, tup: Tuple):
# ...but iloc should handle the tuple as simple integer-location
# instead of checking it as multiindex representation (GH 13797)
if isinstance(ax0, ABCMultiIndex) and self.name != "iloc":
result = self._handle_lowerdim_multi_index_axis0(tup)
if result is not None:
try:
result = self._handle_lowerdim_multi_index_axis0(tup)
return result
except IndexingError:
pass

if len(tup) > self.ndim:
raise IndexingError("Too many indexers. handle elsewhere")
Expand Down Expand Up @@ -816,9 +818,11 @@ def _getitem_nested_tuple(self, tup: Tuple):
if self.name != "loc":
# This should never be reached, but lets be explicit about it
raise ValueError("Too many indices")
result = self._handle_lowerdim_multi_index_axis0(tup)
if result is not None:
try:
result = self._handle_lowerdim_multi_index_axis0(tup)
return result
except IndexingError:
pass

# this is a series with a multi-index specified a tuple of
# selectors
Expand Down Expand Up @@ -1065,7 +1069,7 @@ def _handle_lowerdim_multi_index_axis0(self, tup: Tuple):
if len(tup) <= self.obj.index.nlevels and len(tup) > self.ndim:
raise ek

return None
raise IndexingError("No label returned")

def _getitem_axis(self, key, axis: int):
key = item_from_zerodim(key)
Expand Down
17 changes: 17 additions & 0 deletions pandas/tests/series/indexing/test_indexing.py
Original file line number Diff line number Diff line change
Expand Up @@ -853,6 +853,23 @@ def test_setitem_slice_into_readonly_backing_data():
assert not array.any()


def test_access_none_value_in_multiindex():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you make a new file in this dir: test_multiindex.py

we have the MI tests elsewhere so ok starting a new file here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should I copy all the imports from test_indexing.py?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you need whatever imports make flake happy :->

# GH34318: test that you can access a None value using .loc through a Multiindex

s = Series([None], pd.MultiIndex.from_arrays([["Level1"], ["Level2"]]))
result = s.loc[("Level1", "Level2")]
assert result is None

midx = MultiIndex.from_product([["Level1"], ["Level2_a", "Level2_b"]])
s = Series([None] * len(midx), dtype=object, index=midx)
result = s.loc[("Level1", "Level2_a")]
assert result is None

s = Series([1] * len(midx), dtype=object, index=midx)
result = s.loc[("Level1", "Level2_a")]
assert result == 1


"""
miscellaneous methods
"""
Expand Down