Skip to content

BUG: loc raising KeyError for string slices in list-like indexer and DatetimeIndex #38153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.2.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -639,6 +639,7 @@ Indexing
- Bug in :meth:`DataFrame.loc` raising ``TypeError`` when non-integer slice was given to select values from :class:`MultiIndex` (:issue:`25165`, :issue:`24263`)
- Bug in :meth:`Series.at` returning :class:`Series` with one element instead of scalar when index is a :class:`MultiIndex` with one level (:issue:`38053`)
- Bug in :meth:`DataFrame.loc` returning and assigning elements in wrong order when indexer is differently ordered than the :class:`MultiIndex` to filter (:issue:`31330`, :issue:`34603`)
- Bug in :meth:`DataFrame.loc` raising ``KeyError`` for list-like indexers with string slices for :class:`DatetimeIndex` (:issue:`27180`)
- Bug in :meth:`DataFrame.loc` and :meth:`DataFrame.__getitem__` raising ``KeyError`` when columns were :class:`MultiIndex` with only one level (:issue:`29749`)
- Bug in :meth:`Series.__getitem__` and :meth:`DataFrame.__getitem__` raising blank ``KeyError`` without missing keys for :class:`IntervalIndex` (:issue:`27365`)
- Bug in setting a new label on a :class:`DataFrame` or :class:`Series` with a :class:`CategoricalIndex` incorrectly raising ``TypeError`` when the new label is not among the index's categories (:issue:`38098`)
Expand Down
19 changes: 19 additions & 0 deletions pandas/core/indexes/datetimes.py
Original file line number Diff line number Diff line change
Expand Up @@ -819,6 +819,25 @@ def slice_indexer(self, start=None, end=None, step=None, kind=None):

# --------------------------------------------------------------------

def _convert_listlike_indexer(self, key):
if not isinstance(key, list):
# There are no slices, so we can dispatch back
return super()._convert_listlike_indexer(key)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually you can short-circust if we don't have any slices right?

new_indexer = []
positions = list(range(len(self)))
try:
for k in key:
# Convert slices to list of integers
indexer = positions[self.get_loc(k)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you use .get_indexer() here? (and still then iterate over the results)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed not supporting this. Has this changed?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yah i think we want to close this

if not isinstance(indexer, list):
indexer = [indexer]
new_indexer.extend(indexer)
except KeyError:
# Dispatch to base method for handling of KeyErrors
return super()._convert_listlike_indexer(key)
return np.array(new_indexer), key

@property
def inferred_type(self) -> str:
# b/c datetime is represented as microseconds since the epoch, make
Expand Down
18 changes: 18 additions & 0 deletions pandas/tests/indexing/test_datetime.py
Original file line number Diff line number Diff line change
Expand Up @@ -231,3 +231,21 @@ def test_loc_setitem_with_existing_dst(self):
dtype=object,
)
tm.assert_frame_equal(result, expected)

@pytest.mark.parametrize(
"indexer", [["2001-01", "2001-01-30"], ["2001-01", Timestamp("2001-01-30")]]
)
def test_loc_getitem_partial_strings_in_list(self, indexer):
# GH#27180
ser = Series(1, index=date_range("2001-01-29", periods=60))
result = ser.loc[indexer]
expected = Series(
1,
index=[
Timestamp("2001-01-29"),
Timestamp("2001-01-30"),
Timestamp("2001-01-31"),
Timestamp("2001-01-30"),
],
)
Comment on lines +242 to +250
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick, but this might look a bit better if you define index = on its own line and then here have expected = Series(1, index=index)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

tm.assert_series_equal(result, expected)