Skip to content

API: allow step!=1 slice with IntervalIndex #31658

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 30 commits into from
Mar 8, 2020
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
7a26837
Allow step!=1 in slicing Series with IntervalIndex
jbrockmendel Feb 2, 2020
b371805
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Feb 3, 2020
b78c4c3
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Feb 3, 2020
e3631d9
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Feb 3, 2020
7949ffd
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Feb 3, 2020
4c2e703
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Feb 4, 2020
88a7ace
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Feb 4, 2020
0da92c5
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Feb 4, 2020
f833f7c
Whatsnew
jbrockmendel Feb 5, 2020
041edbc
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Feb 6, 2020
a2eb9bb
doc suggestion
jbrockmendel Feb 6, 2020
6b97b09
test for interval step
jbrockmendel Feb 6, 2020
abdbd76
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Feb 6, 2020
a678ec2
remove commented-out
jbrockmendel Feb 6, 2020
2c684cc
reword whatsnew
jbrockmendel Feb 6, 2020
e4cc38d
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Feb 6, 2020
10165b4
disallow label-based with step!=1
jbrockmendel Feb 6, 2020
8308eb8
black fixup
jbrockmendel Feb 6, 2020
2bffbda
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Feb 17, 2020
0bdb7b5
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Feb 22, 2020
adf0775
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Feb 25, 2020
2744a4b
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Feb 25, 2020
cf1a393
update error message
jbrockmendel Feb 25, 2020
74a40ff
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Mar 2, 2020
9a04d41
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Mar 3, 2020
f6360a6
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Mar 3, 2020
d636108
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Mar 5, 2020
9222513
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Mar 5, 2020
2a698c0
typo fixup
jbrockmendel Mar 7, 2020
d18c7a1
Merge branch 'master' of https://github.com/pandas-dev/pandas into co…
jbrockmendel Mar 7, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/source/whatsnew/v1.1.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ Other enhancements
- When writing directly to a sqlite connection :func:`to_sql` now supports the ``multi`` method (:issue:`29921`)
- `OptionError` is now exposed in `pandas.errors` (:issue:`27553`)
- :func:`timedelta_range` will now infer a frequency when passed ``start``, ``stop``, and ``periods`` (:issue:`32377`)
- Positional slicing on a :class:`IntervalIndex` now supports slices with ``step > 1`` (:issue:`31658`)
-

.. ---------------------------------------------------------------------------
Expand Down Expand Up @@ -246,7 +247,6 @@ Strings

Interval
^^^^^^^^

-
-

Expand Down
29 changes: 29 additions & 0 deletions pandas/core/indexers.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
is_array_like,
is_bool_dtype,
is_extension_array_dtype,
is_integer,
is_integer_dtype,
is_list_like,
)
Expand All @@ -20,6 +21,34 @@
# Indexer Identification


def is_valid_positional_slice(slc: slice) -> bool:
"""
Check if a slice object can be interpretd as a positional indexer.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: interpretd --> interpreted

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated


Parameters
----------
slc : slice

Returns
-------
bool

Notes
-----
A valid positional slice may also be interpreted as a label-based slice
depending on the index being sliced.
"""

def is_int_or_none(val):
return val is None or is_integer(val)

return (
is_int_or_none(slc.start)
and is_int_or_none(slc.stop)
and is_int_or_none(slc.step)
)


def is_list_like_indexer(key) -> bool:
"""
Check if we have a list-like indexer that is *not* a NamedTuple.
Expand Down
12 changes: 11 additions & 1 deletion pandas/core/indexes/interval.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@
from pandas.core.algorithms import take_1d
from pandas.core.arrays.interval import IntervalArray, _interval_shared_docs
import pandas.core.common as com
from pandas.core.indexers import is_valid_positional_slice
import pandas.core.indexes.base as ibase
from pandas.core.indexes.base import (
Index,
Expand Down Expand Up @@ -866,7 +867,16 @@ def get_indexer_for(self, target: AnyArrayLike, **kwargs) -> np.ndarray:

def _convert_slice_indexer(self, key: slice, kind: str):
if not (key.step is None or key.step == 1):
raise ValueError("cannot support not-default step in a slice")
# GH#31658 if label-based, we require step == 1,
# if positional, we disallow float start/stop
msg = "label-based slicing with step!=1 is not supported for IntervalIndex"
if kind == "loc":
raise ValueError(msg)
elif kind == "getitem":
if not is_valid_positional_slice(key):
# i.e. this cannot be interpreted as a positional slice
raise ValueError(msg)

return super()._convert_slice_indexer(key, kind)

@Appender(Index.where.__doc__)
Expand Down
3 changes: 2 additions & 1 deletion pandas/tests/indexes/test_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -2598,7 +2598,8 @@ def test_convert_almost_null_slice(indices):
key = slice(None, None, "foo")

if isinstance(idx, pd.IntervalIndex):
with pytest.raises(ValueError, match="cannot support not-default step"):
msg = "label-based slicing with step!=1 is not supported for IntervalIndex"
with pytest.raises(ValueError, match=msg):
idx._convert_slice_indexer(key, "loc")
else:
msg = "'>=' not supported between instances of 'str' and 'int'"
Expand Down
32 changes: 26 additions & 6 deletions pandas/tests/indexing/interval/test_interval_new.py
Original file line number Diff line number Diff line change
Expand Up @@ -128,9 +128,6 @@ def test_loc_with_slices(self):
with pytest.raises(NotImplementedError, match=msg):
s[Interval(3, 4, closed="left") :]

# TODO with non-existing intervals ?
# s.loc[Interval(-1, 0):Interval(2, 3)]

# slice of scalar

expected = s.iloc[:3]
Expand All @@ -143,9 +140,32 @@ def test_loc_with_slices(self):
tm.assert_series_equal(expected, s[:2.5])
tm.assert_series_equal(expected, s[0.1:2.5])

# slice of scalar with step != 1
with pytest.raises(ValueError):
s[0:4:2]
def test_slice_step_ne1(self):
# GH#31658 slice of scalar with step != 1
s = self.s
expected = s.iloc[0:4:2]

result = s[0:4:2]
tm.assert_series_equal(result, expected)

result2 = s[0:4][::2]
tm.assert_series_equal(result2, expected)

def test_slice_float_start_stop(self):
# GH#31658 slicing with integers is positional, with floats is not
# supported
ser = Series(np.arange(5), IntervalIndex.from_breaks(np.arange(6)))

msg = "label-based slicing with step!=1 is not supported for IntervalIndex"
with pytest.raises(ValueError, match=msg):
ser[1.5:9.5:2]

def test_slice_interval_step(self):
# GH#31658 allows for integer step!=1, not Interval step
s = self.s
msg = "label-based slicing with step!=1 is not supported for IntervalIndex"
with pytest.raises(ValueError, match=msg):
s[0 : 4 : Interval(0, 1)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would test it where start and stop are also intervals.
And can you move it to the test_loc_with_slices a bit above that is testing slicing with Intervals ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

start and stop are also intervals

out of scope

And can you move it to the test_loc_with_slices a bit above that is testing slicing with Intervals ?

sure

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

start and stop are also intervals

out of scope

I think that is not out of scope, as that hits this path, and right now actually ensured there is a proper error message.


def test_loc_with_overlap(self):

Expand Down