-
-
Notifications
You must be signed in to change notification settings - Fork 18.4k
REF: IntervalArray comparisons #37124
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 9 commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
a3b7c45
ENH: IntervalArray comparisons
jbrockmendel 60197d2
Merge branch 'master' of https://github.com/pandas-dev/pandas into re…
jbrockmendel cf20846
CLN: get IntervalIndex comparisons from IntervalArray
jbrockmendel d3c1041
Merge branch 'master' of https://github.com/pandas-dev/pandas into re…
jbrockmendel fcfe47d
update per requests
jbrockmendel 3dfc008
Merge branch 'master' of https://github.com/pandas-dev/pandas into re…
jbrockmendel fa6cecd
Avoid having to tile
jbrockmendel 0d32acd
Merge branch 'master' of https://github.com/pandas-dev/pandas into re…
jbrockmendel ff640ea
handle NA per suggestion
jbrockmendel f501b4e
Merge branch 'master' of https://github.com/pandas-dev/pandas into re…
jbrockmendel 247ce90
comment
jbrockmendel 84b0409
Merge branch 'master' of https://github.com/pandas-dev/pandas into re…
jbrockmendel 1ea125d
Merge branch 'master' of https://github.com/pandas-dev/pandas into re…
jbrockmendel 6e43726
Merge branch 'master' of https://github.com/pandas-dev/pandas into re…
jbrockmendel badb99d
update comment
jbrockmendel File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,4 @@ | ||
import operator | ||
from operator import le, lt | ||
import textwrap | ||
from typing import TYPE_CHECKING, Optional, Tuple, Union, cast | ||
|
@@ -12,6 +13,7 @@ | |
IntervalMixin, | ||
intervals_to_interval_bounds, | ||
) | ||
from pandas._libs.missing import NA | ||
from pandas._typing import ArrayLike, Dtype | ||
from pandas.compat.numpy import function as nv | ||
from pandas.util._decorators import Appender | ||
|
@@ -48,7 +50,7 @@ | |
from pandas.core.construction import array, extract_array | ||
from pandas.core.indexers import check_array_indexer | ||
from pandas.core.indexes.base import ensure_index | ||
from pandas.core.ops import unpack_zerodim_and_defer | ||
from pandas.core.ops import invalid_comparison, unpack_zerodim_and_defer | ||
|
||
if TYPE_CHECKING: | ||
from pandas import Index | ||
|
@@ -520,16 +522,15 @@ def __setitem__(self, key, value): | |
self._left[key] = value_left | ||
self._right[key] = value_right | ||
|
||
@unpack_zerodim_and_defer("__eq__") | ||
def __eq__(self, other): | ||
def _cmp_method(self, other, op): | ||
# ensure pandas array for list-like and eliminate non-interval scalars | ||
if is_list_like(other): | ||
if len(self) != len(other): | ||
raise ValueError("Lengths must match to compare") | ||
other = array(other) | ||
elif not isinstance(other, Interval): | ||
# non-interval scalar -> no matches | ||
return np.zeros(len(self), dtype=bool) | ||
return invalid_comparison(self, other, op) | ||
|
||
# determine the dtype of the elements we want to compare | ||
if isinstance(other, Interval): | ||
|
@@ -543,33 +544,75 @@ def __eq__(self, other): | |
# extract intervals if we have interval categories with matching closed | ||
if is_interval_dtype(other_dtype): | ||
if self.closed != other.categories.closed: | ||
return np.zeros(len(self), dtype=bool) | ||
return invalid_comparison(self, other, op) | ||
other = other.categories.take(other.codes) | ||
|
||
# interval-like -> need same closed and matching endpoints | ||
if is_interval_dtype(other_dtype): | ||
if self.closed != other.closed: | ||
return np.zeros(len(self), dtype=bool) | ||
return (self._left == other.left) & (self._right == other.right) | ||
return invalid_comparison(self, other, op) | ||
elif not isinstance(other, Interval): | ||
other = type(self)(other) | ||
|
||
if op is operator.eq: | ||
jreback marked this conversation as resolved.
Show resolved
Hide resolved
|
||
return (self._left == other.left) & (self._right == other.right) | ||
elif op is operator.ne: | ||
return (self._left != other.left) | (self._right != other.right) | ||
elif op is operator.gt: | ||
return (self._left > other.left) | ( | ||
(self._left == other.left) & (self._right > other.right) | ||
) | ||
elif op is operator.ge: | ||
return (self == other) | (self > other) | ||
elif op is operator.lt: | ||
return (self._left < other.left) | ( | ||
(self._left == other.left) & (self._right < other.right) | ||
) | ||
else: | ||
# operator.lt | ||
return (self == other) | (self < other) | ||
|
||
# non-interval/non-object dtype -> no matches | ||
if not is_object_dtype(other_dtype): | ||
return np.zeros(len(self), dtype=bool) | ||
return invalid_comparison(self, other, op) | ||
|
||
# object dtype -> iteratively check for intervals | ||
result = np.zeros(len(self), dtype=bool) | ||
for i, obj in enumerate(other): | ||
# need object to be an Interval with same closed and endpoints | ||
if ( | ||
isinstance(obj, Interval) | ||
and self.closed == obj.closed | ||
and self._left[i] == obj.left | ||
and self._right[i] == obj.right | ||
): | ||
result[i] = True | ||
|
||
try: | ||
result[i] = op(self[i], obj) | ||
except TypeError: | ||
if obj is NA: | ||
# github.com/pandas-dev/pandas/pull/37124#discussion_r509095092 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can you add a comment with an actual explanation instead of (only) the link? |
||
result[i] = op is operator.ne | ||
else: | ||
raise | ||
return result | ||
|
||
@unpack_zerodim_and_defer("__eq__") | ||
def __eq__(self, other): | ||
return self._cmp_method(other, operator.eq) | ||
|
||
@unpack_zerodim_and_defer("__ne__") | ||
def __ne__(self, other): | ||
return self._cmp_method(other, operator.ne) | ||
|
||
@unpack_zerodim_and_defer("__gt__") | ||
def __gt__(self, other): | ||
return self._cmp_method(other, operator.gt) | ||
|
||
@unpack_zerodim_and_defer("__ge__") | ||
def __ge__(self, other): | ||
return self._cmp_method(other, operator.ge) | ||
|
||
@unpack_zerodim_and_defer("__lt__") | ||
def __lt__(self, other): | ||
return self._cmp_method(other, operator.lt) | ||
|
||
@unpack_zerodim_and_defer("__le__") | ||
def __le__(self, other): | ||
return self._cmp_method(other, operator.le) | ||
|
||
def fillna(self, value=None, method=None, limit=None): | ||
""" | ||
Fill NA/NaN values using the specified method. | ||
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is this no longer needed? or unrelated?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you comment here @jbrockmendel
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In IntervalArray.repeat we call
self._combined.repeat(repeats, 0)
, which when _combined is DTA/TDA gets here withaxis=0
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since there is still discussion about whether
_combined
is meant to stay (#37047), can you leave out this clean-up from this PR?(unless the clean-up is not tied to having
_combined
, but from your comment it seems so)