Skip to content

Bug: Logical operator of Series with Index (#22092) #22293

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Sep 18, 2018
Merged

Bug: Logical operator of Series with Index (#22092) #22293

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
2c1bb1a
Fix bug #GH22092
makbigc Aug 12, 2018
3607834
Update v0.24.0.txt
makbigc Aug 13, 2018
9f7b586
Update v0.24.0.txt
makbigc Aug 14, 2018
fd0f381
Update ops.py
makbigc Aug 14, 2018
f9f63fd
Update test_operators.py
makbigc Aug 14, 2018
ba456e7
Update v0.24.0.txt
makbigc Aug 22, 2018
84ac158
Update test_operators.py
makbigc Aug 22, 2018
aaf8349
Merge branch 'master' into master
makbigc Sep 9, 2018
d671f99
Merge branch 'master' into master
makbigc Sep 16, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/source/whatsnew/v0.24.0.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -497,6 +497,6 @@ Other
- :meth: `~pandas.io.formats.style.Styler.background_gradient` now takes a ``text_color_threshold`` parameter to automatically lighten the text color based on the luminance of the background color. This improves readability with dark background colors without the need to limit the background colormap range. (:issue:`21258`)
- Require at least 0.28.2 version of ``cython`` to support read-only memoryviews (:issue:`21688`)
- :meth: `~pandas.io.formats.style.Styler.background_gradient` now also supports tablewise application (in addition to rowwise and columnwise) with ``axis=None`` (:issue:`15204`)
-
- Bug in the logical operators handling :class:`Series` and :class:`Index` together (:issue:`22092`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Describe the bug briefly. For an example, see #22173.

-
-
2 changes: 1 addition & 1 deletion pandas/core/ops.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1383,7 +1383,7 @@ def na_op(x, y):
if isinstance(y, list):
y = construct_1d_object_array_from_listlike(y)

if isinstance(y, (np.ndarray, ABCSeries)):
if isinstance(y, (np.ndarray, ABCSeries, ABCIndex)):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be ABCIndexClass, not ABCIndex.

if (is_bool_dtype(x.dtype) and is_bool_dtype(y.dtype)):
result = op(x, y) # when would this be hit?
else:
Expand Down
17 changes: 17 additions & 0 deletions pandas/tests/series/test_operators.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -537,6 +537,23 @@ def test_comparison_flex_alignment_fill(self):
exp = pd.Series([True, True, False, False], index=list('abcd'))
assert_series_equal(left.gt(right, fill_value=0), exp)

def test_comparison_with_index(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you check this on Int64Index as well. I think some other index types might work.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"comparison" is a misnomer here. Consider "binary_ops" or "logical_ops"

# GH22092
ser = Series([True, True, False, False])
idx = Index([True, False, True, False])

expected = Series([True, False, False, False])
result = ser & idx
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @jbrockmendel I think we DO allow this on index types which don't makes sense, but we DO raise on Series, e.g.

not sure what to make of [1] and [2]

In [1]: dti = pd.date_range('20130101', periods=3)

In [2]: dti ^ dti
Out[2]: DatetimeIndex([], dtype='datetime64[ns]', freq=None)

In [3]: dti & dti
Out[3]: DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03'], dtype='datetime64[ns]', freq='D')
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

# I think this PR fixes this one?
In [4]: pd.Series(dti) & dti

# this is good
In [5]: pd.Series(dti) & pd.Series(dti)
TypeError: cannot astype a datetimelike from [datetime64[ns]] to [bool]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we DO allow this on index types which don't makes sense, but we DO raise on Series,

Yah I've opened a couple of Issues about this. The existing Series ops don't handle dt64/td64 at all, and are not super-clear for some other dtypes. The reversed op(Index, Series) ops are a separate can of worms because those are set operations.

I don't think this PR fixes [4] (yet), needs to chance ABCIndex to ABCIndexClass (like #22173)

Copy link
Contributor

@jreback jreback Aug 14, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok this is fine then (for now as you have other issues about this)

assert_series_equal(result, expected)

expected = Series([True, True, True, False])
result = ser | idx
assert_series_equal(result, expected)

expected = Series([False, True, True, False])
result = ser ^ idx
assert_series_equal(result, expected)

def test_ne(self):
ts = Series([3, 4, 5, 6, 7], [3, 4, 5, 6, 7], dtype=float)
expected = [True, True, False, True, True]
Expand Down