Skip to content

ENH: Implement subtraction for object-dtype Index #21981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 23, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.24.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,7 @@ Other API Changes
- Invalid construction of ``IntervalDtype`` will now always raise a ``TypeError`` rather than a ``ValueError`` if the subdtype is invalid (:issue:`21185`)
- Trying to reindex a ``DataFrame`` with a non unique ``MultiIndex`` now raises a ``ValueError`` instead of an ``Exception`` (:issue:`21770`)
- :meth:`PeriodIndex.tz_convert` and :meth:`PeriodIndex.tz_localize` have been removed (:issue:`21781`)
- :class:`Index` subtraction will attempt to operate element-wise instead of raising ``TypeError`` (:issue:`19369`)

.. _whatsnew_0240.deprecations:

Expand Down
6 changes: 4 additions & 2 deletions pandas/core/indexes/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -2630,8 +2630,10 @@ def __iadd__(self, other):
return self + other

def __sub__(self, other):
raise TypeError("cannot perform __sub__ with this index type: "
"{typ}".format(typ=type(self).__name__))
return Index(np.array(self) - other)

def __rsub__(self, other):
return Index(other - np.array(self))

def __and__(self, other):
return self.intersection(other)
Expand Down
38 changes: 36 additions & 2 deletions pandas/tests/indexes/test_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import pytest

from datetime import datetime, timedelta

from decimal import Decimal
from collections import defaultdict

import pandas.util.testing as tm
Expand Down Expand Up @@ -864,13 +864,47 @@ def test_add(self):
expected = Index(['1a', '1b', '1c'])
tm.assert_index_equal('1' + index, expected)

def test_sub(self):
def test_sub_fail(self):
index = self.strIndex
pytest.raises(TypeError, lambda: index - 'a')
pytest.raises(TypeError, lambda: index - index)
pytest.raises(TypeError, lambda: index - index.tolist())
pytest.raises(TypeError, lambda: index.tolist() - index)

def test_sub_object(self):
# GH#19369
index = pd.Index([Decimal(1), Decimal(2)])
expected = pd.Index([Decimal(0), Decimal(1)])

result = index - Decimal(1)
tm.assert_index_equal(result, expected)

result = index - pd.Index([Decimal(1), Decimal(1)])
tm.assert_index_equal(result, expected)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we also add tests where element-wise subtraction should fail.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are already some in the test just above this, but more couldn’t hurt. Will do.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes, thanks for pointing that out. Two things would be good there:

  • We should rename that test above to be test_sub_fail
  • Add a couple of tests where element-wise subtraction should fail

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You got it.

with pytest.raises(TypeError):
index - 'foo'

with pytest.raises(TypeError):
index - np.array([2, 'foo'])

def test_rsub_object(self):
# GH#19369
index = pd.Index([Decimal(1), Decimal(2)])
expected = pd.Index([Decimal(1), Decimal(0)])

result = Decimal(2) - index
tm.assert_index_equal(result, expected)

result = np.array([Decimal(2), Decimal(2)]) - index
tm.assert_index_equal(result, expected)

with pytest.raises(TypeError):
'foo' - index

with pytest.raises(TypeError):
np.array([True, pd.Timestamp.now()]) - index

def test_map_identity_mapping(self):
# GH 12766
# TODO: replace with fixture
Expand Down