Skip to content

Tests for TDI issues already fixed #19044

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Jan 5, 2018
4 changes: 3 additions & 1 deletion doc/source/whatsnew/v0.23.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -300,7 +300,9 @@ Conversion
- Bug in :class:`FY5253` where ``datetime`` addition and subtraction incremented incorrectly for dates on the year-end but not normalized to midnight (:issue:`18854`)
- Bug in :class:`DatetimeIndex` where adding or subtracting an array-like of ``DateOffset`` objects either raised (``np.array``, ``pd.Index``) or broadcast incorrectly (``pd.Series``) (:issue:`18849`)
- Bug in :class:`Series` floor-division where operating on a scalar ``timedelta`` raises an exception (:issue:`18846`)

- Bug in :class:`Series`` with ``dtype='timedelta64[ns]`` where addition or subtraction of ``TimedeltaIndex`` had results cast to ``dtype='int64'`` (:issue:`17250`)
- Bug in :class:`TimedeltaIndex` where multiplication or division by a ``Series`` would return a ``TimedeltaIndex`` instead of a ``Series`` (issue:`19042`)
- Bug in :class:`Series` with ``dtype='timedelta64[ns]`` where addition or subtraction of ``TimedeltaIndex`` could return a ``Series`` with an incorrect name (issue:`19043`)

Indexing
^^^^^^^^
Expand Down
5 changes: 5 additions & 0 deletions pandas/core/indexes/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -4020,6 +4020,11 @@ def _add_numeric_methods_binary(cls):

def _make_evaluate_binop(op, opstr, reversed=False, constructor=Index):
def _evaluate_numeric_binop(self, other):
if (isinstance(other, ABCSeries) and
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like this at all. either let's remove this for now (and the corresponding test) or simplemente this by an override in the appopriate subclass.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removing this for now.

Overriding in subclass is a PITA b/c TimedeltaIndex._add_numeric_methods() is called below the definition of TimedeltaIndex and will override any __mul__ or __div__ methods we put in the class itself. So to override we need to implement it outside the class and pin it on after the call to add_numeric_methods. Since this is among my least favorite patterns, I'm punting on it.

self.dtype == "timedelta64[ns]"):
# GH#19042 This needs to be changed for all Index classes,
# but is only being handled for TimedeltaIndex in this PR.
return NotImplemented
Copy link
Member

@gfyoung gfyoung Jan 3, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explain why you're doing this in a comment.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll expand the comment. This actually needs to be done for all Index classes because the problem is wide-spread, but I really want to avoid scope creep.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough. I agree that we should try to keep the scope well-defined, though I might consider adding tests that you can xfail for now and patch in a follow-up (if you can add / make tests generic enough to apply to all Index classes).

other = self._validate_for_numeric_binop(other, op, opstr)

# handle time-based others
Expand Down
6 changes: 6 additions & 0 deletions pandas/core/ops.py
Original file line number Diff line number Diff line change
Expand Up @@ -775,6 +775,12 @@ def wrapper(left, right, name=name, na_op=na_op):
res_name = left.name

result = wrap_results(safe_na_op(lvalues, rvalues))
try:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would do this in _construct_result and simply pass in name = res_name or getattr(result, 'name', None)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call.

# if res_name is None we may need to override `result.name`
result.name = res_name
except AttributeError:
# np.ndarray has no name
pass
return construct_result(
left,
result,
Expand Down
16 changes: 16 additions & 0 deletions pandas/tests/indexes/test_numeric.py
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,22 @@ def test_numeric_compat(self):
result = idx**2.0
tm.assert_index_equal(result, expected)

@pytest.mark.xfail(reason="Index._add_numeric_methods_binary does not "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there an issue number ? I don't actually like adding a lot of xfails, if its only 1 ok

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yah, 19042. I'm removing this test.

"return NotImplemented when operating against "
"a Series.")
def test_index_with_series(self):
# GH#19044
idx = self.create_index()
didx = idx * idx
arr_dtype = 'uint64' if isinstance(idx, UInt64Index) else 'int64'
result = idx * Series(np.arange(5, dtype=arr_dtype))
tm.assert_series_equal(result, Series(didx))

result = idx * Series(np.arange(5, dtype='float64') + 0.1)
expected = Series(Float64Index(np.arange(5, dtype='float64') *
(np.arange(5, dtype='float64') + 0.1)))
tm.assert_series_equal(result, expected)

def test_explicit_conversions(self):

# GH 8608
Expand Down
10 changes: 6 additions & 4 deletions pandas/tests/indexes/timedeltas/test_arithmetic.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,11 +70,13 @@ def test_numeric_compat(self):
tm.assert_index_equal(result, didx)

result = idx * Series(np.arange(5, dtype='int64'))
tm.assert_index_equal(result, didx)
expected = Series(didx)
tm.assert_series_equal(result, expected)

result = idx * Series(np.arange(5, dtype='float64') + 0.1)
tm.assert_index_equal(result, self._holder(np.arange(
5, dtype='float64') * (np.arange(5, dtype='float64') + 0.1)))
rng5 = np.arange(5, dtype='float64')
result = idx * Series(rng5 + 0.1)
expected = Series(self._holder(rng5 * (rng5 + 0.1)))
tm.assert_series_equal(result, expected)

# invalid
pytest.raises(TypeError, lambda: idx * idx)
Expand Down
65 changes: 65 additions & 0 deletions pandas/tests/series/test_operators.py
Original file line number Diff line number Diff line change
Expand Up @@ -1038,6 +1038,71 @@ def test_timedelta_floordiv(self, scalar_td):
expected = Series([0, 0, np.nan])
tm.assert_series_equal(result, expected)

@pytest.mark.parametrize('names', [(None, None, None),
('Egon', 'Venkman', None),
('NCC1701D', 'NCC1701D', 'NCC1701D')])
def test_td64_series_with_tdi(self, names):
# GH#17250 make sure result dtype is correct
# GH#19043 make sure names are propogated correctly
tdi = pd.TimedeltaIndex(['0 days', '1 day'], name=names[0])
ser = Series([Timedelta(hours=3), Timedelta(hours=4)], name=names[1])
expected = Series([Timedelta(hours=3), Timedelta(days=1, hours=4)],
name=names[2])

result = tdi + ser
tm.assert_series_equal(result, expected)
assert result.dtype == 'timedelta64[ns]'

result = ser + tdi
tm.assert_series_equal(result, expected)
assert result.dtype == 'timedelta64[ns]'

expected = Series([Timedelta(hours=-3), Timedelta(days=1, hours=-4)],
name=names[2])

result = tdi - ser
tm.assert_series_equal(result, expected)
assert result.dtype == 'timedelta64[ns]'

result = ser - tdi
tm.assert_series_equal(result, -expected)
assert result.dtype == 'timedelta64[ns]'

@pytest.mark.parametrize('names', [(None, None, None),
('Egon', 'Venkman', None),
('NCC1701D', 'NCC1701D', 'NCC1701D')])
def test_tdi_mul_int_series(self, names):
# GH#19042
tdi = pd.TimedeltaIndex(['0days', '1day', '2days', '3days', '4days'],
name=names[0])
ser = Series([0, 1, 2, 3, 4], dtype=np.int64, name=names[1])

expected = Series(['0days', '1day', '4days', '9days', '16days'],
dtype='timedelta64[ns]',
name=names[2])

result = tdi * ser
tm.assert_series_equal(result, expected)

result = ser * tdi
tm.assert_series_equal(result, expected)

@pytest.mark.parametrize('names', [(None, None, None),
('Egon', 'Venkman', None),
('NCC1701D', 'NCC1701D', 'NCC1701D')])
def test_tdi_div_float_series(self, names):
# GH#19042
tdi = pd.TimedeltaIndex(['0days', '1day', '2days', '3days', '4days'],
name=names[0])
ser = Series([1.5, 3, 4.5, 6, 7.5], dtype=np.float64, name=names[1])

expected = Series([tdi[n] / ser[n] for n in range(len(ser))],
dtype='timedelta64[ns]',
name=names[2])

result = tdi / ser
tm.assert_series_equal(result, expected)


class TestDatetimeSeriesArithmetic(object):
@pytest.mark.parametrize(
Expand Down