Skip to content

TST: Extend timedelta64 arithmetic tests to TimedeltaArray #23642

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 24 commits into from
Nov 20, 2018
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
e6463be
implement box_with_timedelta, assert_timedelta_array_equal, and minim…
jbrockmendel Nov 12, 2018
f5a7891
extend tests for floordiv
jbrockmendel Nov 12, 2018
291dd51
extend tests to multiplication
jbrockmendel Nov 12, 2018
52319fd
extend tests for floordiv
jbrockmendel Nov 12, 2018
da1b45a
Merge branch 'master' of https://github.com/pandas-dev/pandas into pr…
jbrockmendel Nov 12, 2018
0b21797
extend tests to int/float
jbrockmendel Nov 12, 2018
cb63fe7
Merge branch 'master' of https://github.com/pandas-dev/pandas into pr…
jbrockmendel Nov 12, 2018
6ccff81
isort fixup
jbrockmendel Nov 12, 2018
1ffc21f
suggested edits
jbrockmendel Nov 12, 2018
8c9f296
Fixup unused import
jbrockmendel Nov 12, 2018
3c53183
Merge branch 'master' of https://github.com/pandas-dev/pandas into pr…
jbrockmendel Nov 15, 2018
74046ba
implement abs, tests
jbrockmendel Nov 15, 2018
1c9fb1a
test fixups
jbrockmendel Nov 15, 2018
9f4d4d9
Merge branch 'pre-arith' of https://github.com/jbrockmendel/pandas in…
jbrockmendel Nov 15, 2018
07c858e
Merge branch 'master' of https://github.com/pandas-dev/pandas into pr…
jbrockmendel Nov 17, 2018
b8e2127
Merge branch 'master' of https://github.com/pandas-dev/pandas into pr…
jbrockmendel Nov 17, 2018
6dd6b3b
Merge branch 'master' of https://github.com/pandas-dev/pandas into pr…
jbrockmendel Nov 18, 2018
9c11a6c
use box_with_array instead of box_with_timedelta
jbrockmendel Nov 18, 2018
e4c7e98
update fixture usages
jbrockmendel Nov 18, 2018
cf97665
Merge branch 'master' of https://github.com/pandas-dev/pandas into pr…
jbrockmendel Nov 18, 2018
62edbc3
Merge branch 'master' of https://github.com/pandas-dev/pandas into pr…
jbrockmendel Nov 18, 2018
fa3e8d1
Merge branch 'master' of https://github.com/pandas-dev/pandas into pr…
jbrockmendel Nov 19, 2018
4cc3cbd
Merge branch 'master' of https://github.com/pandas-dev/pandas into pr…
jbrockmendel Nov 19, 2018
2df7d65
Merge branch 'master' of https://github.com/pandas-dev/pandas into pr…
jbrockmendel Nov 20, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 42 additions & 1 deletion pandas/core/arrays/timedeltas.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# -*- coding: utf-8 -*-
from __future__ import division

from datetime import timedelta
import operator
import warnings

import numpy as np
Expand All @@ -22,11 +25,13 @@
is_datetime64_dtype,
is_list_like,
ensure_int64)
from pandas.core.dtypes.generic import ABCSeries, ABCTimedeltaIndex
from pandas.core.dtypes.generic import (
ABCDataFrame, ABCSeries, ABCTimedeltaIndex, ABCIndexClass)
from pandas.core.dtypes.missing import isna

import pandas.core.common as com
from pandas.core.algorithms import checked_add_with_arr
from pandas.core import ops

from pandas.tseries.offsets import Tick
from pandas.tseries.frequencies import to_offset
Expand Down Expand Up @@ -107,8 +112,29 @@ def wrapper(self, other):
return compat.set_function_name(wrapper, opname, cls)


def _wrap_tdi_op(op):
"""
Instead of re-implementing multiplication/division etc operations
in the Array class, for now we dispatch to the Timedelta implementations.
"""
def method(self, other):
if isinstance(other, (ABCSeries, ABCDataFrame, ABCIndexClass)):
return NotImplemented

from pandas import TimedeltaIndex
obj = TimedeltaIndex(self)
result = op(obj, other)
if is_timedelta64_dtype(result):
return type(self)(result)
return np.array(result)

method.__name__ = '__{name}__'.format(name=op.__name__)
return method


class TimedeltaArrayMixin(dtl.DatetimeLikeArrayMixin):
_typ = "timedeltaarray"
__array_priority__ = 1000

@property
def _box_func(self):
Expand Down Expand Up @@ -288,6 +314,21 @@ def _evaluate_with_timedelta_like(self, other, op):

return NotImplemented

__mul__ = _wrap_tdi_op(operator.mul)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should use the already existing machinery and simply implement this _create_arithmetic_method
and a limited form of:

@classmethod
    def _add_arithmetic_ops(cls):

otherwise you have different styles here than elsewhere.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We use this pattern in plenty of places: nattype, Timedelta, field accessors TDA/DTA/PA, ... Using the classmethod in this context doesn't add anything except a layer of indirection.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right but that'st the point, we should be using this here as this IS using the mixin that implements this. This is creating yet another pattern.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as this IS using the mixin that implements this

No, it isn't. It inherits from ExtensionOpsMixin to use _add_comparison_comps, NOT _add_arithmetic_ops. In fact if were to try to use _add_arithmetic_ops that would be working at cross-purposes with _add_datetimelike_methods, which specifically implements add/sub-like methods.

I'm as big a fan of internal consistency and de-duplication as the next guy, but this isn't the place to make that stand.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jreback ok for now?

(anyway, I think the goal is to rework this a bit at some point, to not wrap TimedeltaIndex here, rather the other way around)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the goal is to rework this a bit at some point, to not wrap TimedeltaIndex here, rather the other way around

Correct

__rmul__ = __mul__
__truediv__ = _wrap_tdi_op(operator.truediv)
__floordiv__ = _wrap_tdi_op(operator.floordiv)
__rfloordiv__ = _wrap_tdi_op(ops.rfloordiv)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how are you implmenting rfloordiv but not rdiv / rtruediv?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very specifically implementing only the methods needed to extend the affected tests. I'll double-check why i couldn't extend tests for rdiv and rtruediv

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jbrockmendel any findings here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAICT we don't have test cases for the other methods, at least not in tests.arithmetic.test_timedelta64. I've scoured the other test files to centralize arithmetic tests, but it's conceivable I've missed some.

We definitely need such tests, but I'd rather do that in a dedicated PR, and would prefer not to implement the methods here without the corresponding tests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We definitely need such tests, but I'd rather do that in a dedicated PR, and would prefer not to implement the methods here without the corresponding tests.

+1


if compat.PY2:
__div__ = __truediv__

# Note: TimedeltaIndex overrides this in call to cls._add_numeric_methods
def __neg__(self):
if self.freq is not None:
return type(self)(-self._data, freq=-self.freq)
return type(self)(-self._data)

# ----------------------------------------------------------------
# Conversion Methods - Vectorized analogues of Timedelta methods

Expand Down
4 changes: 3 additions & 1 deletion pandas/core/indexes/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
ABCSeries, ABCDataFrame,
ABCMultiIndex,
ABCPeriodIndex, ABCTimedeltaIndex, ABCDatetimeIndex,
ABCTimedeltaArray,
ABCDateOffset)
from pandas.core.dtypes.missing import isna, array_equivalent
from pandas.core.dtypes.cast import maybe_cast_to_integer_array
Expand Down Expand Up @@ -123,7 +124,8 @@ def index_arithmetic_method(self, other):
elif isinstance(other, ABCTimedeltaIndex):
# Defer to subclass implementation
return NotImplemented
elif isinstance(other, np.ndarray) and is_timedelta64_dtype(other):
elif (isinstance(other, (np.ndarray, ABCTimedeltaArray)) and
is_timedelta64_dtype(other)):
# GH#22390; wrap in Series for op, this will in turn wrap in
# TimedeltaIndex, but will correctly raise TypeError instead of
# NullFrequencyError for add/sub ops
Expand Down
8 changes: 8 additions & 0 deletions pandas/core/indexes/timedeltas.py
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,14 @@ def _format_native_types(self, na_rep=u'NaT', date_format=None, **kwargs):
# -------------------------------------------------------------------
# Wrapping TimedeltaArray

__mul__ = Index.__mul__
__rmul__ = Index.__rmul__
__truediv__ = Index.__truediv__
__floordiv__ = Index.__floordiv__
__rfloordiv__ = Index.__rfloordiv__
if compat.PY2:
__div__ = Index.__div__

days = wrap_field_accessor(TimedeltaArray.days)
seconds = wrap_field_accessor(TimedeltaArray.seconds)
microseconds = wrap_field_accessor(TimedeltaArray.microseconds)
Expand Down
3 changes: 2 additions & 1 deletion pandas/core/ops.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,8 @@
ABCSeries,
ABCDataFrame, ABCPanel,
ABCIndex, ABCIndexClass,
ABCSparseSeries, ABCSparseArray)
ABCSparseSeries, ABCSparseArray,
ABCTimedeltaArray)


# -----------------------------------------------------------------------------
Expand Down
14 changes: 13 additions & 1 deletion pandas/tests/arithmetic/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@
import pandas as pd

from pandas.compat import long
from pandas.core.arrays import PeriodArray, DatetimeArrayMixin as DatetimeArray
from pandas.core.arrays import (
PeriodArray,
DatetimeArrayMixin as DatetimeArray,
TimedeltaArrayMixin as TimedeltaArray)


@pytest.fixture(params=[1, np.array(1, dtype=np.int64)])
Expand Down Expand Up @@ -189,3 +192,12 @@ def box_with_datetime(request):
Like `box`, but specific to datetime64 for also testing DatetimeArray
"""
return request.param


@pytest.fixture(params=[pd.Index, pd.Series, pd.DataFrame, TimedeltaArray],
ids=lambda x: x.__name__)
def box_with_timedelta(request):
"""
Like `box`, but specific to timedelta64 for also testing TimedeltaArray
"""
return request.param
Loading