Skip to content

implement Timedelta mod, divmod, rmod, rdivmod, fix and test scalar methods #19365

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 19 commits into from
Closed
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions doc/source/timedeltas.rst
Original file line number Diff line number Diff line change
Expand Up @@ -283,6 +283,18 @@ Rounded division (floor-division) of a ``timedelta64[ns]`` Series by a scalar
td // pd.Timedelta(days=3, hours=4)
pd.Timedelta(days=3, hours=4) // td

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a ref tag here

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean a "(:issue:19365)"?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no a section reference

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean something like ".. _timedeltas.divmod:" after line 297?

The mod (%) and divmod operations are defined for ``Timedelta`` when operating with another timedelta-like or with a numeric argument.

.. ipython:: python

pd.Timedelta(hours=37) % datetime.timedelta(hours=2)

# divmod against a timedelta-like returns a pair (int, Timedelta)
divmod(datetime.timedelta(hours=2), pd.Timedelta(minutes=11))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

give a comment or 2 hear, a dense block of code is not very friendly

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just give 2 examples not need to have every case covered

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't show the array case, I find that unfriendly


# divmod against a numeric returns a pair (Timedelta, Timedelta)
pd.Timedelta(hours=25) % 86400000000000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing the divmod here


Attributes
----------

Expand Down
21 changes: 21 additions & 0 deletions doc/source/whatsnew/v0.23.0.txt
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,25 @@ resetting indexes. See the :ref:`Sorting by Indexes and Values
# Sort by 'second' (index) and 'A' (column)
df_multi.sort_values(by=['second', 'A'])

.. _whatsnew_0230.enhancements.timedelta_mod

Timedelta mod method
^^^^^^^^^^^^^^^^^^^^

``mod`` (%) and ``divmod`` operations are now defined on ``Timedelta`` objects when operating with either timedelta-like or with numeric arguments. (:issue:`19365`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can put in a reference to the docs in the timedelta section.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm unclear on what this means.


.. ipython:: python

td = pd.Timedelta(hours=37)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

show the td, then in another ipython block show operations on it.

td

Current Behavior
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you don't need the Current Behavior here (as there isn't any previous)


.. ipython:: python
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is repetetive


td % pd.Timedelta(hours=2)
divmod(td, np.array([2, 3], dtype='timedelta64[h]'))

.. _whatsnew_0230.enhancements.ran_inf:

``.rank()`` handles ``inf`` values when ``NaN`` are present
Expand Down Expand Up @@ -570,6 +589,7 @@ Other API Changes
- Set operations (union, difference...) on :class:`IntervalIndex` with incompatible index types will now raise a ``TypeError`` rather than a ``ValueError`` (:issue:`19329`)
- :class:`DateOffset` objects render more simply, e.g. "<DateOffset: days=1>" instead of "<DateOffset: kwds={'days': 1}>" (:issue:`19403`)
- :func:`pandas.merge` provides a more informative error message when trying to merge on timezone-aware and timezone-naive columns (:issue:`15800`)
- :func:`Timedelta.__mod__`, :func:`Timedelta.__divmod__` now accept timedelta-like and numeric arguments instead of raising ``TypeError`` (:issue:`19365`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are covering this above, don't repeat

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As in remove this line entirely b/c the "Timedelta mod method" section above exists?


.. _whatsnew_0230.deprecations:

Expand Down Expand Up @@ -716,6 +736,7 @@ Datetimelike
- Bug in :func:`to_datetime` where passing an out-of-bounds datetime with ``errors='coerce'`` and ``utc=True`` would raise ``OutOfBoundsDatetime`` instead of parsing to ``NaT`` (:issue:`19612`)
-


Timezones
^^^^^^^^^

Expand Down
60 changes: 50 additions & 10 deletions pandas/_libs/tslibs/timedeltas.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -482,11 +482,14 @@ def _binary_op_method_timedeltalike(op, name):
# the PyDateTime_CheckExact case is for a datetime object that
# is specifically *not* a Timestamp, as the Timestamp case will be
# handled after `_validate_ops_compat` returns False below
from ..tslib import Timestamp
from timestamps import Timestamp
return op(self, Timestamp(other))
# We are implicitly requiring the canonical behavior to be
# defined by Timestamp methods.

elif is_timedelta64_object(other):
return op(self, Timedelta(other))
Copy link
Member

@jorisvandenbossche jorisvandenbossche Feb 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can apply the same idea here? (add the and not is_timedelta64_object(other) on the line below)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How strong is your opinion on this one? I have a slight preference for the explicitness of this ordering over the sometimes-vagueness of a catch-all.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that strong, but the thing is that this will then again go through this method, instead of being run once.
If you want to deal with it explicitly, you can also do something like

elif is_timedelta64_object(other):
    pass  # other coerced to Timedelta at the bottom


elif hasattr(other, 'dtype'):
# nd-array like
if other.dtype.kind not in ['m', 'M']:
Expand Down Expand Up @@ -1044,6 +1047,11 @@ class Timedelta(_Timedelta):
__rsub__ = _binary_op_method_timedeltalike(lambda x, y: y - x, '__rsub__')

def __mul__(self, other):
if is_integer_object(other) or is_float_object(other):
# includes numpy scalars that would otherwise be caught by dtype
# check below
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another option to do this would be if hasattr(other, 'dtype') and not (is_integer_object(other) or is_float_object(other)):
that way you don't need to repeat the Timedelta(other * self.value, unit='ns') twice

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea. I like having if/elif branches that are explicitly mutually explicit and so don't rely on the ordering.

return Timedelta(other * self.value, unit='ns')

if hasattr(other, 'dtype'):
# ndarray-like
return other * self.to_timedelta64()
Expand All @@ -1060,7 +1068,10 @@ class Timedelta(_Timedelta):
__rmul__ = __mul__

def __truediv__(self, other):
if hasattr(other, 'dtype'):
if is_timedelta64_object(other):
return self / Timedelta(other)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment here (and some below as well)


elif hasattr(other, 'dtype'):
return self.to_timedelta64() / other

elif is_integer_object(other) or is_float_object(other):
Expand All @@ -1076,7 +1087,10 @@ class Timedelta(_Timedelta):
return self.value / float(other.value)

def __rtruediv__(self, other):
if hasattr(other, 'dtype'):
if is_timedelta64_object(other):
return Timedelta(other) / self

elif hasattr(other, 'dtype'):
return other / self.to_timedelta64()

elif not _validate_ops_compat(other):
Expand All @@ -1096,25 +1110,28 @@ class Timedelta(_Timedelta):
# just defer
if hasattr(other, '_typ'):
# Series, DataFrame, ...
if other._typ == 'dateoffset' and hasattr(other, 'delta'):
# Tick offset
return self // other.delta
return NotImplemented

elif is_timedelta64_object(other):
return self // Timedelta(other)

elif is_integer_object(other) or is_float_object(other):
return Timedelta(self.value // other, unit='ns')

if hasattr(other, 'dtype'):
if other.dtype.kind == 'm':
# also timedelta-like
return _broadcast_floordiv_td64(self.value, other, _floordiv)
elif other.dtype.kind in ['i', 'u', 'f']:
if other.ndim == 0:
return Timedelta(self.value // other)
else:
return self.to_timedelta64() // other
return self.to_timedelta64() // other
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this not need wrapping in Timedelta?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jorisvandenbossche convinced me that special-casing zero-dim np.ndarrays didn't make much sense.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that’s not the point
it’s returning a np.timedelta64 and not a Timedelta

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, will revert. @jorisvandenbossche if you want to revisit this particular edit, mention it in #17652.


raise TypeError('Invalid dtype {dtype} for '
'{op}'.format(dtype=other.dtype,
op='__floordiv__'))

elif is_integer_object(other) or is_float_object(other):
return Timedelta(self.value // other, unit='ns')

elif not _validate_ops_compat(other):
return NotImplemented

Expand All @@ -1128,8 +1145,14 @@ class Timedelta(_Timedelta):
# just defer
if hasattr(other, '_typ'):
# Series, DataFrame, ...
if other._typ == 'dateoffset' and hasattr(other, 'delta'):
# Tick offset
return other.delta // self
return NotImplemented

elif is_timedelta64_object(other):
return Timedelta(other) // self

if hasattr(other, 'dtype'):
if other.dtype.kind == 'm':
# also timedelta-like
Expand All @@ -1149,6 +1172,23 @@ class Timedelta(_Timedelta):
return np.nan
return other.value // self.value

def __mod__(self, other):
# Naive implementation, room for optimization
return self.__divmod__(other)[1]

def __rmod__(self, other):
# Naive implementation, room for optimization
return self.__rdivmod__(other)[1]

def __divmod__(self, other):
# Naive implementation, room for optimization
div = self // other
return div, self - div * other

def __rdivmod__(self, other):
div = other // self
return div, other - div * self


cdef _floordiv(int64_t value, right):
return value // right
Expand Down
Loading