Skip to content

BUG: NaT comparison with numpy array doesn't work element-wise #40722

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jorisvandenbossche opened this issue Apr 1, 2021 · 2 comments · Fixed by #40723
Closed

BUG: NaT comparison with numpy array doesn't work element-wise #40722

jorisvandenbossche opened this issue Apr 1, 2021 · 2 comments · Fixed by #40723
Labels
Datetime Datetime data dtype Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Numeric Operations Arithmetic, Comparison, and Logical operations Regression Functionality that used to work in a prior pandas version
Milestone

Comments

@jorisvandenbossche
Copy link
Member

I noticed the following behaviour:

In [43]: np.array([1, 2, 3]) == pd.Timedelta("1 days")
Out[43]: array([False, False, False])

In [44]: np.array([1, 2, 3]) == pd.Timedelta("nat")
Out[44]: False

@jbrockmendel do you know if this is something we control? (it's similar as the numeric/string comparison with numpy, but here the scalar is one defined by pandas)

@jorisvandenbossche jorisvandenbossche added Datetime Datetime data dtype Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Numeric Operations Arithmetic, Comparison, and Logical operations labels Apr 1, 2021
@jorisvandenbossche
Copy link
Member Author

Comparing the richcmp implementations of NaT and Timedelta:

elif util.is_array(other):
# TODO: watch out for zero-dim
if other.dtype.kind == "m":
return PyObject_RichCompare(self.asm8, other, op)
elif other.dtype.kind == "O":
# operate element-wise
return np.array(
[PyObject_RichCompare(self, x, op) for x in other],
dtype=bool,
)
if op == Py_EQ:
return np.zeros(other.shape, dtype=bool)
elif op == Py_NE:
return np.ones(other.shape, dtype=bool)
return NotImplemented # let other raise TypeError

elif util.is_array(other):
if other.dtype.kind in "mM":
result = np.empty(other.shape, dtype=np.bool_)
result.fill(_nat_scalar_rules[op])
elif other.dtype.kind == "O":
result = np.array([PyObject_RichCompare(self, x, op) for x in other])
else:
return NotImplemented
return result

We indeed explicitly cover this case for Timedelta, but not for NaTType

@jorisvandenbossche
Copy link
Member Author

jorisvandenbossche commented Apr 1, 2021

This appears to be a regression in 1.2.x

@jorisvandenbossche jorisvandenbossche added the Regression Functionality that used to work in a prior pandas version label Apr 1, 2021
@jorisvandenbossche jorisvandenbossche added this to the 1.2.4 milestone Apr 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Datetime Datetime data dtype Missing-data np.nan, pd.NaT, pd.NA, dropna, isnull, interpolate Numeric Operations Arithmetic, Comparison, and Logical operations Regression Functionality that used to work in a prior pandas version
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant