Skip to content

BUG: Index[object].astype(td64) failing to raise with invalid NAs #45722

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 30, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.5.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -222,6 +222,7 @@ Datetimelike
- Bug in :meth:`DataFrame.quantile` with datetime-like dtypes and no rows incorrectly returning ``float64`` dtype instead of retaining datetime-like dtype (:issue:`41544`)
- Bug in :func:`to_datetime` with sequences of ``np.str_`` objects incorrectly raising (:issue:`32264`)
- Bug in :class:`Timestamp` construction when passing datetime components as positional arguments and ``tzinfo`` as a keyword argument incorrectly raising (:issue:`31929`)
- Bug in :meth:`Index.astype` when casting from object dtype to ``timedelta64[ns]`` dtype incorrectly casting ``np.datetime64("NaT")`` values to ``np.timedelta64("NaT")`` instead of raising (:issue:`45722`)
-

Timedelta
Expand Down
10 changes: 6 additions & 4 deletions pandas/core/dtypes/astype.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
import numpy as np

from pandas._libs import lib
from pandas._libs.tslibs.timedeltas import array_to_timedelta64
from pandas._typing import (
ArrayLike,
DtypeObj,
Expand Down Expand Up @@ -139,7 +140,7 @@ def astype_nansafe(
elif is_object_dtype(arr.dtype):

# work around NumPy brokenness, #1987
if np.issubdtype(dtype.type, np.integer):
if dtype.kind in ["i", "u"]:
return lib.astype_intsafe(arr, dtype)

# if we have a datetime/timedelta array of objects
Expand All @@ -154,9 +155,10 @@ def astype_nansafe(
copy=copy,
)
elif is_timedelta64_dtype(dtype):
from pandas import to_timedelta

return astype_nansafe(to_timedelta(arr)._values, dtype, copy=copy)
# bc we know arr.dtype == object, this is equivalent to
# `np.asarray(to_timedelta(arr))`, but using a lower-level API that
# does not require a circular import.
return array_to_timedelta64(arr).view("m8[ns]").astype(dtype, copy=False)

if dtype.name in ("datetime64", "timedelta64"):
msg = (
Expand Down
16 changes: 15 additions & 1 deletion pandas/tests/indexes/object/test_astype.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
from pandas import Index
import pytest

from pandas import (
Index,
NaT,
)
import pandas._testing as tm


Expand All @@ -8,3 +13,12 @@ def test_astype_str_from_bytes():
result = idx.astype(str)
expected = Index(["あ", "a"], dtype="object")
tm.assert_index_equal(result, expected)


def test_astype_invalid_nas_to_tdt64_raises():
# GH#45722 don't cast np.datetime64 NaTs to timedelta64 NaT
idx = Index([NaT.asm8] * 2, dtype=object)

msg = r"Cannot cast Index to dtype timedelta64\[ns\]"
with pytest.raises(TypeError, match=msg):
idx.astype("m8[ns]")