Skip to content

BUG: Timestamp(pd.NA) raising TypeError #54102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jul 17, 2023
Merged

BUG: Timestamp(pd.NA) raising TypeError #54102

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
bd0aa06
BUG: Timestamp(pd.NA) raising TypeError
lukemanley Jul 13, 2023
9064762
Update pandas/tests/scalar/timestamp/test_constructors.py
MarcoGorelli Jul 13, 2023
96ce267
Merge remote-tracking branch 'upstream/main' into timestamp-ctor-na
lukemanley Jul 13, 2023
718d810
move check to conversion.pyx
lukemanley Jul 13, 2023
616a55a
Merge branch 'timestamp-ctor-na' of https://github.com/lukemanley/pan…
lukemanley Jul 13, 2023
8717ffe
Merge remote-tracking branch 'upstream/main' into timestamp-ctor-na
lukemanley Jul 16, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v2.1.0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -393,6 +393,7 @@ Datetimelike
- Bug in :meth:`Timestamp.round` with values close to the implementation bounds returning incorrect results instead of raising ``OutOfBoundsDatetime`` (:issue:`51494`)
- Bug in :meth:`arrays.DatetimeArray.map` and :meth:`DatetimeIndex.map`, where the supplied callable operated array-wise instead of element-wise (:issue:`51977`)
- Bug in constructing a :class:`Series` or :class:`DataFrame` from a datetime or timedelta scalar always inferring nanosecond resolution instead of inferring from the input (:issue:`52212`)
- Bug in constructing a :class:`Timestamp` with ``ts_input=pd.NA`` raising ``TypeError`` (:issue:`45481`)
- Bug in parsing datetime strings with weekday but no day e.g. "2023 Sept Thu" incorrectly raising ``AttributeError`` instead of ``ValueError`` (:issue:`52659`)

Timedelta
Expand Down
4 changes: 4 additions & 0 deletions pandas/_libs/tslibs/timestamps.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ import_datetime()

import datetime as dt

from pandas._libs.missing cimport checknull_with_nat_and_na
from pandas._libs.tslibs cimport ccalendar
from pandas._libs.tslibs.base cimport ABCTimestamp

Expand Down Expand Up @@ -1881,6 +1882,9 @@ class Timestamp(_Timestamp):
hour or 0, minute or 0, second or 0, fold=fold or 0)
unit = None

elif checknull_with_nat_and_na(ts_input):
return NaT
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can the

        if ts.value == NPY_NAT:
            return NaT

be removed from below now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't look like it can be removed. Just tried it and this starts to raise if its removed:

pd.Timestamp(np.datetime64("NaT"))

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah right, thanks, checknull checks for pandas NaT, not numpy NaT

does it work to do the check a bit deeper? i.e.

diff --git a/pandas/_libs/tslibs/conversion.pyx b/pandas/_libs/tslibs/conversion.pyx
index 57b7754b08..4d494d2282 100644
--- a/pandas/_libs/tslibs/conversion.pyx
+++ b/pandas/_libs/tslibs/conversion.pyx
@@ -1,4 +1,5 @@
 import numpy as np
+from pandas._libs.missing cimport checknull_with_nat_and_na
 
 cimport numpy as cnp
 from libc.math cimport log10
@@ -268,7 +269,7 @@ cdef _TSObject convert_to_tsobject(object ts, tzinfo tz, str unit,
     if isinstance(ts, str):
         return convert_str_to_tsobject(ts, tz, unit, dayfirst, yearfirst)
 
-    if ts is None or ts is NaT:
+    if checknull_with_nat_and_na(ts):
         obj.value = NPY_NAT
     elif is_datetime64_object(ts):
         reso = get_supported_reso(get_datetime64_unit(ts))
diff --git a/pandas/_libs/tslibs/timestamps.pyx b/pandas/_libs/tslibs/timestamps.pyx
index 80b7c2d2f5..59640a3ef4 100644
--- a/pandas/_libs/tslibs/timestamps.pyx
+++ b/pandas/_libs/tslibs/timestamps.pyx
@@ -1882,9 +1882,6 @@ class Timestamp(_Timestamp):
                                 hour or 0, minute or 0, second or 0, fold=fold or 0)
             unit = None
 
-        elif checknull_with_nat_and_na(ts_input):
-            return NaT
-
         if getattr(ts_input, "tzinfo", None) is not None and tz is not None:
             raise ValueError("Cannot pass a datetime or Timestamp with tzinfo with "
                              "the tz parameter. Use tz_convert instead.")

?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes - updated. thanks


if getattr(ts_input, "tzinfo", None) is not None and tz is not None:
raise ValueError("Cannot pass a datetime or Timestamp with tzinfo with "
"the tz parameter. Use tz_convert instead.")
Expand Down
9 changes: 9 additions & 0 deletions pandas/tests/scalar/timestamp/test_constructors.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@
from pandas.errors import OutOfBoundsDatetime

from pandas import (
NA,
NaT,
Period,
Timedelta,
Timestamp,
Expand Down Expand Up @@ -893,3 +895,10 @@ def test_timestamp_constructor_adjust_value_for_fold(tz, ts_input, fold, value_o
result = ts._value
expected = value_out
assert result == expected


@pytest.mark.parametrize("na_value", [None, np.nan, NA])
def test_timestamp_constructor_na_value(na_value):
result = Timestamp(na_value)
expected = NaT
assert result is expected