Skip to content

Commit e87c458

Browse files
Debian Science Teamrebecca-palmer
Debian Science Team
authored andcommitted
float-to-datetime conversion fixes
Avoid assuming that NaN casts to NaT (= fails on riscv64/hppa ?) Don't round to int for the bounds check when we don't for the real conversion (wrong near the bounds, and maybe also a waste of time) Author: Rebecca N. Palmer <[email protected]> Forwarded: pandas-dev/pandas#50183 Gbp-Pq: Name float_to_datetime.patch
1 parent 6dd81bc commit e87c458

File tree

1 file changed

+10
-6
lines changed

1 file changed

+10
-6
lines changed

pandas/_libs/tslib.pyx

+10-6
Original file line numberDiff line numberDiff line change
@@ -296,12 +296,18 @@ def array_with_unit_to_datetime(
296296
# if we have nulls that are not type-compat
297297
# then need to iterate
298298

299-
if values.dtype.kind in ["i", "f", "u"]:
299+
if values.dtype.kind in ["i", "u"]:
300300
iresult = values.astype("i8", copy=False)
301301
# fill missing values by comparing to NPY_NAT
302302
mask = iresult == NPY_NAT
303303
iresult[mask] = 0
304-
fvalues = iresult.astype("f8") * mult
304+
fvalues = iresult.astype("f8") * mult # TODO as this is only used for bounds checking, would it be more efficient to divide the bounds by mult? (don't just use integer iresult * mult, we need arithmetic overflow to be an error not a wraparound)
305+
need_to_iterate = False
306+
307+
if values.dtype.kind in ["f",]:
308+
mask = (values != values) | (values == NPY_NAT) # first is NaNs
309+
fvalues = (values * mult).astype("f8") # TODO would values.astype('f8')*mult have less rounding error? (or does this need to support longer-than-double floats where it's worse?) would copy=False be faster?
310+
fvalues[mask] = 0
305311
need_to_iterate = False
306312

307313
if not need_to_iterate:
@@ -315,11 +321,9 @@ def array_with_unit_to_datetime(
315321
result = (iresult * mult).astype("M8[ns]")
316322

317323
elif values.dtype.kind == "f":
318-
fresult = (values * mult).astype("f8")
319-
fresult[mask] = 0
320324
if prec:
321-
fresult = round(fresult, prec)
322-
result = fresult.astype("M8[ns]", copy=False)
325+
fvalues = round(fvalues, prec) # TODO why does this exist, it looks like prec's always 0 (was p from precision_to_unit meant to be prec?)
326+
result = fvalues.astype("M8[ns]", copy=False)
323327

324328
iresult = result.view("i8")
325329
iresult[mask] = NPY_NAT

0 commit comments

Comments
 (0)