Skip to content

Commit 6cbe941

Browse files
authored
BUG: Fix float32 precision issues in pd.to_datetime (pandas-dev#60510)
* BUG: Fix float32 precision issues in pd.to_datetime * BUG: Add note to whatsnew
1 parent b667fdf commit 6cbe941

File tree

3 files changed

+18
-0
lines changed

3 files changed

+18
-0
lines changed

doc/source/whatsnew/v3.0.0.rst

+1
Original file line numberDiff line numberDiff line change
@@ -626,6 +626,7 @@ Datetimelike
626626
- Bug in :meth:`DatetimeIndex.union` and :meth:`DatetimeIndex.intersection` when ``unit`` was non-nanosecond (:issue:`59036`)
627627
- Bug in :meth:`Series.dt.microsecond` producing incorrect results for pyarrow backed :class:`Series`. (:issue:`59154`)
628628
- Bug in :meth:`to_datetime` not respecting dayfirst if an uncommon date string was passed. (:issue:`58859`)
629+
- Bug in :meth:`to_datetime` on float32 df with year, month, day etc. columns leads to precision issues and incorrect result. (:issue:`60506`)
629630
- Bug in :meth:`to_datetime` reports incorrect index in case of any failure scenario. (:issue:`58298`)
630631
- Bug in :meth:`to_datetime` wrongly converts when ``arg`` is a ``np.datetime64`` object with unit of ``ps``. (:issue:`60341`)
631632
- Bug in setting scalar values with mismatched resolution into arrays with non-nanosecond ``datetime64``, ``timedelta64`` or :class:`DatetimeTZDtype` incorrectly truncating those scalars (:issue:`56410`)

pandas/core/tools/datetimes.py

+5
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@
4444
from pandas.core.dtypes.common import (
4545
ensure_object,
4646
is_float,
47+
is_float_dtype,
4748
is_integer,
4849
is_integer_dtype,
4950
is_list_like,
@@ -1153,6 +1154,10 @@ def coerce(values):
11531154
# we allow coercion to if errors allows
11541155
values = to_numeric(values, errors=errors)
11551156

1157+
# prevent prevision issues in case of float32 # GH#60506
1158+
if is_float_dtype(values.dtype):
1159+
values = values.astype("float64")
1160+
11561161
# prevent overflow in case of int8 or int16
11571162
if is_integer_dtype(values.dtype):
11581163
values = values.astype("int64")

pandas/tests/tools/test_to_datetime.py

+12
Original file line numberDiff line numberDiff line change
@@ -2084,6 +2084,18 @@ def test_dataframe_str_dtype(self, df, cache):
20842084
)
20852085
tm.assert_series_equal(result, expected)
20862086

2087+
def test_dataframe_float32_dtype(self, df, cache):
2088+
# GH#60506
2089+
# coerce to float64
2090+
result = to_datetime(df.astype(np.float32), cache=cache)
2091+
expected = Series(
2092+
[
2093+
Timestamp("20150204 06:58:10.001002003"),
2094+
Timestamp("20160305 07:59:11.001002003"),
2095+
]
2096+
)
2097+
tm.assert_series_equal(result, expected)
2098+
20872099
def test_dataframe_coerce(self, cache):
20882100
# passing coerce
20892101
df2 = DataFrame({"year": [2015, 2016], "month": [2, 20], "day": [4, 5]})

0 commit comments

Comments
 (0)