Skip to content

Commit b126638

Browse files
authored
BUG: to_sql fails for pyarrow date dtype (#53854) (#53856)
* BUG: to_sql fails for pyarrow date dtype (#53854) * use ArrowExtensionArray.to_numpy instead of ArrowExtensionArray.astype
1 parent a784fd8 commit b126638

File tree

3 files changed

+13
-4
lines changed

3 files changed

+13
-4
lines changed

doc/source/whatsnew/v2.1.0.rst

+2
Original file line numberDiff line numberDiff line change
@@ -364,12 +364,14 @@ Datetimelike
364364
- :meth:`DatetimeIndex.map` with ``na_action="ignore"`` now works as expected. (:issue:`51644`)
365365
- Bug in :class:`DateOffset` which had inconsistent behavior when multiplying a :class:`DateOffset` object by a constant (:issue:`47953`)
366366
- Bug in :func:`date_range` when ``freq`` was a :class:`DateOffset` with ``nanoseconds`` (:issue:`46877`)
367+
- Bug in :meth:`DataFrame.to_sql` raising ``ValueError`` for pyarrow-backed date like dtypes (:issue:`53854`)
367368
- Bug in :meth:`Timestamp.date`, :meth:`Timestamp.isocalendar`, :meth:`Timestamp.timetuple`, and :meth:`Timestamp.toordinal` were returning incorrect results for inputs outside those supported by the Python standard library's datetime module (:issue:`53668`)
368369
- Bug in :meth:`Timestamp.round` with values close to the implementation bounds returning incorrect results instead of raising ``OutOfBoundsDatetime`` (:issue:`51494`)
369370
- Bug in :meth:`arrays.DatetimeArray.map` and :meth:`DatetimeIndex.map`, where the supplied callable operated array-wise instead of element-wise (:issue:`51977`)
370371
- Bug in constructing a :class:`Series` or :class:`DataFrame` from a datetime or timedelta scalar always inferring nanosecond resolution instead of inferring from the input (:issue:`52212`)
371372
- Bug in parsing datetime strings with weekday but no day e.g. "2023 Sept Thu" incorrectly raising ``AttributeError`` instead of ``ValueError`` (:issue:`52659`)
372373

374+
373375
Timedelta
374376
^^^^^^^^^
375377
- :meth:`TimedeltaIndex.map` with ``na_action="ignore"`` now works as expected (:issue:`51644`)

pandas/io/sql.py

+10-4
Original file line numberDiff line numberDiff line change
@@ -974,10 +974,16 @@ def insert_data(self) -> tuple[list[str], list[np.ndarray]]:
974974
for i, (_, ser) in enumerate(temp.items()):
975975
if ser.dtype.kind == "M":
976976
if isinstance(ser._values, ArrowExtensionArray):
977-
with warnings.catch_warnings():
978-
warnings.filterwarnings("ignore", category=FutureWarning)
979-
# GH#52459 to_pydatetime will return Index[object]
980-
d = np.asarray(ser.dt.to_pydatetime(), dtype=object)
977+
import pyarrow as pa
978+
979+
if pa.types.is_date(ser.dtype.pyarrow_dtype):
980+
# GH#53854 to_pydatetime not supported for pyarrow date dtypes
981+
d = ser._values.to_numpy(dtype=object)
982+
else:
983+
with warnings.catch_warnings():
984+
warnings.filterwarnings("ignore", category=FutureWarning)
985+
# GH#52459 to_pydatetime will return Index[object]
986+
d = np.asarray(ser.dt.to_pydatetime(), dtype=object)
981987
else:
982988
d = ser._values.to_pydatetime()
983989
elif ser.dtype.kind == "m":

pandas/tests/io/test_sql.py

+1
Original file line numberDiff line numberDiff line change
@@ -561,6 +561,7 @@ def test_dataframe_to_sql_arrow_dtypes(conn, request):
561561
"datetime": pd.array(
562562
[datetime(2023, 1, 1)], dtype="timestamp[ns][pyarrow]"
563563
),
564+
"date": pd.array([date(2023, 1, 1)], dtype="date32[day][pyarrow]"),
564565
"timedelta": pd.array([timedelta(1)], dtype="duration[ns][pyarrow]"),
565566
"string": pd.array(["a"], dtype="string[pyarrow]"),
566567
}

0 commit comments

Comments
 (0)