Skip to content

Commit 54f51f0

Browse files
author
Sylvain MARIE
committed
Performance improvement in Series.to_sql and DataFrame.to_sql (SQLiteTable): processing time arrays can be up to 65% faster ! (related to pandas-dev#44764)
1 parent d302fe1 commit 54f51f0

File tree

2 files changed

+17
-2
lines changed

2 files changed

+17
-2
lines changed

doc/source/whatsnew/v1.5.0.rst

+3-1
Original file line numberDiff line numberDiff line change
@@ -387,7 +387,9 @@ Performance improvements
387387
- Performance improvement in :class:`DataFrame` and :class:`Series` constructors for extension dtype scalars (:issue:`45854`)
388388
- Performance improvement in :class:`BusinessHour`, ``str`` and ``repr`` are now 4 times faster ! (related to :issue:`44764`)
389389
- Performance improvement in :class:`DatetimeArray` and :class:`DatetimeIndex`: string formatting is now up to 80% faster (as fast as default) when one of the default strftime formats ``"%Y-%m-%d %H:%M:%S"`` or ``"%Y-%m-%d %H:%M:%S.%f"`` is used. (:issue:`44764`)
390-
- Performance improvement in :meth:`Series.to_excel` and :meth:`DataFrame.to_excel` (:class:`ExcelFormatter`): processing dates can be up to 4% faster. (related to :issue:`44764`)
390+
- Performance improvement in :meth:`Series.to_excel` and :meth:`DataFrame.to_excel` (:class:`ExcelFormatter`): processing date cells can be up to 4% faster. (related to
391+
:issue:`44764`)
392+
- Performance improvement in :meth:`Series.to_sql` and :meth:`DataFrame.to_sql` (:class:`SQLiteTable`): processing time arrays can be up to 65% faster ! (related to :issue:`44764`)
391393

392394
.. ---------------------------------------------------------------------------
393395
.. _whatsnew_150.bug_fixes:

pandas/io/sql.py

+14-1
Original file line numberDiff line numberDiff line change
@@ -1836,7 +1836,20 @@ def __init__(self, *args, **kwargs) -> None:
18361836

18371837
# this will transform time(12,34,56,789) into '12:34:56.000789'
18381838
# (this is what sqlalchemy does)
1839-
sqlite3.register_adapter(time, lambda _: _.strftime("%H:%M:%S.%f"))
1839+
def _adapt(dt):
1840+
if dt.tzinfo is None:
1841+
# This is faster than strftime
1842+
return "%02d:%02d:%02d.%06d" % (
1843+
dt.hour,
1844+
dt.minute,
1845+
dt.second,
1846+
dt.microsecond,
1847+
)
1848+
else:
1849+
# Slow TODO we can probably find a way to use string formatting too
1850+
return dt.strftime("%H:%M:%S.%f")
1851+
1852+
sqlite3.register_adapter(time, _adapt)
18401853
super().__init__(*args, **kwargs)
18411854

18421855
def sql_schema(self):

0 commit comments

Comments
 (0)