Skip to content

Commit aaf5037

Browse files
authored
BUG: read_csv raising for arrow engine and parse_dates (pandas-dev#53295)
1 parent b684666 commit aaf5037

File tree

3 files changed

+24
-1
lines changed

3 files changed

+24
-1
lines changed

doc/source/whatsnew/v2.0.2.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ Bug fixes
2929
- Bug in :func:`api.interchange.from_dataframe` was returning :class:`DataFrame`'s of incorrect sizes when called on slices (:issue:`52824`)
3030
- Bug in :func:`api.interchange.from_dataframe` was unnecessarily raising on bitmasks (:issue:`49888`)
3131
- Bug in :func:`merge` when merging on datetime columns on different resolutions (:issue:`53200`)
32+
- Bug in :func:`read_csv` raising ``OverflowError`` for ``engine="pyarrow"`` and ``parse_dates`` set (:issue:`53295`)
3233
- Bug in :func:`to_datetime` was inferring format to contain ``"%H"`` instead of ``"%I"`` if date contained "AM" / "PM" tokens (:issue:`53147`)
3334
- Bug in :func:`to_timedelta` was raising ``ValueError`` with ``pandas.NA`` (:issue:`52909`)
3435
- Bug in :meth:`DataFrame.__getitem__` not preserving dtypes for :class:`MultiIndex` partial keys (:issue:`51895`)
@@ -38,7 +39,6 @@ Bug fixes
3839
- Bug in :meth:`Series.rename` not making a lazy copy when Copy-on-Write is enabled when a scalar is passed to it (:issue:`52450`)
3940
- Bug in :meth:`pd.array` raising for ``NumPy`` array and ``pa.large_string`` or ``pa.large_binary`` (:issue:`52590`)
4041

41-
4242
.. ---------------------------------------------------------------------------
4343
.. _whatsnew_202.other:
4444

pandas/io/parsers/base_parser.py

+3
Original file line numberDiff line numberDiff line change
@@ -1137,6 +1137,9 @@ def unpack_if_single_element(arg):
11371137
return arg
11381138

11391139
def converter(*date_cols, col: Hashable):
1140+
if len(date_cols) == 1 and date_cols[0].dtype.kind in "Mm":
1141+
return date_cols[0]
1142+
11401143
if date_parser is lib.no_default:
11411144
strs = parsing.concat_date_cols(date_cols)
11421145
date_fmt = (

pandas/tests/io/parser/test_parse_dates.py

+20
Original file line numberDiff line numberDiff line change
@@ -2217,3 +2217,23 @@ def test_parse_dates_dict_format_index(all_parsers):
22172217
index=Index([Timestamp("2019-12-31"), Timestamp("2020-12-31")], name="a"),
22182218
)
22192219
tm.assert_frame_equal(result, expected)
2220+
2221+
2222+
def test_parse_dates_arrow_engine(all_parsers):
2223+
# GH#53295
2224+
parser = all_parsers
2225+
data = """a,b
2226+
2000-01-01 00:00:00,1
2227+
2000-01-01 00:00:01,1"""
2228+
2229+
result = parser.read_csv(StringIO(data), parse_dates=["a"])
2230+
expected = DataFrame(
2231+
{
2232+
"a": [
2233+
Timestamp("2000-01-01 00:00:00"),
2234+
Timestamp("2000-01-01 00:00:01"),
2235+
],
2236+
"b": 1,
2237+
}
2238+
)
2239+
tm.assert_frame_equal(result, expected)

0 commit comments

Comments
 (0)