Skip to content

BUG: to_datetime with out-of-bounds np.datetime64 and errors='ignore' doesn't return input #50992

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Feb 2, 2023

Conversation

MarcoGorelli
Copy link
Member

@mroeschke mroeschke added the Datetime Datetime data dtype label Jan 26, 2023
@@ -982,6 +982,7 @@ Datetimelike
- Bug in :func:`to_datetime` was raising ``ValueError`` when parsing mixed-offset :class:`Timestamp` with ``errors='ignore'`` (:issue:`50585`)
- Bug in :func:`to_datetime` was incorrectly handling floating-point inputs within 1 ``unit`` of the overflow boundaries (:issue:`50183`)
- Bug in :func:`to_datetime` with unit of "Y" or "M" giving incorrect results, not matching pointwise :class:`Timestamp` results (:issue:`50870`)
- Bug in :func:`to_datetime` was not returning input with ``errors='ignore'`` when passed out-of-bounds np.datetime64 (:issue:`50587`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this sounds like it is just np.datetime64 objects, but i think it can also be strings or ints/floats

@@ -1084,7 +1084,7 @@ def test_to_datetime_array_of_dt64s(self, cache, unit):
# numpy is either a python datetime.datetime or datetime.date
tm.assert_index_equal(
to_datetime(dts_with_oob, errors="ignore", cache=cache),
Index([dt.item() for dt in dts_with_oob]),
Index(dts_with_oob),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you write a small dedicated test for this. iirc some of the non-nano stuff is liable to change this test and i dont want to accidentally get rid of coverage for this behavior

@jbrockmendel
Copy link
Member

I'm on board with this change, should probably make sure others agree

@MarcoGorelli
Copy link
Member Author

I'm on board with this change, should probably make sure others agree

sure - @mroeschke any objections? Anyone I should ping who might be opinionated on this?

Copy link
Member

@mroeschke mroeschke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Guessing we have tests for the errors='coerce' behavior?

@MarcoGorelli
Copy link
Member Author

thanks! and yes, errors='coerce' with oob is covered, e.g.

tm.assert_index_equal(
to_datetime(dts_with_oob, errors="coerce", cache=cache),
DatetimeIndex(
[Timestamp(dts_with_oob[0]).asm8, Timestamp(dts_with_oob[1]).asm8] * 30
+ [NaT],
),
)

@MarcoGorelli MarcoGorelli added this to the 2.0 milestone Feb 2, 2023
@mroeschke mroeschke merged commit 24a9073 into pandas-dev:main Feb 2, 2023
@mroeschke
Copy link
Member

Thanks @MarcoGorelli (failure unrelated)

@jbrockmendel
Copy link
Member

jbrockmendel commented Feb 2, 2023

nice!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Datetime Datetime data dtype
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: to_datetime with out-of-bounds np.datetime64 and errors='ignore' doesn't return input
3 participants