Skip to content

BUG: List of years (as string) raises UserWarning with to_datetime #57672

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 of 3 tasks
jrmylow opened this issue Feb 29, 2024 · 3 comments · Fixed by #57674
Closed
2 of 3 tasks

BUG: List of years (as string) raises UserWarning with to_datetime #57672

jrmylow opened this issue Feb 29, 2024 · 3 comments · Fixed by #57674
Labels
Bug datetime.date stdlib datetime.date support

Comments

@jrmylow
Copy link
Contributor

jrmylow commented Feb 29, 2024

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

>>> pandas.to_datetime(["2019"])
DatetimeIndex(['2019-01-01'], dtype='datetime64[ns]', freq=None)
>>> pandas.to_datetime(["2030-01-01", "2020-01-01"]) 
DatetimeIndex(['2030-01-01', '2020-01-01'], dtype='datetime64[ns]', freq=None)

>>> pandas.to_datetime(["2019", "2020"]) 
<stdin>:1: UserWarning: Could not infer format, so each element will be parsed individually, falling back to `dateutil`. To ensure parsing is consistent and as-expected, please specify a format.
DatetimeIndex(['2019-01-01', '2020-01-01'], dtype='datetime64[ns]', freq=None)

Issue Description

List of years (as strings) provided to to_datetime raises UserWarning. For some reason, single years and lists of dates do not trigger this warning. This is breaking tests in CI.

Expected Behavior

Expect no userwarning.

Installed Versions

N/A

@jrmylow jrmylow added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Feb 29, 2024
@jrmylow jrmylow changed the title BUG: List of BUG: List of years (as string) raises UserWarning with to_datetime Feb 29, 2024
@jrmylow
Copy link
Contributor Author

jrmylow commented Feb 29, 2024

Update: The calling chain is:

  • to_datetime is called from tests/series/methods/test_nlargest.py
  • Within core/tools/datetimes.py it calls:
    • to_datetime
    • convert_listlike
    • _convert_listlike_datetimes
    • _guess_datetime_format_for_array
    • guess_datetime_format which returns None
  • Over in _libs/tslibs/parsing.pyx:
    • guess_datetime_format
    • dateutil_parse which raises ValueError
    • default.replace(**repl) raises DateParseError

When testing with python:

import datetime as dt
a = dt.datetime.today()
a.replace(**{'year': 2003})

# alternatively
a.replace(year=2003)

Both will raise ValueError: day is out of range for month.

I suspect this may be a weird change in cpython? Let me check there

@jrmylow
Copy link
Contributor Author

jrmylow commented Feb 29, 2024

Update: "year is out of day for month" is raised because today is Feb 29th.

@MarcoGorelli
Copy link
Member

MarcoGorelli commented Feb 29, 2024

For some reason, single years and lists of dates do not trigger this warning.

if there's only a single element, then there's nothing to be inconsistent with

I think this is expected

Update: "year is out of day for month" is raised because today is Feb 29th.

yup, this is the issue, thanks for the triage :)

@rhshadrach rhshadrach added datetime.date stdlib datetime.date support and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Mar 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug datetime.date stdlib datetime.date support
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants