Skip to content

BUG: Fix to_datetime(errors='coerce') not swallowing all parser exceptions... #28367

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Sep 12, 2019

Conversation

miggec
Copy link
Contributor

@miggec miggec commented Sep 10, 2019

… (#28299)

@miggec miggec changed the title Fix to_datetime(errors='coerce') not swallowing all parser exceptions... BUG: Fix to_datetime(errors='coerce') not swallowing all parser exceptions... Sep 10, 2019
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jreback
Copy link
Contributor

jreback commented Sep 10, 2019

this might solve #25143, #23447, #9107

if you'd have a look (if so pls add a test for whatever this solves)

@jreback jreback added Bug Error Reporting Incorrect or improved errors from pandas Datetime Datetime data dtype labels Sep 10, 2019
@miggec
Copy link
Contributor Author

miggec commented Sep 11, 2019

this might solve #25143, #23447, #9107

if you'd have a look (if so pls add a test for whatever this solves)

Looks like none of these is affected. At least one seems to be handled by a different change

@TomAugspurger TomAugspurger added this to the 1.0 milestone Sep 11, 2019
@jreback
Copy link
Contributor

jreback commented Sep 11, 2019

@squigmig

Looks like none of these is affected. At least one seems to be handled by a different change

care to do a PR with a test for (which?) one

@miggec miggec mentioned this pull request Sep 11, 2019
5 tasks
@miggec
Copy link
Contributor Author

miggec commented Sep 11, 2019

Looks like none of these is affected. At least one seems to be handled by a different change

care to do a PR with a test for (which?) one

sure, have raised #28386

...

Failing checks on this PR seem to be intermittent. What is the preferred way to resolve or force a re-run?

https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=17284
Checks: Setup environment and build pandas is probably the originator of the rest of the failures

Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

CondaHTTPError: HTTP 404 NOT FOUND for url <https://repo.anaconda.com/pkgs/main/linux-64/widgetsnbextension-3.5.1-py37_0.conda>
Elapsed: 00:00.006773
CF-RAY: 5148a6736939e0b2-IAD

An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.

##[error]Bash exited with code '1'.
##[section]Finishing: Setup environment and build pandas```

@TomAugspurger
Copy link
Contributor

TomAugspurger commented Sep 11, 2019 via email

@@ -609,16 +609,17 @@ cpdef array_to_datetime(ndarray[object] values, str errors='raise',
py_dt = parse_datetime_string(val,
dayfirst=dayfirst,
yearfirst=yearfirst)
# If the dateutil parser returned tzinfo, capture it
# to check if all arguments have the same tzinfo
tz = py_dt.utcoffset()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can the utcoffset() call raise? if not, why move this up?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jbrockmendel this is an issue with the datetutil parser, see #28299 (comment)

the issue itself:
dateutil/dateutil#188

@jreback jreback merged commit c64f564 into pandas-dev:master Sep 12, 2019
@jreback
Copy link
Contributor

jreback commented Sep 12, 2019

thanks @squigmig

@jbrockmendel
Copy link
Member

The test this introduced is failing for me locally. Anyone else?

@TomAugspurger
Copy link
Contributor

TomAugspurger commented Sep 12, 2019 via email

@jbrockmendel
Copy link
Member

rebuild your C extensions?

Running python3 setup.py build_ext --inplace everything looked up-to-date. Doing the same after python3 setup.py clean I'm now getting Exception: Cython-generated file 'pandas/_libs/algos.c' not found. Did the pyproject thing or something affect the workflow?

@simonjayhawkins
Copy link
Member

yes i've got the same. on Windows. total rebuild, clean and conda env. but currently have two failures. one is the allowed failure #28125. still investigating but the failing test in pandas/tests/indexes/datetimes/test_tools.py now passes.

@jbrockmendel
Copy link
Member

upgrading cython from 0.29.10 seems to fix the build (tentatively). i might make a PR to give a better error message for that case

proost pushed a commit to proost/pandas that referenced this pull request Dec 19, 2019
proost pushed a commit to proost/pandas that referenced this pull request Dec 19, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Datetime Datetime data dtype Error Reporting Incorrect or improved errors from pandas
Projects
None yet
Development

Successfully merging this pull request may close these issues.

to_datetime(foo, errors='coerce') does not swallow all errors
5 participants