REGR: to_datetime, unique with OOB values #31520
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
TomAugspurger
added
Algos
TomAugspurger
commented
Jan 31, 2020
| - | ||
| - | ||
| - Fixed regression in :meth:`to_datetime` when parsing non-nanosecond resolution datetimes (:issue:`31491`) | ||
| - Fixed bug in :meth:`to_datetime` raising when ``cache=True`` and out-of-bound values are present (:issue:`31491`) |
There was a problem hiding this comment.
This was present in 0.25.3, so technically not a regression.
Do we want to collect fixed regression in a dedicated section? I think we've done that in the past.
TomAugspurger
commented
Jan 31, 2020
| cache_array = _maybe_cache(arg, format, cache, convert_listlike) | ||
| try: | ||
| cache_array = _maybe_cache(arg, format, cache, convert_listlike) | ||
| except tslibs.OutOfBoundsDatetime: |
There was a problem hiding this comment.
I think this is the best we can do here, at least without major surgery. It'll involve double-parsing the input for large arrays with errors='coerce'.
There was a problem hiding this comment.
ok, pls create an issue as this should be handled directly in _maybe_cache itself
jreback
approved these changes
Feb 1, 2020
| @@ -191,6 +192,11 @@ def _reconstruct_data(values, dtype, original): | |||
| if isinstance(original, ABCIndexClass): | |||
| values = values.astype(object, copy=False) | |||
| elif dtype is not None: | |||
There was a problem hiding this comment.
is period an issue here? (just comments, can target master)
| cache_array = _maybe_cache(arg, format, cache, convert_listlike) | ||
| try: | ||
| cache_array = _maybe_cache(arg, format, cache, convert_listlike) | ||
| except tslibs.OutOfBoundsDatetime: |
There was a problem hiding this comment.
ok, pls create an issue as this should be handled directly in _maybe_cache itself
meeseeksmachine
pushed a commit
to meeseeksmachine/pandas
that referenced
this pull request
Feb 1, 2020
Closed
jreback
pushed a commit
that referenced
this pull request
Feb 1, 2020
Co-authored-by: Tom Augspurger <[email protected]>
3 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #31491
This turned up two bugs:
to_datetime(oob_values, cache=True, errors='coerce')would raise. We had tests intended to catch this, but we didn't actually hit it because caching is disabled for inputs less than length 50. We only tested with length 3. I've updated the test to pass that thresholdto_datetimewas raising for non-ns resolution values that aren't OOB for ns resolution. This was a bug inunique.