Skip to content

BUG: Fixed inconsistent offset behavior for series #43784 #48129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 15 commits into from

Conversation

srotondo
Copy link
Contributor

@srotondo srotondo commented Aug 17, 2022

@srotondo
Copy link
Contributor Author

I'm not exactly sure how these extra commits got into this new PR, but I fixed the file conflicts so that those extra changes are gone.

@srotondo
Copy link
Contributor Author

I realize test_shift_across_dst in test_offsets_properties.py fails right now and I am working on a fix for that.

@mroeschke mroeschke added Datetime Datetime data dtype Numeric Operations Arithmetic, Comparison, and Logical operations Frequency DateOffsets labels Aug 22, 2022
@srotondo
Copy link
Contributor Author

There seems to be a test failing called test_reindexing_with_float64_NA_log in test_reindex.py, but I've run this test without my code change, and it ends up failing anyways, so I don't think this change is to blame.

@@ -714,7 +718,10 @@ def _add_offset(self, offset) -> DatetimeArray:
result = DatetimeArray._simple_new(result, dtype=result.dtype)
if self.tz is not None:
# FIXME: tz_localize with non-nano
result = result.tz_localize(self.tz)
if not offset._use_relativedelta and type(offset) == DateOffset:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this only is needed for DateOffset exactly and not its subclasses?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct. Almost all of the subclasses of DateOffset deal with units larger than an hour, so they would not be affected. The only ones that could be realistically affected by this would be BusinessHour and CustomBusinessHour. However, due to the variable nature of the time differences in those classes (eg. adding 1 hour to the very end of a Friday actually ends up adding over 2 whole days to the start of Monday) I think they should not be affected by this change.

Copy link
Member

@mroeschke mroeschke Sep 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this will be robust to users subclassing and defining their own DateOffset

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't thinking about user-created subclasses in my original response, rather I was thinking of the predefined subclasses like YearEnd. I have it specifically for DateOffset and not its subclasses because this change relies on _use_relativedelta, which signifies offsets in units smaller than a day in DateOffset. DateOffset's subclasses have it always set to a default False, which I absolutely do not want this change to act on. Do you think there is a better way to do this?

@srotondo srotondo requested a review from mroeschke September 14, 2022 15:27
- Bug when adding a :class:`DateOffset` to a :class:`Series` would not add the ``nanoseconds`` field (:issue:`47856`)
- Bug in :func:`to_datetime` where ``OutOfBoundsDatetime`` would be thrown even if ``errors=coerce`` if there were more than 50 rows (:issue:`45319`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you undo this reordering?

@srotondo srotondo requested a review from mroeschke September 15, 2022 19:36
@srotondo
Copy link
Contributor Author

@mroeschke I've changed the check to take into account user-created subclasses by removing the type check in the if statement. I've also changed the _use_relativedelta field in the existing subclasses to match their behavior.

@github-actions
Copy link
Contributor

This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this.

@github-actions github-actions bot added the Stale label Oct 18, 2022
@mroeschke
Copy link
Member

Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.

@mroeschke mroeschke closed this Nov 17, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Datetime Datetime data dtype Frequency DateOffsets Numeric Operations Arithmetic, Comparison, and Logical operations Stale
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: DatetimeIndex and Timestamp behave inconsistently with respect to pd.DateOffset
3 participants