-
-
Notifications
You must be signed in to change notification settings - Fork 18.5k
BUG: Fix localize_pydatetime using meta datetimes as Timestamp (#25734) #25746
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
2c4ee4d
de72ec6
f585ecd
194caf5
2de4d15
849493c
c773407
27657f1
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -380,9 +380,12 @@ cdef _TSObject convert_datetime_to_tsobject(datetime ts, object tz, | |
obj.value -= int(offset.total_seconds() * 1e9) | ||
|
||
if not PyDateTime_CheckExact(ts): | ||
# datetime instance but not datetime type --> Timestamp | ||
obj.value += ts.nanosecond | ||
obj.dts.ps = ts.nanosecond * 1000 | ||
try: | ||
obj.value += ts.nanosecond | ||
obj.dts.ps = ts.nanosecond * 1000 | ||
except AttributeError: | ||
# probably a subclass of datetime | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. why is this probably? |
||
pass | ||
ArtificialQualia marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
if nanos: | ||
obj.value += nanos | ||
|
@@ -608,8 +611,11 @@ cpdef inline datetime localize_pydatetime(datetime dt, object tz): | |
if tz is None: | ||
return dt | ||
elif not PyDateTime_CheckExact(dt): | ||
# i.e. is a Timestamp | ||
return dt.tz_localize(tz) | ||
try: | ||
return dt.tz_localize(tz) | ||
except AttributeError: | ||
# probably a subclass of datetime | ||
pass | ||
ArtificialQualia marked this conversation as resolved.
Show resolved
Hide resolved
|
||
elif is_utc(tz): | ||
return _localize_pydatetime(dt, tz) | ||
try: | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -5,6 +5,7 @@ | |
from datetime import datetime, timedelta | ||
from itertools import product, starmap | ||
import operator | ||
import sys | ||
import warnings | ||
|
||
import numpy as np | ||
|
@@ -23,6 +24,8 @@ | |
from pandas.core.indexes.datetimes import _to_M8 | ||
import pandas.util.testing as tm | ||
|
||
from pandas.tseries.offsets import CustomBusinessDay | ||
|
||
|
||
def assert_all(obj): | ||
""" | ||
|
@@ -2350,3 +2353,38 @@ def test_shift_months(years, months): | |
for x in dti] | ||
expected = DatetimeIndex(raw) | ||
tm.assert_index_equal(actual, expected) | ||
|
||
|
||
def test_add_with_monkeypatched_datetime(monkeypatch): | ||
# GH 25734 | ||
|
||
class MetaDatetime(type): | ||
@classmethod | ||
def __instancecheck__(self, obj): | ||
return isinstance(obj, datetime) | ||
|
||
class FakeDatetime(MetaDatetime("NewBase", (datetime,), {})): | ||
pass | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. A metaclass is needed in this test case to be able to override |
||
|
||
with monkeypatch.context() as m: | ||
# monkeypatch datetime everywhere | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you expand this comment to describe what |
||
for mod_name, module in list(sys.modules.items()): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. At least some level of monkeypatching is necessary for the original issue to show up. However, this entire |
||
try: | ||
if (mod_name == __name__ or | ||
module.__name__ in ('datetime',)): | ||
continue | ||
module_attributes = dir(module) | ||
except (ImportError, AttributeError, TypeError): | ||
continue | ||
for attribute_name in module_attributes: | ||
try: | ||
attribute_value = getattr(module, attribute_name) | ||
except (ImportError, AttributeError, TypeError): | ||
continue | ||
if id(datetime) == id(attribute_value): | ||
m.setattr(module, attribute_name, FakeDatetime) | ||
|
||
dt = FakeDatetime(2000, 1, 1, tzinfo=pytz.UTC) | ||
result = Timestamp(dt) + CustomBusinessDay() | ||
expected = Timestamp("2000-01-03", tzinfo=pytz.UTC) | ||
assert result == expected |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we don't throw an exception, rather we raise an exception. Can you even be more explicit here, what does this have to do with CBD?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried to scope this description to more what was in the original issue, but really it is due to
freezegun
monkeypatchingdatetime
with a subclass ofdatetime
. A few of theoffsets.py
classes turn things intodatetime
in theirapply
functions. That's why this affectsCustomBusinessDay
and not, for instance,BusinessDay
. After a brief look, a few other classes might be affected by this bug, likeEaster
andFY5253
.However, in writing a custom
datetime
class for testing, another bug was discovered inconvert_datetime_to_tsobject
which prevents any subclasseddatetime
from being converted into aTimestamp
.freezegun
was not affected by this by chance because it happened to implement thenanosecond
property in their customdatetime
class, but that fix has been included in this PR as mentioned in another comment.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ok, so say this affects subclasses of datetime. note btw, we don't provide any compat with this, but as usual will take patches.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The fact that this affects datetime subclasses is already in the release notes, I'm not sure what you would like changed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.