ENH: Support datetime.timezone objects #25065

mroeschke · 2019-01-31T23:47:58Z

tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

Introduced in Python 3.2. First pass is to evaluate what tests fail when included in the tz_aware_fixture

…_object_support

codecov · 2019-02-01T03:55:38Z

Codecov Report

Merging #25065 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #25065   +/-   ##
=======================================
  Coverage   92.37%   92.37%           
=======================================
  Files         166      166           
  Lines       52403    52403           
=======================================
  Hits        48405    48405           
  Misses       3998     3998

Flag	Coverage Δ
#multiple	`90.79% <ø> (ø)`	⬆️
#single	`42.87% <ø> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ea013a2...5076045. Read the comment docs.

codecov · 2019-02-01T03:55:39Z

Codecov Report

Merging #25065 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #25065   +/-   ##
=======================================
  Coverage   91.25%   91.25%           
=======================================
  Files         172      172           
  Lines       52977    52977           
=======================================
  Hits        48342    48342           
  Misses       4635     4635

Flag	Coverage Δ
#multiple	`89.82% <ø> (ø)`	⬆️
#single	`41.74% <ø> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 707c720...1c6165b. Read the comment docs.

jbrockmendel · 2019-02-03T04:07:55Z

pandas/_libs/tslibs/timezones.pyx

@@ -27,7 +28,7 @@ cdef int64_t NPY_NAT = get_nat()
 # ----------------------------------------------------------------------

 cpdef inline bint is_utc(object tz):
-    return tz is UTC or isinstance(tz, _dateutil_tzutc)
+    return tz is UTC or isinstance(tz, _dateutil_tzutc) or tz is timezone.utc


Consider “cdef tzinfo utc_stdlib = timezone.utc” at module level. Much faster lookup for frequently-done check.

Wasn't able to cdef with tzinfo (used object instead). Would it tzinfo have to be cimported?

Would it tzinfo have to be cimported?

Yah, that shouldn't be a problem though. We do it elsewhere.

…_object_support

mroeschke · 2019-02-03T19:15:14Z

Python 2 builds are failing as expected.

By virtue of the tz_fixtures, datetime.timezones are now being tested with

DatetimeIndex & Timestamp & date_range & Series constructor
DatetimeIndex & Timestamp methods
timezone conversion
indexing
comparison and arithmetic ops with arrays & scalars
append/concat & other reshaping methods
NaT ops

jreback · 2019-02-04T13:29:33Z

doc/source/whatsnew/v0.25.0.rst

@@ -20,7 +20,7 @@ Other Enhancements
 ^^^^^^^^^^^^^^^^^^

 - :meth:`Timestamp.replace` now supports the ``fold`` argument to disambiguate DST transition times (:issue:`25017`)
-
+- ``datetime.timezone`` objects are now supported (:issue:`25065`)


can you add a :meth: with a ref to the python docs

add where they are supported

…_object_support

TomAugspurger

LGTM, minor nitpicks if you have a chance.

TomAugspurger · 2019-03-05T03:32:54Z

doc/source/user_guide/timeseries.rst

@@ -2149,12 +2149,9 @@ Time Zone Handling
 ------------------

 pandas provides rich support for working with timestamps in different time
-zones using the ``pytz`` and ``dateutil`` libraries.
+zones using the ``pytz`` and ``dateutil`` libraries or ``datetime.timezone``


I think that :class:datetime.timezone will work.

Looks like there are a few more mentions of what's supported

Reword L272 to include datetime.timezone

Add a daetetime.timezone example in the example ~L2181

TomAugspurger · 2019-03-05T03:33:06Z

doc/source/whatsnew/v0.25.0.rst

@@ -25,7 +25,7 @@ Other Enhancements
 - ``Series.str`` has gained :meth:`Series.str.casefold` method to removes all case distinctions present in a string (:issue:`25405`)
 - :meth:`DataFrame.set_index` now works for instances of ``abc.Iterator``, provided their output is of the same length as the calling frame (:issue:`22484`, :issue:`24984`)
 - :meth:`DatetimeIndex.union` now supports the ``sort`` argument. The behaviour of the sort parameter matches that of :meth:`Index.union` (:issue:`24994`)
-
+- :meth:`datetime.timezone` objects are now supported as arguments to timezone methods and constructors (:issue:`25065`)


meth -> class (not sure if it matters).

TomAugspurger · 2019-03-05T03:34:42Z

pandas/_libs/tslibs/timezones.pyx


 # ----------------------------------------------------------------------

 cpdef inline bint is_utc(object tz):
-    return tz is UTC or isinstance(tz, _dateutil_tzutc)
+    return tz is UTC or isinstance(tz, _dateutil_tzutc) or tz is utc_stdlib


Potential micro optimization: I suspect that tz is utc_stdlib is relatively cheap compared to isinstance(tz, _dateutil_tzutc). Maybe swap the order of those two for a better average case? Not sure if this matters.

…_object_support

TomAugspurger

LGTM. I'd say merge later today if there are no objections.

TomAugspurger · 2019-03-05T14:51:45Z

pandas/_libs/tslibs/timezones.pyx


 # ----------------------------------------------------------------------

 cpdef inline bint is_utc(object tz):
-    return tz is UTC or isinstance(tz, _dateutil_tzutc)
+    return tz is UTC or or tz is utc_stdlib or isinstance(tz, _dateutil_tzutc)


Whoops, an extra or here.

mroeschke · 2019-03-05T19:19:26Z

@TomAugspurger This branch will always fail the PY2 builds though (feature introduced in 3.3). I was thinking we wait until those builds are removed (hopefully soonish/after 0.24.2)

jreback

minor comment, otherwise lgtm.

doc/source/user_guide/timeseries.rst

…_object_support

mroeschke · 2019-03-18T05:23:12Z

This should be good to merge now that the PY2 builds are gone.

jreback · 2019-03-19T23:50:00Z

thanks @mroeschke

* origin/master: DOC: clean bug fix section in whatsnew (pandas-dev#25792) DOC: Fixed PeriodArray api ref (pandas-dev#25526) Move locale code out of tm, into _config (pandas-dev#25757) Unpin pycodestyle (pandas-dev#25789) Add test for rdivmod on EA array (GH23287) (pandas-dev#24047) ENH: Support datetime.timezone objects (pandas-dev#25065) Cython language level 3 (pandas-dev#24538) API: concat on sparse values (pandas-dev#25719) TST: assert_produces_warning works with filterwarnings (pandas-dev#25721) make core.config self-contained (pandas-dev#25613) CLN: replace %s syntax with .format in pandas.io.parsers (pandas-dev#24721) TST: Check pytables<3.5.1 when skipping (pandas-dev#25773) DOC: Fix typo in docstring of DataFrame.memory_usage (pandas-dev#25770) Replace dicts with OrderedDicts in groupby aggregation functions (pandas-dev#25693) TST: Fixturize tests/frame/test_missing.py (pandas-dev#25640) DOC: Improve the docsting of Series.iteritems (pandas-dev#24879) DOC: Fix function name. (pandas-dev#25751) Implementing iso_week_year support for to_datetime (pandas-dev#25541) DOC: clarify corr behaviour when using a callable (pandas-dev#25732) remove unnecessary check_output (pandas-dev#25755) # Conflicts: # doc/source/whatsnew/v0.25.0.rst

* upstream/master: (55 commits) PERF: Improve performance of StataReader (pandas-dev#25780) Speed up tokenizing of a row in csv and xstrtod parsing (pandas-dev#25784) BUG: Fix _binop for operators for serials which has more than one returns (divmod/rdivmod). (pandas-dev#25588) BUG-24971 copying blocks also considers ndim (pandas-dev#25521) CLN: Panel reference from documentation (pandas-dev#25649) ENH: Quoting column names containing spaces with backticks to use them in query and eval. (pandas-dev#24955) BUG: reading windows utf8 filenames in py3.6 (pandas-dev#25769) DOC: clean bug fix section in whatsnew (pandas-dev#25792) DOC: Fixed PeriodArray api ref (pandas-dev#25526) Move locale code out of tm, into _config (pandas-dev#25757) Unpin pycodestyle (pandas-dev#25789) Add test for rdivmod on EA array (GH23287) (pandas-dev#24047) ENH: Support datetime.timezone objects (pandas-dev#25065) Cython language level 3 (pandas-dev#24538) API: concat on sparse values (pandas-dev#25719) TST: assert_produces_warning works with filterwarnings (pandas-dev#25721) make core.config self-contained (pandas-dev#25613) CLN: replace %s syntax with .format in pandas.io.parsers (pandas-dev#24721) TST: Check pytables<3.5.1 when skipping (pandas-dev#25773) DOC: Fix typo in docstring of DataFrame.memory_usage (pandas-dev#25770) ...

Matt Roeschke added 4 commits January 29, 2019 23:54

ENH: Support datetime.timezone objects

8ff8024

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

4ade114

…_object_support

add comment and simplify check

8ff047b

flake8 and modify check

5076045

gfyoung added Enhancement Timezones Timezone data dtype labels Feb 2, 2019

Remove check

f819b26

jbrockmendel reviewed Feb 3, 2019

View reviewed changes

Matt Roeschke added 4 commits February 2, 2019 20:37

cdef timezone.utc

98c3fd2

cannot cdef tzinfo

224fcc9

PY3.5 support

b3634f4

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

2a082ed

…_object_support

mroeschke changed the title ~~[WIP] ENH: Support datetime.timezone objects~~ ENH: Support datetime.timezone objects Feb 3, 2019

Add whatsnew

bfb2fd3

mroeschke added this to the 0.25.0 milestone Feb 3, 2019

jreback requested changes Feb 4, 2019

View reviewed changes

Matt Roeschke added 11 commits February 4, 2019 13:37

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

35b475e

…_object_support

clarify whatsnew

a8f163e

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

07e9b57

…_object_support

update note in timeseries.rst

fcf84ae

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

6d9af30

…_object_support

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

af64d24

…_object_support

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

f052721

…_object_support

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

ac34a1b

…_object_support

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

12e163d

…_object_support

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

a98d1fa

…_object_support

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

c3d01bf

…_object_support

Matt Roeschke added 2 commits February 17, 2019 15:04

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

9f44011

…_object_support

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

6649396

…_object_support

TomAugspurger approved these changes Mar 5, 2019

View reviewed changes

Matt Roeschke added 2 commits March 4, 2019 20:00

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

b8354c2

…_object_support

Address comments

d74673e

TomAugspurger approved these changes Mar 5, 2019

View reviewed changes

TomAugspurger reviewed Mar 5, 2019

View reviewed changes

Remove extra or

c0ad4b4

jreback approved these changes Mar 5, 2019

View reviewed changes

doc/source/user_guide/timeseries.rst Show resolved Hide resolved

Matt Roeschke added 5 commits March 5, 2019 11:43

Add versionadded tag

96e068b

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

67de9c0

…_object_support

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

956822d

…_object_support

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

a843ae1

…_object_support

Merge remote-tracking branch 'upstream/master' into datetime_timezone…

1c6165b

…_object_support

jreback merged commit 9ab270f into pandas-dev:master Mar 19, 2019

mroeschke deleted the datetime_timezone_object_support branch March 19, 2019 23:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Support datetime.timezone objects #25065

ENH: Support datetime.timezone objects #25065

mroeschke commented Jan 31, 2019 •

edited

Loading

codecov bot commented Feb 1, 2019

codecov bot commented Feb 1, 2019 •

edited

Loading

jbrockmendel Feb 3, 2019

mroeschke Feb 3, 2019

jbrockmendel Feb 3, 2019

mroeschke commented Feb 3, 2019

jreback Feb 4, 2019

jreback Feb 4, 2019

TomAugspurger left a comment

TomAugspurger Mar 5, 2019

TomAugspurger Mar 5, 2019

TomAugspurger Mar 5, 2019

TomAugspurger Mar 5, 2019

TomAugspurger left a comment

TomAugspurger Mar 5, 2019

mroeschke commented Mar 5, 2019

jreback left a comment

mroeschke commented Mar 18, 2019

jreback commented Mar 19, 2019

ENH: Support datetime.timezone objects #25065

ENH: Support datetime.timezone objects #25065

Conversation

mroeschke commented Jan 31, 2019 • edited Loading

codecov bot commented Feb 1, 2019

Codecov Report

codecov bot commented Feb 1, 2019 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mroeschke commented Feb 3, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomAugspurger left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomAugspurger left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mroeschke commented Mar 5, 2019

jreback left a comment

Choose a reason for hiding this comment

mroeschke commented Mar 18, 2019

jreback commented Mar 19, 2019

mroeschke commented Jan 31, 2019 •

edited

Loading

codecov bot commented Feb 1, 2019 •

edited

Loading