ENH: Add `milliseconds` field support for `pd.DateOffset` #43598

isVoid · 2021-09-16T04:31:25Z

closes BUG: Lacking milliseconds field in DateOffset #43371
tests added / passed
Ensure all linting tests pass, see here for how to run them
whatsnew entry

jreback · 2021-09-16T22:53:27Z

you would need tests to prove that this works. We don't currently have support for this. Its possible, but non-trivial to add.

isVoid · 2021-09-17T19:13:20Z

@jreback - Thanks for pointing this out. I also noticed there isn't a comprehensive test for offset constructions. Looked into https://github.com/pandas-dev/pandas/blob/26064f0f7ea01607a85b399ee4f3de29a864f338/pandas/tests/tseries/offsets/test_offsets.py and https://github.com/pandas-dev/pandas/blob/master/pandas/tests/tslibs/test_liboffsets.py.

I think we might also want to test binops with Timestamp and millisecond.

jreback

you will have to look thru the tests a bit more, but these don't test very much about this, eg. these test some ops, but these barely test that this works.

pandas/_libs/tslibs/offsets.pyx

pandas/tests/tseries/offsets/test_offsets.py

isVoid · 2021-10-05T01:10:32Z

@jreback The old TestDateOffset seems lacking comprehensive coverage over the various parameters of DateOffset. I've restructured the class - maintaining the corner cases while extending with more coverage over the parameters and operations. Specifically __add__, __radd__, __rsub__, __mul__ and combination of mul and add.

jreback · 2021-10-05T01:20:05Z

hmm look near there thought we had another file (but ok if u made comprehensive)

isVoid · 2021-10-11T23:47:39Z

@jreback , can you point out which file I should look at? I searched around and it looks like each file in offsets/ is very specific to test certain aspect of the offset.

isVoid · 2021-10-11T23:50:29Z

Also, do you have insights to the current failed tests TestDataFrameSortIndex.test_sort_index_inplace in CI? Seems unrelated to this PR.

jreback

test comment.

can you check if the docs need updating https://pandas.pydata.org/pandas-docs/dev/user_guide/timeseries.html#time-date-components or similar

pandas/tests/tseries/offsets/test_offsets.py

jreback · 2021-10-16T17:32:45Z

cc @jbrockmendel if you can have a look.

pandas/tests/tseries/offsets/test_offsets.py

…id/pandas into add_dateoffset_milliseconds

isVoid · 2021-10-20T04:09:26Z

can you check if the docs need updating https://pandas.pydata.org/pandas-docs/dev/user_guide/timeseries.html#time-date-components or similar

@jreback I checked that the time-date component still cannot be accessed with addition from this PR. In the DateOffset section below, there's no mentioning of the parameters for the generic DateOffset object. However, I did noticed in this section there's no mentioning of the usage of replacing certain component of the timestamp with DateOffset, and I made a small addition to the paragraph and the example.

pandas/tests/tseries/offsets/test_offsets.py

jreback · 2022-01-16T18:06:51Z

@isVoid if you can move the note to 1.5 and merge master

…teoffset_milliseconds

isVoid · 2022-01-17T19:53:37Z

@jreback done

pandas/_libs/tslibs/offsets.pyx

jreback · 2022-01-31T00:05:20Z

status here?

…teoffset_milliseconds

isVoid · 2022-01-31T18:04:52Z

@jreback just responded the most recent review message above. (And applied changes).

jreback · 2022-03-06T23:20:03Z

@isVoid if you can merge master and get this passing

…teoffset_milliseconds

isVoid · 2022-03-07T21:40:32Z

@jreback Merged and checks passing.

doc/source/user_guide/timeseries.rst

isVoid · 2022-03-08T21:40:13Z

https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=74993&view=logs&j=beebc4e0-08c3-587a-a9f8-8370dd8c77d0&t=4e13e18d-382c-5754-9d84-b74ae5d60514&l=119

This failure seems unrelated to this PR?

jreback · 2022-03-16T01:11:17Z

@isVoid can you merge master

…et_milliseconds

jreback · 2022-03-18T00:47:51Z

thanks @isVoid for sticking with it!

simonjayhawkins · 2022-03-18T13:12:36Z

pandas/tests/tseries/offsets/test_offsets.py

-
-        assert (self.d + DateOffset(months=2)) == datetime(2008, 3, 2)
-        assert (self.d - DateOffset(months=2)) == datetime(2007, 11, 2)
+    @pytest.mark.parametrize("relativedelta_kwd", list(liboffsets._relativedelta_kwds))


This needs to be list -> sorted to prevent pytest collection errors with pytest-xdist.

Also, for consistency with elsewhere in this module, could maybe also change "relativedelta_kwd" -> "kwd"?

…#43598)

isVoid added 3 commits September 16, 2021 04:20

initial

62918ce

docstrings

a04bf64

Remove unwanted change

b61c14d

jreback added Enhancement Frequency DateOffsets labels Sep 16, 2021

isVoid added 3 commits September 28, 2021 21:30

Add tests

c067d24

add license

d0814a8

Add to whats new section

8e96489

isVoid marked this pull request as ready for review September 29, 2021 16:42

jreback requested changes Oct 3, 2021

View reviewed changes

pandas/_libs/tslibs/offsets.pyx Outdated Show resolved Hide resolved

pandas/tests/tseries/offsets/test_offsets.py Outdated Show resolved Hide resolved

isVoid and others added 4 commits October 5, 2021 01:00

restructure TestDateOffset and extend with more coverage over parameters

a02be79

remove license

255fe23

remove stale comments

b5a42e6

Merge branch 'master' into add_dateoffset_milliseconds

a5dc38e

isVoid requested a review from jreback October 14, 2021 22:25

jreback requested changes Oct 16, 2021

View reviewed changes

pandas/tests/tseries/offsets/test_offsets.py Outdated Show resolved Hide resolved

Merge branch 'master' into add_dateoffset_milliseconds

073c319

jbrockmendel reviewed Oct 16, 2021

View reviewed changes

pandas/tests/tseries/offsets/test_offsets.py Outdated Show resolved Hide resolved

isVoid added 4 commits October 20, 2021 03:06

Removing existing mul tests

4801de3

change test parameter name

dbadf9f

Merge branch 'add_dateoffset_milliseconds' of https://github.com/isVo…

6a98391

…id/pandas into add_dateoffset_milliseconds

Add some docs for replace component usage

30d3403

jbrockmendel reviewed Dec 24, 2021

View reviewed changes

pandas/tests/tseries/offsets/test_offsets.py Outdated Show resolved Hide resolved

xfail instead of skip

99574ea

isVoid requested a review from jbrockmendel December 29, 2021 20:19

isVoid added 2 commits January 17, 2022 19:49

Merge branch 'main' of git://github.com/pandas-dev/pandas into add_da…

b7eeb4d

…teoffset_milliseconds

Move notes to 1.5

e271461

jbrockmendel reviewed Jan 24, 2022

View reviewed changes

pandas/_libs/tslibs/offsets.pyx Outdated Show resolved Hide resolved

isVoid added 2 commits January 31, 2022 17:42

Merge branch 'main' of git://github.com/pandas-dev/pandas into add_da…

3a85b4e

…teoffset_milliseconds

Add alternative to message

b334f69

isVoid requested a review from jbrockmendel January 31, 2022 18:04

Merge branch 'main' of git://github.com/pandas-dev/pandas into add_da…

f9bfadd

…teoffset_milliseconds

mroeschke reviewed Mar 8, 2022

View reviewed changes

doc/source/user_guide/timeseries.rst Outdated Show resolved Hide resolved

Move replacement doc to docstring

0fb51a1

jreback modified the milestones: Contributions Welcome, 1.5 Mar 16, 2022

Merge branch 'main' of github.com:pandas-dev/pandas into add_dateoffs…

40f986a

…et_milliseconds

mroeschke approved these changes Mar 17, 2022

View reviewed changes

jreback approved these changes Mar 18, 2022

View reviewed changes

jreback merged commit 76fa98b into pandas-dev:main Mar 18, 2022

simonjayhawkins reviewed Mar 18, 2022

View reviewed changes

simonjayhawkins mentioned this pull request Mar 18, 2022

TST: fix pytest collection errors with pytest-xdist #46416

Merged

yehoshuadimarsky pushed a commit to yehoshuadimarsky/pandas that referenced this pull request Jul 13, 2022

ENH: Add milliseconds field support for pd.DateOffset (pandas-dev…

5b578b0

…#43598)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Add `milliseconds` field support for `pd.DateOffset` #43598

ENH: Add `milliseconds` field support for `pd.DateOffset` #43598

isVoid commented Sep 16, 2021 •

edited

Loading

jreback commented Sep 16, 2021

isVoid commented Sep 17, 2021

jreback left a comment

isVoid commented Oct 5, 2021 •

edited

Loading

jreback commented Oct 5, 2021

isVoid commented Oct 11, 2021

isVoid commented Oct 11, 2021

jreback left a comment

jreback commented Oct 16, 2021

isVoid commented Oct 20, 2021

jreback commented Jan 16, 2022

isVoid commented Jan 17, 2022

jreback commented Jan 31, 2022

isVoid commented Jan 31, 2022 •

edited

Loading

jreback commented Mar 6, 2022

isVoid commented Mar 7, 2022

isVoid commented Mar 8, 2022

jreback commented Mar 16, 2022

jreback commented Mar 18, 2022

simonjayhawkins Mar 18, 2022 •

edited

Loading

ENH: Add milliseconds field support for pd.DateOffset #43598

ENH: Add milliseconds field support for pd.DateOffset #43598

Conversation

isVoid commented Sep 16, 2021 • edited Loading

jreback commented Sep 16, 2021

isVoid commented Sep 17, 2021

jreback left a comment

Choose a reason for hiding this comment

isVoid commented Oct 5, 2021 • edited Loading

jreback commented Oct 5, 2021

isVoid commented Oct 11, 2021

isVoid commented Oct 11, 2021

jreback left a comment

Choose a reason for hiding this comment

jreback commented Oct 16, 2021

isVoid commented Oct 20, 2021

jreback commented Jan 16, 2022

isVoid commented Jan 17, 2022

jreback commented Jan 31, 2022

isVoid commented Jan 31, 2022 • edited Loading

jreback commented Mar 6, 2022

isVoid commented Mar 7, 2022

isVoid commented Mar 8, 2022

jreback commented Mar 16, 2022

jreback commented Mar 18, 2022

simonjayhawkins Mar 18, 2022 • edited Loading

Choose a reason for hiding this comment

ENH: Add `milliseconds` field support for `pd.DateOffset` #43598

ENH: Add `milliseconds` field support for `pd.DateOffset` #43598

isVoid commented Sep 16, 2021 •

edited

Loading

isVoid commented Oct 5, 2021 •

edited

Loading

isVoid commented Jan 31, 2022 •

edited

Loading

simonjayhawkins Mar 18, 2022 •

edited

Loading