Skip to content

Implement roll_monthday, simplify SemiMonthOffset #18762

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Dec 30, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
6100c3f
implement roll_monthday, roll_qtrday, notes on possible bugs
jbrockmendel Dec 13, 2017
aac0832
simplify SemiMonth.apply
jbrockmendel Dec 13, 2017
c5bc5b2
whitespace/comment fixup
jbrockmendel Dec 13, 2017
15b7916
Merge branch 'master' of https://github.com/pandas-dev/pandas into qt…
jbrockmendel Dec 13, 2017
cb07e88
flake8 indentation cleanup
jbrockmendel Dec 13, 2017
eb5e72f
deprivatize and type roll_monthday
jbrockmendel Dec 13, 2017
ec3b24d
docstrings, de-privatize, types
jbrockmendel Dec 13, 2017
c6f025b
avoid while loop[ in BusinessDay.apply
jbrockmendel Dec 13, 2017
f5694de
docstring edits
jbrockmendel Dec 14, 2017
2566bf5
add typing
jbrockmendel Dec 14, 2017
0838271
Merge branch 'master' of https://github.com/pandas-dev/pandas into qt…
jbrockmendel Dec 14, 2017
a649238
dummy commit to force CI
jbrockmendel Dec 14, 2017
425d3b4
Merge branch 'master' of https://github.com/pandas-dev/pandas into qt…
jbrockmendel Dec 15, 2017
71f138d
Separate int and datetime roll funcs
jbrockmendel Dec 16, 2017
091acc2
Merge branch 'master' of https://github.com/pandas-dev/pandas into qt…
jbrockmendel Dec 18, 2017
365cdb8
Merge branch 'master' of https://github.com/pandas-dev/pandas into qt…
jbrockmendel Dec 24, 2017
9993a91
add tests for liboffsets funcs
jbrockmendel Dec 29, 2017
5d773a5
Merge branch 'master' of https://github.com/pandas-dev/pandas into qt…
jbrockmendel Dec 29, 2017
a5d9dee
Merge branch 'master' of https://github.com/pandas-dev/pandas into qt…
jbrockmendel Dec 29, 2017
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 95 additions & 9 deletions pandas/_libs/tslibs/offsets.pyx
Original file line number Diff line number Diff line change
Expand Up @@ -523,11 +523,9 @@ def shift_quarters(int64_t[:] dtindex, int quarters,
n = quarters

months_since = (dts.month - q1start_month) % modby
compare_month = dts.month - months_since
compare_month = compare_month or 12
# compare_day is only relevant for comparison in the case
# where months_since == 0.
compare_day = get_firstbday(dts.year, compare_month)
compare_day = get_firstbday(dts.year, dts.month)

if n <= 0 and (months_since != 0 or
(months_since == 0 and dts.day > compare_day)):
Expand Down Expand Up @@ -556,11 +554,9 @@ def shift_quarters(int64_t[:] dtindex, int quarters,
n = quarters

months_since = (dts.month - q1start_month) % modby
compare_month = dts.month - months_since
compare_month = compare_month or 12
# compare_day is only relevant for comparison in the case
# where months_since == 0.
compare_day = get_lastbday(dts.year, compare_month)
compare_day = get_lastbday(dts.year, dts.month)

if n <= 0 and (months_since != 0 or
(months_since == 0 and dts.day > compare_day)):
Expand Down Expand Up @@ -827,7 +823,55 @@ cpdef int get_day_of_month(datetime other, day_opt) except? -1:
raise ValueError(day_opt)


cpdef int roll_yearday(other, n, month, day_opt='start') except? -1:
cpdef int roll_convention(int other, int n, int compare):
"""
Possibly increment or decrement the number of periods to shift
based on rollforward/rollbackward conventions.

Parameters
----------
other : int, generally the day component of a datetime
n : number of periods to increment, before adjusting for rolling
compare : int, generally the day component of a datetime, in the same
month as the datetime form which `other` was taken.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is very very confusing. you are using this in 2 separate ways. I would prefer 2 functions one for day one for month

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So is the issue with the docstring in roll_convention? or the name? I dont think the function itself can be broken down any further.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this concern be ameliorated if roll_monthday were removed? I expect all of the places where it is used are eventually going to be simplified/fixed to use roll_convention (or something equivalent)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine, but you are using this two differen types so make
roll_monthday_interger, roll_monthday_datetime or whatever. have 1 function do 1 thing.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible the diff is obfuscating the fact that roll_monthday was split several commits back into roll_monthday (which takes datetimes) and roll_convention (which takes ints)?


Returns
-------
n : int number of periods to increment
"""
if n > 0 and other < compare:
n -= 1
elif n <= 0 and other > compare:
# as if rolled forward already
n += 1
return n


cpdef int roll_monthday(datetime other, int n, datetime compare):
"""
Possibly increment or decrement the number of periods to shift
based on rollforward/rollbackward conventions.

Parameters
----------
other : datetime
n : number of periods to increment, before adjusting for rolling
compare : datetime

Returns
-------
n : int number of periods to increment
"""
if n > 0 and other < compare:
n -= 1
elif n <= 0 and other > compare:
# as if rolled forward already
n += 1
return n


cpdef int roll_qtrday(datetime other, int n, int month, object day_opt,
int modby=3) except? -1:
"""
Possibly increment or decrement the number of periods to shift
based on rollforward/rollbackward conventions.
Expand All @@ -836,6 +880,48 @@ cpdef int roll_yearday(other, n, month, day_opt='start') except? -1:
----------
other : datetime or Timestamp
n : number of periods to increment, before adjusting for rolling
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does not match the signature

month : int reference month giving the first month of the year
day_opt : 'start', 'end', 'business_start', 'business_end'
The convention to use in finding the day in a given month against
which to compare for rollforward/rollbackward decisions.
modby : int 3 for quarters, 12 for years

Returns
-------
n : int number of periods to increment
"""
# TODO: Merge this with roll_yearday by setting modby=12 there?
# code de-duplication versus perf hit?
# TODO: with small adjustments this could be used in shift_quarters
months_since = other.month % modby - month % modby

if n > 0:
if months_since < 0 or (months_since == 0 and
other.day < get_day_of_month(other,
day_opt)):
# pretend to roll back if on same month but
# before compare_day
n -= 1
else:
if months_since > 0 or (months_since == 0 and
other.day > get_day_of_month(other,
day_opt)):
# make sure to roll forward, so negate
n += 1
return n


cpdef int roll_yearday(datetime other, int n, int month,
object day_opt) except? -1:
"""
Possibly increment or decrement the number of periods to shift
based on rollforward/rollbackward conventions.

Parameters
----------
other : datetime or Timestamp
n : number of periods to increment, before adjusting for rolling
month : reference month giving the first month of the year
day_opt : 'start', 'end'
'start': returns 1
'end': returns last day of the month
Expand All @@ -846,7 +932,7 @@ cpdef int roll_yearday(other, n, month, day_opt='start') except? -1:

Notes
-----
* Mirrors `roll_check` in tslib.shift_months
* Mirrors `roll_check` in shift_months

Examples
-------
Expand Down Expand Up @@ -888,7 +974,7 @@ cpdef int roll_yearday(other, n, month, day_opt='start') except? -1:
other.day < get_day_of_month(other,
day_opt)):
n -= 1
elif n <= 0:
else:
if other.month > month or (other.month == month and
other.day > get_day_of_month(other,
day_opt)):
Expand Down
91 changes: 91 additions & 0 deletions pandas/tests/tseries/offsets/test_liboffsets.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
from pandas import Timestamp

import pandas._libs.tslibs.offsets as liboffsets
from pandas._libs.tslibs.offsets import roll_qtrday


def test_get_lastbday():
Expand Down Expand Up @@ -95,3 +96,93 @@ def test_roll_yearday():
assert liboffsets.roll_yearday(other, 5, month, day_opt) == 5
assert liboffsets.roll_yearday(other, -7, month, day_opt) == -6
assert liboffsets.roll_yearday(other, 0, month, day_opt) == 1


def test_roll_qtrday():
other = Timestamp(2072, 10, 1, 6, 17, 18) # Saturday
for day_opt in ['start', 'end', 'business_start', 'business_end']:
# as long as (other.month % 3) != (month % 3), day_opt is irrelevant
# the `day_opt` doesn't matter.
month = 5 # (other.month % 3) < (month % 3)
assert roll_qtrday(other, 4, month, day_opt, modby=3) == 3
assert roll_qtrday(other, -3, month, day_opt, modby=3) == -3

month = 3 # (other.month % 3) > (month % 3)
assert roll_qtrday(other, 4, month, day_opt, modby=3) == 4
assert roll_qtrday(other, -3, month, day_opt, modby=3) == -2

month = 2
other = datetime(1999, 5, 31) # Monday
# has (other.month % 3) == (month % 3)

n = 2
assert roll_qtrday(other, n, month, 'start', modby=3) == n
assert roll_qtrday(other, n, month, 'end', modby=3) == n
assert roll_qtrday(other, n, month, 'business_start', modby=3) == n
assert roll_qtrday(other, n, month, 'business_end', modby=3) == n

n = -1
assert roll_qtrday(other, n, month, 'start', modby=3) == n + 1
assert roll_qtrday(other, n, month, 'end', modby=3) == n
assert roll_qtrday(other, n, month, 'business_start', modby=3) == n + 1
assert roll_qtrday(other, n, month, 'business_end', modby=3) == n

other = Timestamp(2072, 10, 1, 6, 17, 18) # Saturday
month = 4 # (other.month % 3) == (month % 3)
n = 2
assert roll_qtrday(other, n, month, 'start', modby=3) == n
assert roll_qtrday(other, n, month, 'end', modby=3) == n - 1
assert roll_qtrday(other, n, month, 'business_start', modby=3) == n - 1
assert roll_qtrday(other, n, month, 'business_end', modby=3) == n - 1

n = -1
assert roll_qtrday(other, n, month, 'start', modby=3) == n
assert roll_qtrday(other, n, month, 'end', modby=3) == n
assert roll_qtrday(other, n, month, 'business_start', modby=3) == n
assert roll_qtrday(other, n, month, 'business_end', modby=3) == n

other = Timestamp(2072, 10, 3, 6, 17, 18) # First businessday
month = 4 # (other.month % 3) == (month % 3)
n = 2
assert roll_qtrday(other, n, month, 'start', modby=3) == n
assert roll_qtrday(other, n, month, 'end', modby=3) == n - 1
assert roll_qtrday(other, n, month, 'business_start', modby=3) == n
assert roll_qtrday(other, n, month, 'business_end', modby=3) == n - 1

n = -1
assert roll_qtrday(other, n, month, 'start', modby=3) == n + 1
assert roll_qtrday(other, n, month, 'end', modby=3) == n
assert roll_qtrday(other, n, month, 'business_start', modby=3) == n
assert roll_qtrday(other, n, month, 'business_end', modby=3) == n


def test_roll_monthday():
other = Timestamp('2017-12-29', tz='US/Pacific')
before = Timestamp('2017-12-01', tz='US/Pacific')
after = Timestamp('2017-12-31', tz='US/Pacific')

n = 42
assert liboffsets.roll_monthday(other, n, other) == n
assert liboffsets.roll_monthday(other, n, before) == n
assert liboffsets.roll_monthday(other, n, after) == n - 1

n = -4
assert liboffsets.roll_monthday(other, n, other) == n
assert liboffsets.roll_monthday(other, n, before) == n + 1
assert liboffsets.roll_monthday(other, n, after) == n


def test_roll_convention():
other = 29
before = 1
after = 31

n = 42
assert liboffsets.roll_convention(other, n, other) == n
assert liboffsets.roll_convention(other, n, before) == n
assert liboffsets.roll_convention(other, n, after) == n - 1

n = -4
assert liboffsets.roll_convention(other, n, other) == n
assert liboffsets.roll_convention(other, n, before) == n + 1
assert liboffsets.roll_convention(other, n, after) == n
Loading