Skip to content

BUG/ENH - base argument no longer ignored in period resample #23941

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 41 commits into from
Dec 14, 2018
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
4fbb93c
ENH - account for base argument in period resample
sds9995 Nov 27, 2018
ac3f301
BUG - closer on perfectly emulating previous behavior
sds9995 Nov 27, 2018
2a72bac
BUG - all original resample tests now pass
sds9995 Nov 28, 2018
cb03d4e
TST - add preliminary tests for pandas period resample with base
sds9995 Nov 28, 2018
4833c97
CLN - pep8 adherence
sds9995 Nov 28, 2018
5a69414
TST - add a couple more tests
sds9995 Nov 28, 2018
5987a2e
TST - parameterize tests
sds9995 Nov 29, 2018
8f3c976
DOC- add whatsnew entry
sds9995 Dec 2, 2018
58a59a6
DOC - add comments and modify whatsnew
sds9995 Dec 4, 2018
6b6d1a9
CLN - minor refactor of label creation and whatsnew
sds9995 Dec 4, 2018
ba10dcf
CLN - pep8 adherence
sds9995 Dec 4, 2018
fb1465e
Merge branch 'master' into bug/period_resample_base
sds9995 Dec 4, 2018
b51841a
BUG - fix cases where period doesnt start at the 0 base of the freq
sds9995 Dec 4, 2018
5afea5c
CLN - remove unneccesary code
sds9995 Dec 4, 2018
5ea4d2c
CLN - pep8 adherence
sds9995 Dec 4, 2018
9df9179
Merge branch 'master' into bug/period_resample_base
sds9995 Dec 5, 2018
ed975ee
Merge branch 'master' into bug/period_resample_base
sds9995 Dec 6, 2018
99e32a7
BUG - reset resample file
sds9995 Dec 6, 2018
126ae7a
BUG - add original changes back in after master reset
sds9995 Dec 6, 2018
bf076de
BUG - add case back in where not start is not onOffset and add tests …
sds9995 Dec 7, 2018
47d7f7b
ENH - allow for edge compensation in _get_range_edges
sds9995 Dec 7, 2018
eb05501
BUG/CLN - offsets.Day(n>2) not properly anchoring dates, and make cod…
sds9995 Dec 7, 2018
f2b0661
TST - update test to reflect more up-to-date assumption of what expec…
sds9995 Dec 8, 2018
0b9a8fd
Merge branch 'master' into bug/period_resample_base
sds9995 Dec 8, 2018
8020acb
CLN - rever changes for other resample fix, to be fixed in another PR
sds9995 Dec 8, 2018
334eb0b
CLN - split up range edge functions for timestamp and period
sds9995 Dec 8, 2018
c14dbce
CLN - pep8 adherence
sds9995 Dec 8, 2018
9bb348a
Merge branch 'master' into bug/period_resample_base
sds9995 Dec 8, 2018
8e07a95
TST/DOC - add tests for get range edges and update whatsnew
sds9995 Dec 10, 2018
1264f36
Merge branch 'master' into bug/period_resample_base
sds9995 Dec 10, 2018
bca4196
Merge branch 'master' into bug/period_resample_base
sds9995 Dec 11, 2018
0a9c7dc
BUG - account for new period diff behavior
sds9995 Dec 11, 2018
cfefa36
TST - add test for bad input to get range edges
sds9995 Dec 12, 2018
93eaab7
TST - add one more test case to bad get_range_edges
sds9995 Dec 12, 2018
e5286f8
DOC - add docstrings
sds9995 Dec 13, 2018
bef9118
TST - remove unneccesary test
sds9995 Dec 13, 2018
9b7d261
DOC - update docstrings
sds9995 Dec 13, 2018
5034455
Merge branch 'master' into bug/period_resample_base
sds9995 Dec 13, 2018
a49d129
CLN - rename offset to bin_shift, so as not to confuse with pd.offsets
sds9995 Dec 13, 2018
8f1e290
DOC - add params and returns sections
sds9995 Dec 13, 2018
c234cac
Merge branch 'master' into bug/period_resample_base
sds9995 Dec 13, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.24.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,7 @@ Other Enhancements
- :meth:`round`, :meth:`ceil`, and meth:`floor` for :class:`DatetimeIndex` and :class:`Timestamp` now support a ``nonexistent`` argument for handling datetimes that are rounded to nonexistent times. See :ref:`timeseries.timezone_nonexistent` (:issue:`22647`)
- :class:`Resampler` now is iterable like :class:`GroupBy` (:issue:`15314`).
- :meth:`Series.resample` and :meth:`DataFrame.resample` have gained the :meth:`Resampler.quantile` (:issue:`15023`).
- :meth:`TimeGrouper._get_period_bins` now accounts for the base argument passed through the :class:`PeriodIndexResampler` (:issue:`23882`)
- :meth:`pandas.core.dtypes.is_list_like` has gained a keyword ``allow_sets`` which is ``True`` by default; if ``False``,
all instances of ``set`` will not be considered "list-like" anymore (:issue:`23061`)
- :meth:`Index.to_frame` now supports overriding column name(s) (:issue:`22580`).
Expand Down
20 changes: 18 additions & 2 deletions pandas/core/resample.py
Original file line number Diff line number Diff line change
Expand Up @@ -1519,9 +1519,24 @@ def _get_period_bins(self, ax):

start = ax.min().asfreq(self.freq, how=self.convention)
end = ax.max().asfreq(self.freq, how='end')
if self.base:
start = start.to_timestamp()
end = end.to_timestamp()

p_start, p_end = _get_range_edges(start,
end,
self.freq,
closed=self.closed,
base=self.base)
i = None if self.freq.onOffset(start) else 1
j = -1 if self.freq.onOffset(end) else None
else:
p_start, p_end = start, end
i = j = None

labels = binner = PeriodIndex(start=start, end=end,
freq=self.freq, name=ax.name)
labels = binner = PeriodIndex(start=p_start, end=p_end,
freq=self.freq,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a lint error

name=ax.name)[slice(i, j)]

i8 = memb.asi8
freq_mult = self.freq.n
Expand All @@ -1531,6 +1546,7 @@ def _get_period_bins(self, ax):
i8_extend = expected_bins_count - (i8[-1] - i8[0])
rng = np.arange(i8[0], i8[-1] + i8_extend, freq_mult)
rng += freq_mult
rng -= ((freq_mult - self.base) % freq_mult)
bins = memb.searchsorted(rng, side='left')

if nat_count > 0:
Expand Down
20 changes: 20 additions & 0 deletions pandas/tests/resample/test_period_index.py
Original file line number Diff line number Diff line change
Expand Up @@ -740,3 +740,23 @@ def test_resample_with_only_nat(self):
expected = DataFrame([], index=expected_index)
result = frame.resample('1s').mean()
assert_frame_equal(result, expected)

@pytest.mark.parametrize('start,end,start_freq,end_freq,base', [
('19910905', '19910909 03:00', 'H', '24H', 10),
('19910905', '19910909 12:00', 'H', '24H', 10),
('19910905 12:00', '19910909 03:00', 'H', '24H', 10),
('19910905 12:00', '19910909 12:00', 'H', '24H', 10),
('19910905', '19910913 06:00', '2H', '24H', 10),
('19910905', '19910905 01:39', 'Min', '5Min', 3),
('19910905', '19910905 03:18', '2Min', '5Min', 3),
])
def test_resample_with_non_zero_base(self, start, end, start_freq,
end_freq, base):
# GH 23882
s = pd.Series(range(100), index=pd.period_range('19910905',
periods=100,
freq=start_freq))
pr = (s.resample(end_freq, base=base).mean().to_timestamp()
.asfreq(end_freq)) # to_timestamp casts 24H -> D
tr = s.to_timestamp().resample(end_freq, base=base).mean()
assert_series_equal(pr, tr)