Skip to content

Resample fix dst transition #36264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Sep 12, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v1.2.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,7 @@ Groupby/resample/rolling

- Bug in :meth:`DataFrameGroupBy.count` and :meth:`SeriesGroupBy.sum` returning ``NaN`` for missing categories when grouped on multiple ``Categoricals``. Now returning ``0`` (:issue:`35028`)
- Bug in :meth:`DataFrameGroupBy.apply` that would some times throw an erroneous ``ValueError`` if the grouping axis had duplicate entries (:issue:`16646`)
- Bug in :meth:`DataFrame.resample(...)` that would throw a ``ValueError`` when resampling from "D" to "24H" over a transition into daylight savings time (DST) (:issue:`35219`)
- Bug when combining methods :meth:`DataFrame.groupby` with :meth:`DataFrame.resample` and :meth:`DataFrame.interpolate` raising an ``TypeError`` (:issue:`35325`)
- Bug in :meth:`DataFrameGroupBy.apply` where a non-nuisance grouping column would be dropped from the output columns if another groupby method was called before ``.apply()`` (:issue:`34656`)
- Bug in :meth:`DataFrameGroupby.apply` would drop a :class:`CategoricalIndex` when grouped on. (:issue:`35792`)
Expand Down
6 changes: 5 additions & 1 deletion pandas/core/resample.py
Original file line number Diff line number Diff line change
Expand Up @@ -1087,7 +1087,11 @@ def _upsample(self, method, limit=None, fill_value=None):
res_index = self._adjust_binner_for_upsample(binner)

# if we have the same frequency as our axis, then we are equal sampling
if limit is None and to_offset(ax.inferred_freq) == self.freq:
if (
limit is None
and to_offset(ax.inferred_freq) == self.freq
and len(obj) == len(res_index)
):
result = obj.copy()
result.index = res_index
else:
Expand Down
47 changes: 47 additions & 0 deletions pandas/tests/resample/test_datetime_index.py
Original file line number Diff line number Diff line change
Expand Up @@ -1740,3 +1740,50 @@ def test_resample_apply_product():
columns=["A", "B"],
)
tm.assert_frame_equal(result, expected)


@pytest.mark.parametrize(
"first,last,freq_in,freq_out,exp_last",
[
(
"2020-03-28",
"2020-03-31",
"D",
"24H",
"2020-03-30 01:00",
), # includes transition into DST
(
"2020-03-28",
"2020-10-27",
"D",
"24H",
"2020-10-27 00:00",
), # includes transition into and out of DST
(
"2020-10-25",
"2020-10-27",
"D",
"24H",
"2020-10-26 23:00",
), # includes transition out of DST
(
"2020-03-28",
"2020-03-31",
"24H",
"D",
"2020-03-30 00:00",
), # same as above, but from 24H to D
("2020-03-28", "2020-10-27", "24H", "D", "2020-10-27 00:00"),
("2020-10-25", "2020-10-27", "24H", "D", "2020-10-26 00:00"),
],
)
def test_resample_calendar_day_with_dst(
first: str, last: str, freq_in: str, freq_out: str, exp_last: str
):
# GH 35219
ts = pd.Series(1.0, pd.date_range(first, last, freq=freq_in, tz="Europe/Amsterdam"))
result = ts.resample(freq_out).pad()
expected = pd.Series(
1.0, pd.date_range(first, exp_last, freq=freq_out, tz="Europe/Amsterdam")
)
tm.assert_series_equal(result, expected)