Skip to content

BUG: Offset-based rolling window, with only one raw in dataframe and closed='left', max and min functions make python crash #24811

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 18, 2019
Merged

BUG: Offset-based rolling window, with only one raw in dataframe and closed='left', max and min functions make python crash #24811

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
417ca64
BUG: GH24718 fixed
jwu-pb Jan 17, 2019
e394748
BUG: closes GH24718
jwu-pb Jan 17, 2019
2e513db
BUG:GH24718 remove the unnecessary parens in pytest.mark.parametrize
jwu-pb Jan 17, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v0.24.0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1808,6 +1808,7 @@ Plotting
Groupby/Resample/Rolling
^^^^^^^^^^^^^^^^^^^^^^^^

- Bug in :func:`pandas.core.window.Rolling.min` and :func:`pandas.core.window.Rolling.max` with ``closed='left'``, a datetime-like index and only one entry in the series leading to segfault (:issue:`24718`)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this truly this specific? Off the top of my head seems strange that this would only be applicable to date time like indices

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The datetime-like index makes the program going into the function _roll_min_max_variable of window.pyx, where the bug is. I am not very sure if there is any other case triggering the function. But the program normally goes into _roll_min_max_fixed if it isn't a datetime-like index.

- Bug in :func:`pandas.core.groupby.GroupBy.first` and :func:`pandas.core.groupby.GroupBy.last` with ``as_index=False`` leading to the loss of timezone information (:issue:`15884`)
- Bug in :meth:`DateFrame.resample` when downsampling across a DST boundary (:issue:`8531`)
- Bug in date anchoring for :meth:`DateFrame.resample` with offset :class:`Day` when n > 1 (:issue:`24127`)
Expand Down
5 changes: 4 additions & 1 deletion pandas/_libs/window.pyx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1339,7 +1339,10 @@ cdef _roll_min_max_variable(ndarray[numeric] values,
Q.push_back(i)
W.push_back(i)

output[N-1] = calc_mm(minp, nobs, values[Q.front()])
if not Q.empty():
output[N-1] = calc_mm(minp, nobs, values[Q.front()])
else:
output[N-1] = NaN

return output

Expand Down
20 changes: 20 additions & 0 deletions pandas/tests/test_window.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -520,6 +520,26 @@ def test_closed(self):
with pytest.raises(ValueError):
df.rolling(window=3, closed='neither')

@pytest.mark.parametrize("func", [('min'), ('max')])
def test_closed_one_entry(self, func):
# GH24718
ser = pd.Series(data=[2], index=pd.date_range('2000', periods=1))
result = getattr(ser.rolling('10D', closed='left'), func)()
tm.assert_series_equal(result, pd.Series([np.nan], index=ser.index))

@pytest.mark.parametrize("func", [('min'), ('max')])
def test_closed_one_entry_groupby(self, func):
# GH24718
ser = pd.DataFrame(data={'A': [1, 1, 2], 'B': [3, 2, 1]},
index=pd.date_range('2000', periods=3))
result = getattr(
ser.groupby('A', sort=False)['B'].rolling('10D', closed='left'),
func)()
exp_idx = pd.MultiIndex.from_arrays(arrays=[[1, 1, 2], ser.index],
names=('A', None))
expected = pd.Series(data=[np.nan, 3, np.nan], index=exp_idx, name='B')
tm.assert_series_equal(result, expected)

@pytest.mark.parametrize("input_dtype", ['int', 'float'])
@pytest.mark.parametrize("func,closed,expected", [
('min', 'right', [0.0, 0, 0, 1, 2, 3, 4, 5, 6, 7]),
Expand Down