Skip to content

ENH: gb.is_monotonic_increasing #17015 #17453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
d2759d5
ENH: gb.is_monotonic_increasing #17015 fix rebase conflicts
No-Stream Oct 31, 2017
4c70dae
ENH: gb.is_monotonic_increasing #17015 fix rebase conflicts
No-Stream Oct 31, 2017
d88acdd
parametrized tests for gb.is_monotonic_increasing/decreasing
No-Stream Sep 8, 2017
53e5a2b
ENH: gb.is_monotonic_increasing, is_monotonic_decreasing #17015
No-Stream Sep 6, 2017
5a30ee4
added tests for gb.is_monotonically_increasing()/decreasing
No-Stream Sep 7, 2017
2e4bb15
parametrized tests for gb.is_monotonic_increasing/decreasing
No-Stream Sep 8, 2017
3122f1f
removed edits to whatsnew 0.21.0
No-Stream Oct 31, 2017
a6d0640
ENH: gb.is_monotonic_increasing #17015 fix rebase conflicts
No-Stream Oct 31, 2017
4ecc479
ENH: gb.is_monotonic_increasing #17015 fix rebase conflicts
No-Stream Oct 31, 2017
ea42697
rebase and cleanup
jreback Nov 1, 2017
ffb6200
DOC: add docstring for MultiIndex.fillna (#18018) (#18269)
ghasemnaddaf Nov 13, 2017
ee9eb01
ENH: gb.is_monotonic_increasing #17015 fix merge/rebase conflicts
No-Stream Nov 14, 2017
ee9fa7e
ENH: gb.is_monotonic_increasing #17015 tests revised and passing
No-Stream Nov 14, 2017
2c3002f
ENH: gb.is_monotonic_increasing #17015 merged to current upstream/master
No-Stream Nov 29, 2017
f24e476
ENH: gb.is_monotonic_increasing #17015 minor fixes for @jreback
No-Stream Dec 11, 2017
d01b62e
ENH: gb.is_monotonic_increasing #17015 rebase to master
No-Stream Dec 12, 2017
db3e6c0
ENH: gb.is_monotonic_increasing #17015 fix changed whatsnew
No-Stream Dec 12, 2017
fb0bc3c
ENH: gb.is_monotonic_increasing #17015 fix 22.0 whatsnew
No-Stream Dec 12, 2017
e5a90fa
NH: gb.is_monotonic_increasing #17015 alternate version to remove fun…
No-Stream Dec 15, 2017
1a15209
ENH: gb.is_monotonic_increasing #17015 rebase to master
No-Stream Dec 15, 2017
3c13163
Merge branch 'master' into PR_TOOL_MERGE_PR_17453
jreback Jan 5, 2018
d747d35
fixup whatsnew
jreback Jan 5, 2018
dd9a7fc
Resolve v0.23.0.txt merge conflict
gfyoung Feb 11, 2018
495718c
Merge branch 'master' into is-monotonic-increasing-gb-method
gfyoung Feb 11, 2018
02d4369
typo
jorisvandenbossche Feb 11, 2018
13d4dd9
Add ticks for function rendering
gfyoung Feb 12, 2018
787c436
Make whatsnew statement more concise
gfyoung Feb 12, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions doc/source/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2052,6 +2052,8 @@ The following methods are available only for ``SeriesGroupBy`` objects.
SeriesGroupBy.nunique
SeriesGroupBy.unique
SeriesGroupBy.value_counts
SeriesGroupBy.is_monotonic_increasing
SeriesGroupBy.is_monotonic_decreasing

The following methods are available only for ``DataFrameGroupBy`` objects.

Expand Down
2 changes: 1 addition & 1 deletion doc/source/whatsnew/v0.21.1.txt
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Other Enhancements
^^^^^^^^^^^^^^^^^^

- :meth:`Timestamp.timestamp` is now available in Python 2.7. (:issue:`17329`)
-
- :func: groupby.is_monotonic_increasing and .is_monotonic_decreasing extend Series.is_monotonic_increasing to groups, returning whether each group is monotonically increasing or decreasing, respectively. (:issue:`17015`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can add :func: where needed here

-

.. _whatsnew_0211.deprecations:
Expand Down
49 changes: 49 additions & 0 deletions pandas/core/groupby.py
Original file line number Diff line number Diff line change
Expand Up @@ -1743,6 +1743,55 @@ def pipe(self, func, *args, **kwargs):
"""
return _pipe(self, func, *args, **kwargs)

@Substitution(name='groupby')
@Appender(_doc_template)
def is_monotonic_increasing(self):
"""
Returns whether each group is monotonically increasing.

Equivalent to ``.apply(lambda x: x.is_monotonic_increasing)``.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a .. versionadded:: 0.22.0 line here (and in the other as well)?

I also think we should keep this for 0.22.0, as it is a new feature, 0.21.1 is for bug fixes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jorisvandenbossche thanks for the feedback. I've gone ahead and fixed the whatsnew references and added the .. versionadded:: 0.22.0 on my local branch.

I wanted to clarify the _series_apply_whitelist thing. It appears there's a long list of methods in _common_apply_whitelist - you'd want me to add the new methods there? And would there need to be any other changes?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need to add it to _series_apply_whitelist, as it is only a Series method, not a DataFrame method. If you add that, you can remove the def is_monotonic_.. functions/

Examples
--------
>>> source_dict = {
... 'A': ['this', 'col', 'is', 'entirely', 'irrelevant', '.'],
... 'B': ['cat_a', 'cat_a', 'cat_a', 'cat_b', 'cat_b', 'cat_b'],
... 'C': [1, 2, 3, 2, 2, 0]}

>>> df = pd.DataFrame(source_dict)
... df.groupby(['B']).C.is_monotonic_increasing()
B
cat_a True
cat_b False
Name: C, dtype: bool

"""
return self.apply(lambda x: x.is_monotonic_increasing)

@Substitution(name='groupby')
@Appender(_doc_template)
def is_monotonic_decreasing(self):
"""
Returns whether each group is monotonically decreasing.

Equivalent to ``.apply(lambda x: x.is_monotonic_decreasing)``.

Examples
--------
>>> source_dict = {
... 'A': ['this', 'col', 'is', 'entirely', 'irrelevant', '.'],
... 'B': ['cat_a', 'cat_a', 'cat_a', 'cat_b', 'cat_b', 'cat_b'],
... 'C': [1, 2, 3, 2, 2, 0]}

>>> df = pd.DataFrame(source_dict)
... df.groupby(['B']).C.is_monotonic_decreasing()
B
cat_a False
cat_b True
Name: C, dtype: bool
"""
return self.apply(lambda x: x.is_monotonic_decreasing)


GroupBy._add_numeric_operations()

Expand Down
58 changes: 58 additions & 0 deletions pandas/tests/groupby/test_groupby.py
Original file line number Diff line number Diff line change
Expand Up @@ -3701,6 +3701,64 @@ def test_cummin_cummax(self):
expected = pd.Series([1, 2, 1], name='b')
tm.assert_series_equal(result, expected)

@pytest.mark.parametrize('in_vals, out_vals', [
# Basics: strictly increasing (T), strictly decreasing (F),
# abs val increasing (F), non-strictly increasing (T)
([1, 2, 5, 3, 2, 0, 4, 5, -6, 1, 1],
[True, False, False, True]),
# Test with inf vals
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a blank line before comments

([1, 2.1, np.inf, 3, 2, np.inf, -np.inf, 5, 11, 1, -np.inf],
[True, False, True, False]),
# Test with nan vals; should always be False
([1, 2, np.nan, 3, 2, np.nan, np.nan, 5, -np.inf, 1, np.nan],
[False, False, False, False]),
])
def test_is_monotonic_increasing(self, in_vals, out_vals):
# GH 17015
source_dict = {
'A': ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11'],
'B': ['a', 'a', 'a', 'b', 'b', 'b', 'c', 'c', 'c', 'd', 'd'],
'C': in_vals}
df = pd.DataFrame(source_dict)
result = df.groupby(['B']).C.is_monotonic_increasing()
expected = pd.Series(index=['a', 'b', 'c', 'd'],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can write this as Index(list('abcd'), name='B')

data=out_vals,
name='C')
expected.index.name = 'B'
tm.assert_series_equal(result, expected)

# Also check result equal to manually taking x.is_monotonic_increasing.
expected = (
df.groupby(['B']).C.apply(lambda x: x.is_monotonic_increasing))
tm.assert_series_equal(result, expected)

@pytest.mark.parametrize('in_vals, out_vals', [
# Basics: strictly decreasing (T), strictly increasing (F),
# abs val decreasing (F), non-strictly increasing (T)
([10, 9, 7, 3, 4, 5, -3, 2, 0, 1, 1],
[True, False, False, True]),
# Test with inf vals
([np.inf, 1, -np.inf, np.inf, 2, -3, -np.inf, 5, -3, -np.inf, -np.inf],
[True, True, False, True]),
# Test with nan vals; should always be False
([1, 2, np.nan, 3, 2, np.nan, np.nan, 5, -np.inf, 1, np.nan],
[False, False, False, False]),
])
def test_is_monotonic_decreasing(self, in_vals, out_vals):
# GH 17015
source_dict = {
'A': ['1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11'],
'B': ['a', 'a', 'a', 'b', 'b', 'b', 'c', 'c', 'c', 'd', 'd'],
'C': in_vals}

df = pd.DataFrame(source_dict)
result = df.groupby('B').C.is_monotonic_decreasing()
expected = pd.Series(index=['a', 'b', 'c', 'd'],
data=out_vals,
name='C')
expected.index.name = 'B'
tm.assert_series_equal(result, expected)

def test_apply_numeric_coercion_when_datetime(self):
# In the past, group-by/apply operations have been over-eager
# in converting dtypes to numeric, in the presence of datetime
Expand Down
23 changes: 12 additions & 11 deletions pandas/tests/groupby/test_whitelist.py
Original file line number Diff line number Diff line change
Expand Up @@ -239,17 +239,18 @@ def test_groupby_blacklist(df_letters):
def test_tab_completion(mframe):
grp = mframe.groupby(level='second')
results = set([v for v in dir(grp) if not v.startswith('_')])
expected = {
'A', 'B', 'C', 'agg', 'aggregate', 'apply', 'boxplot', 'filter',
'first', 'get_group', 'groups', 'hist', 'indices', 'last', 'max',
'mean', 'median', 'min', 'ngroups', 'nth', 'ohlc', 'plot',
'prod', 'size', 'std', 'sum', 'transform', 'var', 'sem', 'count',
'nunique', 'head', 'describe', 'cummax', 'quantile',
'rank', 'cumprod', 'tail', 'resample', 'cummin', 'fillna',
'cumsum', 'cumcount', 'ngroup', 'all', 'shift', 'skew',
'take', 'tshift', 'pct_change', 'any', 'mad', 'corr', 'corrwith',
'cov', 'dtypes', 'ndim', 'diff', 'idxmax', 'idxmin',
'ffill', 'bfill', 'pad', 'backfill', 'rolling', 'expanding', 'pipe'}
expected = set(
['A', 'B', 'C', 'agg', 'aggregate', 'apply', 'boxplot', 'filter',
'first', 'get_group', 'groups', 'hist', 'indices', 'last', 'max',
'mean', 'median', 'min', 'ngroups', 'nth', 'ohlc', 'plot',
'prod', 'size', 'std', 'sum', 'transform', 'var', 'sem', 'count',
'nunique', 'head', 'describe', 'cummax', 'quantile',
'rank', 'cumprod', 'tail', 'resample', 'cummin', 'fillna',
'cumsum', 'cumcount', 'ngroup', 'all', 'shift', 'skew',
'take', 'tshift', 'pct_change', 'any', 'mad', 'corr', 'corrwith',
'cov', 'dtypes', 'ndim', 'diff', 'idxmax', 'idxmin',
'ffill', 'bfill', 'pad', 'backfill', 'rolling', 'expanding', 'pipe',
'is_monotonic_increasing', 'is_monotonic_decreasing'])
assert results == expected


Expand Down