Skip to content

Docstring shift #21039

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 24 commits into from
Closed
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
77f2a2b
added an example to shift
PyJay May 14, 2018
c935896
fixing pep8 errors
PyJay May 14, 2018
a6ac960
Merge branch 'master' of https://github.com/pandas-dev/pandas into do…
PyJay May 15, 2018
c892655
Merge branch 'master' of https://github.com/pandas-dev/pandas into do…
PyJay May 26, 2018
3639e6c
Merge branch 'master' of https://github.com/pandas-dev/pandas into do…
PyJay May 31, 2018
61a4ae1
Merge branch 'master' of https://github.com/pandas-dev/pandas into do…
PyJay Jun 2, 2018
206b2da
making changes per feedback
PyJay Jun 2, 2018
c31d1a1
fixing pep8 errors
PyJay Jun 2, 2018
73a640f
fix trailing whitespace
PyJay Jun 2, 2018
5bee1e3
Merge branch 'master' of https://github.com/pandas-dev/pandas into do…
PyJay Jun 28, 2018
de0b7a1
WIP: adding more detail in docstring
PyJay Jun 28, 2018
739f8a1
Merge branch 'master' of https://github.com/pandas-dev/pandas into do…
PyJay Jul 1, 2018
a9550fa
displaying difference between shift with/without freq
PyJay Jul 1, 2018
67b6ee2
Merge branch 'master' of https://github.com/pandas-dev/pandas into do…
PyJay Jul 15, 2018
ab4e2f5
wip: updating docstring
PyJay Jul 22, 2018
e7d6f0a
Merge branch 'master' of https://github.com/pandas-dev/pandas into do…
PyJay Jul 22, 2018
5bcae81
wip: updating docstring
PyJay Jul 22, 2018
17f55d5
wip: updating docstring
PyJay Jul 22, 2018
4dd276a
Docstring updated
PyJay Jul 22, 2018
0b4f41b
fixing pep8 issues
PyJay Jul 22, 2018
7c15b08
fix whitespace in frames
PyJay Jul 22, 2018
950cc2e
fixing whitespace
PyJay Jul 22, 2018
9ec5ed7
fixing whitespace
PyJay Jul 22, 2018
888b97b
fixing pep8 errors
PyJay Jul 22, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 87 additions & 2 deletions pandas/core/generic.py
Original file line number Diff line number Diff line change
Expand Up @@ -8012,12 +8012,12 @@ def mask(self, cond, other=np.nan, inplace=False, axis=None, level=None,
errors=errors)

_shared_docs['shift'] = ("""
Shift index by desired number of periods with an optional time freq
Shift index by desired number of periods with an optional time freq.

Parameters
----------
periods : int
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add the default 1

Number of periods to move, can be positive or negative
Number of periods to move, can be positive or negative.
freq : DateOffset, timedelta, or time rule string, optional
Increment to use from the tseries module or time rule (e.g. 'EOM').
See Notes.
Expand All @@ -8029,6 +8029,91 @@ def mask(self, cond, other=np.nan, inplace=False, axis=None, level=None,
is not realigned. That is, use freq if you would like to extend the
index when shifting and preserve the original data.

Examples
--------

>>> df = pd.DataFrame({'group': ['A', 'A', 'A', 'B', 'B', 'B'],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This probably isn't enforceable and maybe it's just a personal preference but I think using key and val would be preferable to group and myvalue. I've seen the former used in quite a few more places so for consistency would be better to use those names.

@datapythonista maybe something to think about

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general I think it makes things clearer if the example is with real world data. In this case, if we just show basic examples of shift (and we avoid using ffill, reindexing, groupby and concat), I don't think we need the column group.

I think the examples are quite good for the cookbook, or the time series documentation, but for the docstring, I think we should show the basic stuff. In this case what I'd do:

  • Create a simple dataframe (one column and 4 rows for example)
  • Show .shift() with default arguments
  • Show how period can change the shifted periods
  • Show how freq can be used

And I don't know exactly what axis does in this method, it could be shown too.

For all the rest of the stuff, feel free to contribute it to the other documents.

... 'myvalue': [1, 2, 3, 4, 5, 6]},
... index=pd.DatetimeIndex(['2016-06-06',
... '2016-06-08',
... '2016-06-09',
... '2016-06-10',
... '2016-06-12',
... '2016-06-13'],
... name='mydate'))

>>> df
group myvalue
mydate
2016-06-06 A 1
2016-06-08 A 2
2016-06-09 A 3
2016-06-10 B 4
2016-06-12 B 5
2016-06-13 B 6

For the groups compute the difference between current `myvalue` and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I'm missing the point but I don't think you need this sentence

`myvalue` shifted forward by 1 day.

If `myvalue` is shifted then the values will simply move down.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good start but I think the point to stress here is that "move down" makes no consideration of the dates being in order. So maybe to emphasize better can append something like "...move down one row, regardless of the chronology of the dates"


>>> df.myvalue.shift(1)
mydate
2016-06-06 NaN
2016-06-08 1.0
2016-06-09 2.0
2016-06-10 3.0
2016-06-12 4.0
2016-06-13 5.0
Name: myvalue, dtype: float64

We only want to shift `myvalue` forward by one day before computing
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just for clearer wording I'd say "If instead you wanted to shift values forward by a day you can do this by reindexing first and filling key"

the difference. We can do this by reindexing and filling the groups
first

>>> date_range = pd.date_range(df.index.min(), df.index.max())
>>> df = df.reindex(date_range)
>>> df['group'] = df['group'].ffill()
>>> df
group myvalue
2016-06-06 A 1.0
2016-06-07 A NaN
2016-06-08 A 2.0
2016-06-09 A 3.0
2016-06-10 B 4.0
2016-06-11 B NaN
2016-06-12 B 5.0
2016-06-13 B 6.0

After considering the grouping we can calculate the difference
as follows

>>> result = df['myvalue'] - df.groupby('group')['myvalue'].shift(1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure what you are aiming for with the subtraction but I think it's clearer if you just do df.groupby('key').shift() here and show the result

>>> result
2016-06-06 NaN
2016-06-07 NaN
2016-06-08 NaN
2016-06-09 1.0
2016-06-10 NaN
2016-06-11 NaN
2016-06-12 NaN
2016-06-13 1.0
Freq: D, Name: myvalue, dtype: float64

Concatenate result as a column named `delta` to the original data
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate the thoroughness you are aiming for here, but let's get rid of this section to the example. concat will have a separate docstring with examples to teach that piece to users should they want to know


>>> result.name = 'delta'
>>> pd.concat([df, result], axis=1)
group myvalue delta
2016-06-06 A 1.0 NaN
2016-06-07 A NaN NaN
2016-06-08 A 2.0 NaN
2016-06-09 A 3.0 1.0
2016-06-10 B 4.0 NaN
2016-06-11 B NaN NaN
2016-06-12 B 5.0 NaN
2016-06-13 B 6.0 1.0

Returns
-------
shifted : %(klass)s
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you move Return before the examples?

Expand Down