Skip to content

DOC: Added additional example for groupby by indexer. #13276

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jun 28, 2016
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions doc/source/groupby.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1014,6 +1014,33 @@ Regroup columns of a DataFrame according to their sum, and sum the aggregated on
df
df.groupby(df.sum(), axis=1).sum()

Groupby by Indexer to 'resample' data.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a fair bit more explanation of the why and how this does what it does. Maybe show the intent of a resample, then show how one can go about the same idea using non-datetimelike indices.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need a markdown line here like the other examples

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you underline this with ~~~~ to make this a header (see a few lines above this for an example)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you forgot this one


Resampling produces new hypothetical samples(resamples) from already existing observed data or from a data generating mechanism which resemble the underlying population.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"data generating mechanism which resemble the underlying population" seems a bit difficult explanation.


In order to resample to work on indices that are non-datetimelike , the following procedure can be utilized.

In the following examples, **df.index / 5** returns a binary array which is used to determine what get's selected for the groupby operation.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can show this (add another ipython block), df.index / 5


.. note:: The above example shows how we can downsample. Downsampling refers to the throwing away of samples. Here by using **df.index / 5**, we are throwing away half the samples.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

more about aggregating samples in certain bins


.. ipython:: python

df = pd.DataFrame(np.random.randn(10,2))
df
df.groupby(df.index / 5).std()

.. note:: For upsampling, we can again utilize a similar technique. Upsampling inserts values between the original samples. However, we change the indexes to a spaced out interval so that the new samples can fill those vacant indexes. Observe how the indexes of **df_down** are spaced out.

.. ipython:: python

df = pd.DataFrame(np.random.randn(10,2))
df
s = (df.index.to_series() / 5).astype(int)
df_down = df.groupby(df.index / 5).std().set_index(s.index[4::5])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why would you .set_index() here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.set_index() sets index at an interval (eg 4, 8). Then in the next line when upsampling occurs by reindexing followed by bfill it fills in the missing indexes with generated values there increasing number of samples.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know, but why are you doing this. why is it relevant here? To me its just confusing things and making more complicated.

Further I don't think this 'upsampling' example is enlightening.

df_down
df_up = df_down.reindex(range(10)).bfill()
df_up

Returning a Series to propagate names
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down