Skip to content

PERF: improve performance of NDFrame.describe #21274

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 5, 2018
Merged

PERF: improve performance of NDFrame.describe #21274

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
7efefb9
PERF: improve performance of NDFrame.describe
DataOmbudsman May 31, 2018
7e3ad12
Add ASV benchmark
DataOmbudsman Jun 4, 2018
70668a1
Add whatsnew entry
DataOmbudsman Jun 4, 2018
b866d0f
Add blank line for pep8
DataOmbudsman Jun 4, 2018
31216c8
remove random seed
DataOmbudsman Jun 5, 2018
6dda68e
Add issue (PR) number
DataOmbudsman Jun 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions asv_bench/benchmarks/frame_methods.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -512,3 +512,21 @@ def time_nlargest(self, keep):

def time_nsmallest(self, keep):
self.df.nsmallest(100, 'A', keep=keep)


class Describe(object):

goal_time = 0.2

def setup(self):
self.df = DataFrame({
'a': np.random.randint(0, 100, int(1e6)),
'b': np.random.randint(0, 100, int(1e6)),
'c': np.random.randint(0, 100, int(1e6))
})

def time_series_describe(self):
self.df['a'].describe()

def time_dataframe_describe(self):
self.df.describe()
3 changes: 1 addition & 2 deletions doc/source/whatsnew/v0.24.0.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,8 +63,7 @@ Removal of prior version deprecations/changes
Performance Improvements
~~~~~~~~~~~~~~~~~~~~~~~~

-
-
- Improved performance of :func:`Series.describe` in case of numeric dtpyes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add the issue number (this pr number as we don't have an issue)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK but I'm unsure about what format is expected. Do you think a link to an external URL (such as here) would be appropriate? E.g., `pull request #21274 <https://github.com/pandas-dev/pandas/pull/21274/>`_. Or something else?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same format as all the others, just use :issue:`number`

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see now that the URL of the issue is translated to the URL of the PR. That's great.

-

.. _whatsnew_0240.docs:
Expand Down
2 changes: 1 addition & 1 deletion pandas/core/generic.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8519,7 +8519,7 @@ def describe_numeric_1d(series):
stat_index = (['count', 'mean', 'std', 'min'] +
formatted_percentiles + ['max'])
d = ([series.count(), series.mean(), series.std(), series.min()] +
[series.quantile(x) for x in percentiles] + [series.max()])
series.quantile(percentiles).tolist() + [series.max()])
return pd.Series(d, index=stat_index, name=series.name)

def describe_categorical_1d(data):
Expand Down