Skip to content

PERF: For GH23814, return early in Categorical.__init__ #23888

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Nov 30, 2018
Merged

PERF: For GH23814, return early in Categorical.__init__ #23888

Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
5349052
TST: Add test case for GH14080 for overflow exception
erikoveson Nov 18, 2018
a7df6ea
TST: Add test case for GH14080 for overflow exception
erikoveson Nov 18, 2018
f7be8f3
TST: Add test case for GH14080 for overflow exception
erikoveson Nov 18, 2018
c727003
TST: For GH14080, break up tests, test message
erikoveson Nov 18, 2018
2eaa8fd
Merge remote-tracking branch 'upstream/master'
erikoveson Nov 19, 2018
eecede1
TST: For GH4861, Period and datetime in multiindex
erikoveson Nov 19, 2018
ae90f93
TST: GH4861 move changes to different test file
erikoveson Nov 19, 2018
90af4a5
Merge branch 'master' into PR_TOOL_MERGE_PR_23776
jreback Nov 20, 2018
4f16c8b
fix sorting
jreback Nov 20, 2018
89871a0
Merge remote-tracking branch 'upstream/master'
erikoveson Nov 20, 2018
3e96734
PERF: For GH23814, return early in Categorical.__init__
erikoveson Nov 24, 2018
f6d10b8
PERF: For GH23814, add whatsnew entry
erikoveson Nov 24, 2018
2173c89
Consolidating code to fit existing pattern better
erikoveson Nov 29, 2018
325be92
Fix comment
erikoveson Nov 29, 2018
9e270e9
Add asv test that exercises code, update docs
erikoveson Nov 29, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions asv_bench/benchmarks/categoricals.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ def setup(self):
self.values_some_nan = list(np.tile(self.categories + [np.nan], N))
self.values_all_nan = [np.nan] * len(self.values)
self.values_all_int8 = np.ones(N, 'int8')
self.categorical = pd.Categorical(self.values, self.categories)

def time_regular(self):
pd.Categorical(self.values, self.categories)
Expand All @@ -68,6 +69,9 @@ def time_all_nan(self):
def time_from_codes_all_int8(self):
pd.Categorical.from_codes(self.values_all_int8, self.categories)

def time_existing_categorical(self):
pd.Categorical(self.categorical)


class ValueCounts(object):

Expand Down
2 changes: 1 addition & 1 deletion doc/source/whatsnew/v0.24.0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1150,7 +1150,7 @@ Performance Improvements
- Improved performance of :func:`pd.concat` for `Series` objects (:issue:`23404`)
- Improved performance of :meth:`DatetimeIndex.normalize` and :meth:`Timestamp.normalize` for timezone naive or UTC datetimes (:issue:`23634`)
- Improved performance of :meth:`DatetimeIndex.tz_localize` and various ``DatetimeIndex`` attributes with dateutil UTC timezone (:issue:`23772`)

- Improved performance of :meth:`Categorical.__init__` (:issue:`23814`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

say constructor rather than referring to __init__

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jreback , updated doc string, and also added asv test that exercises the code (first one didn't, but left it since still useful) (you can see my comment about asv results to gfyoung)


.. _whatsnew_0240.docs:

Expand Down
10 changes: 10 additions & 0 deletions pandas/core/arrays/categorical.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,16 @@ def __init__(self, values, categories=None, ordered=None, dtype=None,
# the "ordered" and "categories" arguments
dtype = values.dtype._from_categorical_dtype(values.dtype,
categories, ordered)

# GH23814, for perf, if values._values already an instance of
# Categorical, set values to codes, and run fastpath
if (isinstance(values, (ABCSeries, ABCIndexClass)) and
isinstance(values._values, type(self))):
values = values._values.codes.copy()
if categories is None:
categories = dtype.categories
fastpath = True

else:
# If dtype=None and values is not categorical, create a new dtype
dtype = CategoricalDtype(categories, ordered)
Expand Down