CLN: ASV categoricals benchmark #18465

mroeschke · 2017-11-24T07:55:01Z

Split classes from Categoricals/2/3 --> Concat, ValueCounts, Rank, Repr, SetCategories, and Constructor
Utilized params and param_names for ValueCounts
Added np.random.seed(1234) in setup classes where random data is created xref BENCH: put in np.random.seed on vbenches #8144
Ran flake8 and replaced star imports

$ asv run -q -b ^categoricals
[  0.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[  5.88%] ··· Running categoricals.Concat.time_concat                                                                               14.2ms
[ 11.76%] ··· Running categoricals.Concat.time_union                                                                                12.2ms
[ 17.65%] ··· Running categoricals.Constructor.time_constructor_all_nan                                                             54.7ms
[ 23.53%] ··· Running categoricals.Constructor.time_constructor_datetimes                                                           2.22ms
[ 29.41%] ··· Running categoricals.Constructor.time_constructor_datetimes_with_nat                                                  2.32ms
[ 35.29%] ··· Running categoricals.Constructor.time_constructor_fastpath                                                            1.59ms
[ 41.18%] ··· Running categoricals.Constructor.time_constructor_regular                                                             48.8ms
[ 47.06%] ··· Running categoricals.Constructor.time_constructor_with_nan                                                             510ms
[ 52.94%] ··· Running categoricals.Rank.time_rank_int                                                                               11.7ms
[ 58.82%] ··· Running categoricals.Rank.time_rank_int_cat                                                                           12.9ms
[ 64.71%] ··· Running categoricals.Rank.time_rank_int_cat_ordered                                                                   12.2ms
[ 70.59%] ··· Running categoricals.Rank.time_rank_string                                                                             520ms
[ 76.47%] ··· Running categoricals.Rank.time_rank_string_cat                                                                        18.3ms
[ 82.35%] ··· Running categoricals.Rank.time_rank_string_cat_ordered                                                                10.2ms
[ 88.24%] ··· Running categoricals.Repr.time_rendering                                                                              1.58ms
[ 94.12%] ··· Running categoricals.SetCategories.time_set_categories                                                                70.7ms
[100.00%] ··· Running categoricals.ValueCounts.time_value_counts                                                                81.2ms;...

jorisvandenbossche

Nice clean-up! Left a few comments

jorisvandenbossche · 2017-11-24T09:19:38Z

asv_bench/benchmarks/categoricals.py

-    try:
-        from pandas.types.concat import union_categoricals
-    except ImportError:
-        pass


can you leave this try .. except in place? It is meant to be able to run the benchmarks with different older versions

jorisvandenbossche · 2017-11-24T09:21:05Z

asv_bench/benchmarks/categoricals.py

        np.random.seed(2718281)
        arr = ['s%04d' % i for i in np.random.randint(0, n // 10, size=n)]
-        self.ts = Series(arr).astype('category')
+        self.ts = pd.Series(arr).astype('category')
+        self.dropna = dropna


This is not needed I think? You can directly use the arg dropna in the time method

jorisvandenbossche · 2017-11-24T09:21:19Z

asv_bench/benchmarks/categoricals.py

+        self.dropna = dropna
+
+    def time_value_counts(self, dropna):
+        self.ts.value_counts(dropna=self.dropna)


self.dropna -> dropna

jorisvandenbossche · 2017-11-24T09:24:13Z

asv_bench/benchmarks/categoricals.py

-
-    def time_union(self):
-        union_categoricals([self.a, self.b])
-
    def time_constructor_regular(self):


Do we wan to remove the _constructor part in the time names? As you already have it in the class name (now you have 'constructor' twice and making the name long)

codecov · 2017-11-24T11:03:13Z

Codecov Report

Merging #18465 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #18465      +/-   ##
==========================================
- Coverage   91.34%   91.32%   -0.02%     
==========================================
  Files         163      163              
  Lines       49717    49717              
==========================================
- Hits        45413    45404       -9     
- Misses       4304     4313       +9

Flag	Coverage Δ
#multiple	`89.12% <ø> (ø)`	⬆️
#single	`40.51% <ø> (-0.07%)`	⬇️

Impacted Files	Coverage Δ
pandas/io/gbq.py	`25% <0%> (-58.34%)`	⬇️
pandas/core/frame.py	`97.8% <0%> (-0.1%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 412988e...3fcf30f. Read the comment docs.

More splitting in classes & flake 8

mroeschke · 2017-11-25T00:47:22Z

Kept the removed imports, changed the redundant method names, and changed self.dropna -> dropna

jreback · 2017-11-25T14:37:02Z

thanks!

jorisvandenbossche reviewed Nov 24, 2017

View reviewed changes

jorisvandenbossche added the Benchmark Performance (ASV) benchmarks label Nov 24, 2017

jorisvandenbossche reviewed Nov 24, 2017

View reviewed changes

mroeschke added 2 commits November 24, 2017 16:44

CLN: ASV categoricals benchmark

50407cf

More splitting in classes & flake 8

Address comments

3fcf30f

mroeschke force-pushed the asv_clean_categoricals branch from b73774c to 3fcf30f Compare November 25, 2017 00:44

jreback added this to the 0.22.0 milestone Nov 25, 2017

jreback merged commit 200227e into pandas-dev:master Nov 25, 2017

mroeschke deleted the asv_clean_categoricals branch November 25, 2017 18:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CLN: ASV categoricals benchmark #18465

CLN: ASV categoricals benchmark #18465

mroeschke commented Nov 24, 2017

jorisvandenbossche left a comment

jorisvandenbossche Nov 24, 2017

jorisvandenbossche Nov 24, 2017

jorisvandenbossche Nov 24, 2017

jorisvandenbossche Nov 24, 2017

codecov bot commented Nov 24, 2017 •

edited

Loading

mroeschke commented Nov 25, 2017

jreback commented Nov 25, 2017

CLN: ASV categoricals benchmark #18465

CLN: ASV categoricals benchmark #18465

Conversation

mroeschke commented Nov 24, 2017

jorisvandenbossche left a comment

Choose a reason for hiding this comment

jorisvandenbossche Nov 24, 2017

Choose a reason for hiding this comment

jorisvandenbossche Nov 24, 2017

Choose a reason for hiding this comment

jorisvandenbossche Nov 24, 2017

Choose a reason for hiding this comment

jorisvandenbossche Nov 24, 2017

Choose a reason for hiding this comment

codecov bot commented Nov 24, 2017 • edited Loading

Codecov Report

mroeschke commented Nov 25, 2017

jreback commented Nov 25, 2017

codecov bot commented Nov 24, 2017 •

edited

Loading