Skip to content

PERF: speed up .ix by moving deprecation string out of __init__ #24747

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jan 13, 2019

Conversation

qwhelan
Copy link
Contributor

@qwhelan qwhelan commented Jan 13, 2019

While .ix is deprecated, the DeprecationWarning itself adds a decent fraction of overhead to calling it. Simply storing the warning string on the class (or global scope) yields a sizable speedup:

$ asv compare v0.24.0rc1 HEAD -s --only-changed --sort ratio
       before           after         ratio
     [fdc4db25]       [d0821a4c]
     <v0.24.0rc1^0>       <ix_speedup>
-      2.57±0.1ms      2.29±0.01ms     0.89  indexing.NumericSeriesIndexing.time_ix_list_like(<class 'pandas.core.indexes.numeric.UInt64Index'>, 'nonunique_monotonic_inc')
-      2.69±0.2ms      2.39±0.02ms     0.89  indexing.NumericSeriesIndexing.time_ix_list_like(<class 'pandas.core.indexes.numeric.Float64Index'>, 'nonunique_monotonic_inc')
-         124±2μs          103±5μs     0.83  indexing.NumericSeriesIndexing.time_ix_scalar(<class 'pandas.core.indexes.numeric.UInt64Index'>, 'unique_monotonic_inc')
-         149±4μs          121±4μs     0.81  indexing.NumericSeriesIndexing.time_ix_slice(<class 'pandas.core.indexes.numeric.Float64Index'>, 'unique_monotonic_inc')
-     2.43±0.08ms      1.96±0.05ms     0.81  indexing.NumericSeriesIndexing.time_ix_list_like(<class 'pandas.core.indexes.numeric.Int64Index'>, 'unique_monotonic_inc')
-        179±20μs          138±7μs     0.77  indexing.NumericSeriesIndexing.time_ix_slice(<class 'pandas.core.indexes.numeric.Int64Index'>, 'unique_monotonic_inc')
-         506±6μs          384±6μs     0.76  indexing.NumericSeriesIndexing.time_ix_scalar(<class 'pandas.core.indexes.numeric.Float64Index'>, 'nonunique_monotonic_inc')
-        84.6±1μs         63.5±2μs     0.75  indexing.NumericSeriesIndexing.time_ix_scalar(<class 'pandas.core.indexes.numeric.Float64Index'>, 'unique_monotonic_inc')
-        69.1±4μs         50.4±2μs     0.73  indexing.NumericSeriesIndexing.time_ix_scalar(<class 'pandas.core.indexes.numeric.Int64Index'>, 'unique_monotonic_inc')
-     1.22±0.02ms          728±2μs     0.60  indexing.NumericSeriesIndexing.time_ix_slice(<class 'pandas.core.indexes.numeric.Float64Index'>, 'nonunique_monotonic_inc')
-        31.3±4μs       8.67±0.9μs     0.28  indexing.DataFrameStringIndexing.time_ix
-        16.5±1μs       2.67±0.3μs     0.16  indexing.MethodLookup.time_lookup_ix
  • closes #xxxx
  • tests added / passed
  • passes git diff upstream/master -u -- "*.py" | flake8 --diff
  • whatsnew entry

@codecov
Copy link

codecov bot commented Jan 13, 2019

Codecov Report

Merging #24747 into master will not change coverage.
The diff coverage is 100%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master   #24747   +/-   ##
=======================================
  Coverage   92.38%   92.38%           
=======================================
  Files         166      166           
  Lines       52358    52358           
=======================================
  Hits        48373    48373           
  Misses       3985     3985
Flag Coverage Δ
#multiple 90.81% <100%> (ø) ⬆️
#single 43.07% <66.66%> (ø) ⬆️
Impacted Files Coverage Δ
pandas/core/indexing.py 93.87% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 33f91d8...d0821a4. Read the comment docs.

1 similar comment
@codecov
Copy link

codecov bot commented Jan 13, 2019

Codecov Report

Merging #24747 into master will not change coverage.
The diff coverage is 100%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master   #24747   +/-   ##
=======================================
  Coverage   92.38%   92.38%           
=======================================
  Files         166      166           
  Lines       52358    52358           
=======================================
  Hits        48373    48373           
  Misses       3985     3985
Flag Coverage Δ
#multiple 90.81% <100%> (ø) ⬆️
#single 43.07% <66.66%> (ø) ⬆️
Impacted Files Coverage Δ
pandas/core/indexing.py 93.87% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 33f91d8...d0821a4. Read the comment docs.

@gfyoung gfyoung added Performance Memory or execution speed performance Strings String extension data type and string data labels Jan 13, 2019
@gfyoung
Copy link
Member

gfyoung commented Jan 13, 2019

That makes sense, given that it requires fewer looks to find the string. Surprising how the big change in performance is though...

@gfyoung gfyoung requested a review from jreback January 13, 2019 05:37
@jreback jreback added Indexing Related to indexing on series/frames, not to indexes themselves and removed Strings String extension data type and string data labels Jan 13, 2019
@jreback
Copy link
Contributor

jreback commented Jan 13, 2019

ok looks fine. ping on green.

@jreback jreback added this to the 0.24.0 milestone Jan 13, 2019
@jreback jreback merged commit cef5e8a into pandas-dev:master Jan 13, 2019
@jreback
Copy link
Contributor

jreback commented Jan 13, 2019

thanks @qwhelan

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Indexing Related to indexing on series/frames, not to indexes themselves Performance Memory or execution speed performance
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants