PERF: slicing

[jbrockmendel]

import pandas as pd
ser = pd.Series(range(300_000))
df = ser.to_frame()

%timeit ser[:30]
14.6 µs ± 474 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)  # <- main
12.6 µs ± 421 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)  # <- PR

%timeit df[:30]
16.2 µs ± 1.23 µs per loop (mean ± std. dev. of 7 runs, 100,000 loops each)  # <- main
11.9 µs ± 184 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)   # <- PR

[mroeschke]

Are these changes supposed to be in this PR?

[jbrockmendel]

Yes. __finalize__ adds up.

[mroeschke]

It looks like .attrs and .flags just call ._attrs and ._flags respectively? Did the attribute call significantly impact the slicing benchmark?

[jbrockmendel]

I'm not seeing much difference, will revert.

[jbrockmendel]

reverted+greenish

[mroeschke]

Thanks @jbrockmendel

[alippai]

Interesting, why is the df faster than the series?

[jbrockmendel]

Interesting, why is the df faster than the series?

In [4]: %prun -s cumtime for n in range(10**4): ser[:30]
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    10000    0.038    0.000    0.436    0.000 series.py:971(__getitem__)
    10000    0.019    0.000    0.336    0.000 generic.py:4153(_getitem_slice)
    10000    0.029    0.000    0.269    0.000 series.py:964(_slice)
    10000    0.053    0.000    0.111    0.000 managers.py:1990(get_slice)
    10000    0.049    0.000    0.069    0.000 generic.py:6020(__finalize__)
    10000    0.020    0.000    0.058    0.000 series.py:371(__init__)
    [ommitted stuff that is roughly the same in both]

In [5]: %prun -s cumtime for n in range(10**4): df[:30]
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    10000    0.028    0.000    0.386    0.000 frame.py:3736(__getitem__)
    10000    0.018    0.000    0.314    0.000 generic.py:4153(_getitem_slice)
    10000    0.039    0.000    0.252    0.000 generic.py:4172(_slice)
    10000    0.039    0.000    0.078    0.000 {method 'get_slice' of 'pandas._libs.internals.BlockManager' objects}
    10000    0.018    0.000    0.056    0.000 frame.py:651(__init__)
    10000    0.034    0.000    0.050    0.000 generic.py:6020(__finalize__)
    [ommitted stuff that is roughly the same in both]

Further investigation welcome.