0.13rc1+ vs. 0.12 vbench, perf regressions. #5660
Comments
|
Are all of these needing to be fixed before .13 is released? I've got an open issue about a bug in the Panel from_dict constructor so I'm going to be poking around there anyway, but there's no way I'll be able to get to it within the next month. |
|
Not all of these are legitimate regressions (some are false positives and some make sense, but some are unintentional). As to "blocker status", If a bunch of core operation are twice as slow I think that definitely deserves some Any help shrinking the pile is useful, simply bisecting to a commit is a lot of help and requires just a little |
|
Here I have run 5x from 0.12 to current master Most of these slowdowns are from a difference in construction overhead with 0.12. When doing an apply/groupbpy and splitting the object in cython, a user function is called, so the object passed must be a Series. Their is construction overhead on this. Certainly would appreciate some eyes on this, but IMHO none of these are blockers. |
|
The last two were not recently addressed, that's after the fixes, But they do more work (truncated rather then info view) so no fix required. what does no-issue stand for? false positive? |
|
I just realized that |
|
I agree often < 1ms are false positives, but a few us ops are in inner loops so the noise can actually Seems like the constructor stuff has associated large slowdown, can you name the PR? |
|
non-issue means its there, but its a less common operation (e.g. frame_fancy_lookup is actually the lookup call, not indexing), or the vbench is already on a very large set (e.g. reindx_daterange...), so not a big deal |
|
#3482 started it all (their are benchmarks their about 1/2 down). It is possible things slipped a bit after that. A lot of the duplicate handling code is slow, so generally have a fast path for things like that (e.g. if your frame has dups then need to select columns by |
|
construction overhead is on |
|
What's the panel to_dict and/or set_item issue? we'll add a perf notice to the issue tracking it and punt. |
|
I don't see an issue about panel to_dict (from_dict is small slowdown) |
|
frame_iteritems has O(N) behavior in input, so it's not constant overhead per frame. Is this #3482 as well? what did users get in return? 0.12: 55e624d (0.13rc1+) |
|
I think we are running the same bench, but here is the code-vs-code v0.12 (release) current master |
|
This measure pure Series construction overhead, which is basically 1.5x slower (which causes 99% of things above) 0.12 master |
|
2x vs. 20x? I'll remeasure, maybe I fumbled it. |
|
Same thing again: ~25x My original measurement showed 2x as well. Can't explain it. |
|
It seems (and I may have been the culprit), the test frame was |
|
right, always check out vb_suite/ from master before running the tests on each version, my bad. I'll check again. |
|
yup. 2x is still a drop, what was the upside? |
|
series inherits from ndframe so consistency in methods across pandas |
|
hard sell. |
|
So I'm concerned about the following. Are any of these going to slow more than O(1), eg get slower the more rows you process? Especially the apply and concat functions. groupby_multi_python - |
|
@dragoljub Perhaps it's worth putting in some larger vbench examples of the things your're concerned about? |
|
@dragoljub, as jreback suggests should not see a change in O(), just worse constants. I can't locate further regressions beyond #3482. The jitteriness in vbench measurements I don't plan to chase this any more. @jreback, close? |
|
yep..unless someone has a specific perf regression |
|
Here is some example code. I was wondering how the df.apply() slowdown would impact computing 20k chi squared tests? Currently using the apply method it takes 1.58 seconds on my machine, with pd 0.12.0. If its a import pandas as pd
import numpy as np
from scipy.stats import chi2_contingency
chi2 = lambda x: chi2_contingency(x.reshape(2,2), correction=True)[1]
df_test = pd.DataFrame(np.random.randint(20000, size=(10000,4)), columns=['A','B', 'C', 'D'])
%timeit df_test.apply(chi2, raw=True, axis=1)
1 loops, best of 3: 1.58 s per loop
pd.__version__
Out[5]: '0.12.0'
np.__version__
Out[6]: '1.7.1' |
|
Linear means it grows with more rows, but I understand what you mean. Running that example in 0.12 vs. 0.13rc shows perf is linear in row count for both Closing now, if you come across specific regressions you're concerned about, |
ghost
mentioned this issue
30% slowdown and up.
Ruled out
#5550: truncated repr for large frames. legitimate change.
The text was updated successfully, but these errors were encountered: