REF/BENCH: tslibs-specific parts of asvs #29292
Conversation
|
Not sure, but I imagine that renaming will start a new timeseries. I don't think that's a huge problem though (assuming this benchmark doesn't introduce a slowdown). |
There was a problem hiding this comment.
Do you know what tests are taking the longest? I have an unverified suspicion that parametrized benchmarks like PeriodProperties might be taking longer than you would expect. I've seen something like that with the GroupBy tests where the parametrized ones there alone take close to an hour but don't report a runtime anywhere near that; just haven't dug deep enough yet to confirm
|
|
||
|
|
||
| class TimedeltaProperties: | ||
| def setup_cache(self): |
There was a problem hiding this comment.
Orthogonal but maybe easy to change here if you want - using setup_cache here seems overkill. Mostly used for longer running setup calls, which I can't imagine this is
There was a problem hiding this comment.
generally agreed, but for now prefer to keep this move-only.
Longer-term I hope to get asv to use something like setup_class so that it only gets run once instead of many times
I don't have a good a answer for that, but I can tell you
|
|
Can you post the JSON result file? I think should have start / end time stamps in it
…Sent from my iPhone
On Oct 30, 2019, at 11:51 AM, jbrockmendel ***@***.***> wrote:
Do you know what tests are taking the longest? I have an unverified suspicion that parametrized benchmarks like PeriodProperties might be taking longer than you would expect
I don't have a good a answer for that, but I can tell you
I just did a time asv continuous -E virtualenv -f 1.1 master HEAD -b algorithms and it took 55 minutes.
A few weeks ago I checked and found that a full asv run for a single commit had just over 5000 subprocesses import pandas.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
asv_log.txt This was produced with |
|
For kicks I tried running asv continuous without selecting any benchmarks, and with with environments already built, and it took 15 minutes |
It would be interesting if you ran with the --verbose flag to see where all of that time was spent. Also do you have ccache installed? |
Will give this a go.
I don't know. is this a pip thing or an apt/brew thing? Is it relevant that I make heavy use of |
|
It is a homebrew thing |
|
lgtm. |
|
Yea lgtm as well. Can continue figuring out the ASV nuances outside of this |
|
FWIW looks like ccache helped quite a bit on OSX but not on ubuntu (maybe needs config after install) |
Update: upgrading asv took this down to a few seconds. |
Motivation: asv runs take way too long. One way to get around that is to make it easier to identify* which benchmarks to run for a given PR. To that extent, if we can identify benchmarks that don't rely on anything outside of tslibs**, then a PR that doesn't touch that code can skip those benchmarks. We can do the same thing for other low-dependency modules like _libs.algos.
* Ideally in an automated way, but I'm not there yet.
** Here and elsewhere, "tslibs" informally includes tseries.offsets and parts of tseries.frequences (until we can get all of that into the tslibs directory)
@TomAugspurger will moving benchmarks cause asv to start new time series, or will it successfully link up the new location to the old?