Performance testing script: cleanup, bugfixes, new features #3517
Conversation
tautschnig
requested review from
forejtv,
peterschrammel and
thk123
as code owners
There was a problem hiding this comment.
✔️
Passed Diffblue compatibility checks (cbmc commit: 1fbd642).
Build URL: https://travis-ci.com/diffblue/test-gen/builds/93295865
|
Marking "do not merge" - the "compare-to" code does not seem to be working as it should. |
tautschnig
force-pushed
the
perf-test-cleanup
branch
from
1fbd642 to
3c584e1
Compare
|
The comparison feature is now working as well, benchexec nicely produces tables to compare multiple runs. |
| @@ -0,0 +1,35 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
⛏️ Add docs at the top for how to run this (e.g. it looks like it should be called ec2-terminate.sh start|status)
| @@ -0,0 +1,35 @@ | |||
| #!/bin/bash | |||
| ### BEGIN INIT INFO | |||
There was a problem hiding this comment.
❓ I don't know what the next 8 lines of code do...? That aren't informative to me?
| esac | ||
|
|
||
| # send instance-terminated message | ||
| # http://rogueleaderr.com/post/48795010760/how-to-notifyemail-yourself-when-an-ec2-instance/amp |
There was a problem hiding this comment.
❓ Is this the source of this code? Probably want to make it more obvious that's what this link is for, e.g.
# Code take from
| @@ -0,0 +1,256 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
⛏️ This is a long bash file that looks to be mostly moved:
- move in one commit, bug fixes and other changes in a separate one if possible
- is bash the right language for something this long?
We need to manually upload to S3 rather than using CodeBuild's cache configuration as each perf-test run creates new CodeBuild configurations and thus also new keys to store the cache at.
This reduces overhead during package install.
This simplifies editing. Just upload them to S3 when setting up benchmarking and download them from S3 onto the instance. Also fixed several bugs in the script.
This simplifies performance comparison.
tautschnig
force-pushed
the
perf-test-cleanup
branch
from
3c584e1 to
1614a5f
Compare
There was a problem hiding this comment.
✔️
Passed Diffblue compatibility checks (cbmc commit: 1614a5f).
Build URL: https://travis-ci.com/diffblue/test-gen/builds/95125956
In preparation of SV-COMP'19 several improvements to the script were due, overall also improving the maintainability of the code.