Update performance scaling test rig #1096
Conversation
There was a problem hiding this comment.
looks pretty good!
on the wishlist: it might be nice to have summary info about the results of the run included in the follow-up comment, like the running time of the docker run locust command or a count/percent of the failed queries. this looks like a good way to get information passed between the jobs.
| - name: Build & run Locust | ||
| continue-on-error: true # sometimes ~2-5 queries fail, we shouldn't end the run if that's the case |
There was a problem hiding this comment.
what are the failures? they only happen transiently?
There was a problem hiding this comment.
There are two queries from the "v4 - small" set that consistently fail.
Co-authored-by: Brian Clark <[email protected]>
| docker build -t locust . | ||
| docker run --net=host -v $PWD:/mnt/locust -e CSV=/mnt/locust/v4-requests-smaller.csv locust -f /mnt/locust/v4.py --host http://127.0.0.1:10080/ --users 10 --spawn-rate 1 --headless -t 15m | ||
| export CSV=v4-requests-small.csv |
There was a problem hiding this comment.
question: why move the CSV filename into a variable?
(is it a stub for being able to override it in the future?)
There was a problem hiding this comment.
It's just to avoid repeating the same filename twice, since it's now used in the wc -l bit as well.
Prerequisites:
devbranchdevSummary
Adds several updates to the performance scaling test rig:
cmu-delphi/delphi-adminrepo, so files do not need to be duplicated to this repo.