DOC: Updated link for OVH server benchmark visualization #61108
Conversation
web/pandas/community/benchmarks.md
Outdated
| - Original server: [asv](https://asv-runner.github.io/asv-collection/pandas/) | ||
| - OVH server: [asv](https://pandas.pydata.org/benchmarks/asv/) (benchmarks results can | ||
| also be visualized in this [Conbench PoC](http://57.128.112.95:5000/) | ||
| also be visualized in this [GitHub Pages site](https://pandas-dev.github.io/asv-runner/)) |
There was a problem hiding this comment.
@rhshadrach IIUC based on https://github.com/pandas-dev/asv-runner we are just runnning the benchmarks on Github Actions now right? If so, we just need https://pandas-dev.github.io/asv-runner/ here right?
There was a problem hiding this comment.
I'm not sure the status of the original server point, but if conbench isn't available anymore, we can just remove the comment in brackets. The - OVH server: asv bullet point is still relevant. We do run the benchmarks there.
There was a problem hiding this comment.
@datapythonista - why are we running benchmarks there? Is someone looking at them?
I would like a link to the current ASV page that is being maintained (by me, the 2nd link in @mroeschke's comment).
There was a problem hiding this comment.
Sorry for the late reply, I missed this. I may have partial information, but what I know is that Wes had a machine at home where benchmarks were running. When he moved he sent it to Tom, but we didn't consider this approach reliable, and we wanted to have them running in a data center, not in someone's home. When we signed the agreement with OVH we set them up there. The idea was also to try to make them as fast as possible, so we could run them in PRs. A decent amount of work was done in that direction, but funds were over before we could make them fast enough.
So, if I'm not wrong, what we want in this page is to keep the Original server entry as it's Tom's machine. Add an item in the list for the benchmarks running in GitHub Actions that Matt mentions. And keep the OVH server entry, which I guess should have the most stable results.
Does that sound good?
There was a problem hiding this comment.
So, if I'm not wrong, what we want in this page is to keep the Original server entry as it's Tom's machine.
Tom handed over the reigns of this to me years ago, and the benchmark machine has been running in my house since then. After several outages, I migrated the setup to run in https://github.com/pandas-dev/asv-runner. I no longer have any intention of running the local machine.
There was a problem hiding this comment.
And keep the OVH server entry, which I guess should have the most stable results.
I personally did not find the interface suitable for issue detection - it took a very long time to load pages if I recall correctly. If no one is looking at this (I do not intend to - is someone else looking at it?), then I don't understand why we would spend the compute (free or not) to produce results that go stale.
There was a problem hiding this comment.
I didn't know that part of the story. And seems like discontinuing Tom's server is something recent from just 3 months ago, right? https://github.com/asv-runner/asv-collection
@TomAugspurger would it make sense to delete the `asv-runner organization? We can surely leave it if useful, but since it seems like pandas was the last project to use it, and that's discontinued now, maybe it can avoid some confusion. Just stopping GitHub pages would probably be useful, but if the code isn't needed, maybe good to clean up.
@rhshadrach do you think you can update the benchmarks page so what we discussed is publicly available for anyone? I created it some time ago with all the information I had, but seems like I was missing part of the story. If you have the time, it'd probably also be useful to have a short README in the repo, explaining the goal of the repo, and mentioning that the benchmarks are published with GitHub actions to a branch, which takes a bit of time to find out. And any other relevant information.
With this, I think anyone should be able to understand the current state, make use of the benchmarks, and work on them without needed too much research.
There was a problem hiding this comment.
I can archive the repos under the asv-runner org if everything has been moved over to https://github.com/pandas-dev/asv-runner.
There was a problem hiding this comment.
@rhshadrach do you think you can update the benchmarks page so what we discussed is publicly available for anyone?
Happy to - but I'm not sure what you mean by "the benchmarks page". Can you clarify?
If you have the time, it'd probably also be useful to have a short README
Yea, will do.
@TomAugspurger - Just to be certain, while pandas-dev/asv-runner is based on the same idea as the one in asv-runner, it doesn't use the setup with ansible. I'm good with archiving the repos under asv-runner, but wanted to make sure it's known that the setup there would no longer be in any active repo.
The page being edited in this PR :) |
|
Merging this to fix the wrong and broken links. Thanks @jackbtlr for the PR. Feel free to edit this document as needed in a follow up @rhshadrach |
doc/source/whatsnew/vX.X.X.rstfile if fixing a bug or adding a new feature.