Add Koios benchmark QoR example to the vtr developer guide, and add koios to CI

[vaughnbetz]

The Koios benchmarks have been integrated into the vtr repo, but to have the most impact, we should do two more things:

1. Add a short description (and links to the benchmark README) to the VTR developer guide under the QoR Collection section: https://docs.verilogtorouting.org/en/latest/dev/developing/#collecting-qor-measurements
   It should come after the Titan benchmarks (or maybe after the VTR benchmarks if that flows better as they go all the way through odinII ...)_ and have a similar style.

2. The vtr_reg_nightly4 suite that runs them should be added to CI. @ArashAhmadian can work with you to make the necessary changes and pin Tim Ansell to make the eventual google-side changes.

[aman26kbm]

Thanks for filing this, Vaughn.

I will add to task #2. I need to add protection of benchmarks without the "`define complex_dsp". And I need to update the documentation to mention how to run benchmarks with and without complex_dsp (the new include file method).

I will work on these within next week.

[aman26kbm]

I've created a new PR which includes the changes for task #1 mentioned in this issue.

@ArashAhmadian , can you please guide me on what changes to make changes for task #2? I'll make those changes and push them to this new PR I've created.

[Edit: Fix typo in task number]

[ArashAhmadian]

@aman26kbm Sure, do you mean changes to add vtr_reg_nightly_test4 to CI? Your previous PR actually added the .cfg file and made the necessary changes in vtr_test.sh so I think everything is good from our side.

[aman26kbm]

Right. That's great. (Sorry for the typo in my previous comment. I meant to point to task #2 in the question I asked :P. Fixing it now)
So, that means the Koios tests have been running in nightly regressions over the last week?

[ArashAhmadian]

I don't see them being run. I think @mithro needs to add the configs on the Google side of things for them to start running.

[aman26kbm]

Ok. Thanks. We can discuss this with @mithro in this week's VTR meeting on Thursday.

[ArashAhmadian]

Sounds good.

[ArashAhmadian]

@aman26kbm What's your email? I am drafting the email to Arnaud and I can put you on it.

[vaughnbetz]

Aman is at [email protected]

[github-actions]

This issue has been inactive for a year and has been marked as stale. It will be closed in 15 days if it continues to be stale. If you believe this is still an issue, please add a comment.

[vaughnbetz]

This has been completed; marking closed.