Regression test to check incremental timing analysis results/qor match full timing analysis during optimization

[vaughnbetz]

Proposed Behaviour

We now have an incremental STA mode that is much faster for small changes in the optimizers. To confirm it always gets the same results as the full timing analysis, we should have one or two regtests on small circuits that check that we get the same results (should be identical placement output file, and identical routing output file) with incremental vs. full static timing analysis guiding the optimizers.

I think the best regtest would be two circuits, on the flagship architecture (https://github.com/verilog-to-routing/vtr-verilog-to-routing/blob/master/vtr_flow/arch/timing/k6_frac_N10_frac_chain_mem32K_40nm.xml).
Suggested circuits:
or1200.v (small, good for a quick test, put in vtr_reg_strong)
LU8PEEng.v (bigger, tests more blocks, put in vtr_reg_nightly)
Both are in https://github.com/verilog-to-routing/vtr-verilog-to-routing/blob/master/vtr_flow/benchmarks/verilog/LU8PEEng.v

The VTR developer guide has information on how to create a regtest, and starting with an existing regtest as a template would be a good idea. See https://docs.verilogtorouting.org/en/latest/dev/developing/#adding-tests

[github-actions]

This issue has been inactive for a year and has been marked as stale. It will be closed in 15 days if it continues to be stale. If you believe this is still an issue, please add a comment.

[github-actions]

This issue has been marked stale for 15 days and has been automatically closed.

[vaughnbetz]

We do check this now, using the "-check_incremental_sta_consistency" option to a task. Looking at the tests, they look pretty compete except I think it would be good to have a test with the multiclock circuit that uses an sdc. Sending to Alex to see if he can do that. We can just make a new test that runs that one circuit with this option.

[AlexandreSinger]

PR #3058 has been raised to add this extra test @vaughnbetz

[vaughnbetz]

Great, thanks!