You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.developers.md
+24-8Lines changed: 24 additions & 8 deletions
Original file line number
Diff line number
Diff line change
@@ -199,7 +199,7 @@ There are 4 main regression testing suites:
199
199
**Architectures:** A few simple architectures
200
200
201
201
This regression test is *not* suitable for evaluating QoR or performance.
202
-
It's primary purpose is to make sure the various tools do not crash/fail in the basic VTR flow.
202
+
Its primary purpose is to make sure the various tools do not crash/fail in the basic VTR flow.
203
203
204
204
QoR checks in this regression test are primarily 'canary' checks to catch gross degradations in QoR.
205
205
Occasionally, code changes can cause QoR failures (e.g. due to CAD noise -- particularly on small benchmarks); usually such failures are not a concern if the QoR differences are small.
@@ -217,7 +217,7 @@ Occasionally, code changes can cause QoR failures (e.g. due to CAD noise -- part
217
217
**Architectures:** A variety of architectures, including special architectures to exercise specific features
218
218
219
219
This regression test is *not* suitable for evaluating QoR or performance.
220
-
It's primary purpose is try and achieve high functionality coverage.
220
+
Its primary purpose is try and achieve high functionality coverage.
221
221
222
222
QoR checks in this regression test are primarily 'canary' checks to catch gross degradations in QoR.
223
223
Occasionally, changes can cause QoR failures (e.g. due to CAD noise -- particularly on small benchmarks); usually such failures are not a concern if the QoR differences are small.
@@ -564,6 +564,10 @@ You need at least two sets of QoR measurements:
564
564
1. The baseline QoR (i.e. unmodified VTR).
565
565
2. The modified QoR (i.e. VTR with your changes).
566
566
567
+
The following tests can be run locally by running the given commands on the local machine. In addition, since CI tests are run whenever
568
+
changes are pushed to the remote repository, one can use the CI test results to measure the impact
569
+
of his/her changes. The instructions to gather CI tests' results are [here](./README.developers.md#Example:-CI-Tests-QoR-Measurement).
570
+
567
571
Note that it is important to generate both sets of QoR measurements on the same computing infrastructure to ensure a fair run-time comparison.
568
572
569
573
The following examples show how a single set of QoR measurements can be produced using the VTR flow infrastructure.
@@ -696,15 +700,27 @@ This would download the results for all CI tests.
696
700
3. Scroll down and download "artifact"
697
701

698
702
699
-
Assume that we want to get the QoR results for "vtr_reg_nightly_test3". In artifact, there is a file named
700
-
"qor_results_vtr_reg_nightly_test3.tar.gz." After unzipping this file, a new directory named "vtr_flow" would be
701
-
created. Go to "vtr_flow/tasks/regression_tests/vtr_reg_nightly_test3." In this directory, you can find the **parsed**
702
-
results of this test.
703
-
704
-
In the case that you want to re-run the CI tests, go to the "Action" tab and find your workflow under Test Workflow.
703
+
Assume that we want to get the QoR results for "vtr_reg_nightly_test3". In the artifact, there is a file named
704
+
"qor_results_vtr_reg_nightly_test3.tar.gz." Unzip this file, and a new directory named "vtr_flow" is created. Go to
705
+
"vtr_flow/tasks/regression_tests/vtr_reg_nightly_test3." In this directory, you can find a directory for each test circuit
706
+
containing in this test suit(vtr_reg_nightly_test3.) For instance, results related to *vtr_reg_qor* are located in
707
+
"vtr_flow/tasks/regression_tests/vtr_reg_nightly_test3/vtr_reg_qor." In this directory, results for each run are stored separately.
708
+
For example, if you want to get the results for the first run, there is a file in *run001* directory named *parse_results.txt*.
709
+
Using these parsed results, you can do a detailed QoR comparison using the instructions given [here](./README.developers.md#Comparing QoR Measurements).
0 commit comments