Skip to content

Commit 51c358a

Browse files
sfkhalidvaughnbetz
authored andcommitted
Updated documentation for generating a new golden results for a failing test
1 parent 4c1c28a commit 51c358a

File tree

1 file changed

+22
-0
lines changed

1 file changed

+22
-0
lines changed

README.developers.md

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -738,6 +738,28 @@ $ ./vtr_flow/scripts/qor_compare.py parse_results1.txt parse_results2.txt parse_
738738
```
739739
will produce ratio tables and a summary table for the files parse_results1.txt, parse_results2.txt and parse_results3.txt, where the first file (parse_results1.txt) is assumed to be the baseline used to produce normalized ratios.
740740

741+
### Generating New QoR Golden Result
742+
There may be times when a regression test fails its QoR test because its golden_result needs to be changed due to known changes in code behaviour. In this case, a new golden result needs to be generated so that the test can be passed. To generate a new golden result, follow the steps outlined below.
743+
744+
1. Move to the `vtr_flow/tasks` directory from the VTR root, and run the failing test. For example, if a test called `vtr_ex_test` in `vtr_reg_nightly` was failing:
745+
746+
```shell
747+
#From the VTR root
748+
$ cd vtr_flow/tasks
749+
$ ../scripts/run_vtr_task.pl regression_tests/vtr_reg_nightly/vtr_ex_test
750+
```
751+
2. Next, generate new golden reference results using `parse_vtr_task.pl` and the `-create_golden` option.
752+
753+
```shell
754+
$ ../scripts/parse_vtr_task.pl regression_tests/vtr_reg_nightly/vtr_ex_test -create_golden
755+
```
756+
3. Lastly, check that the results match with the `-check_golden` option
757+
758+
```shell
759+
$ ../scripts/parse_vtr_task.pl regression_tests/vtr_reg_nightly/vtr_ex_test -check_golden
760+
```
761+
Once the `-check_golden` command passes, the changes to the golden result can be committed so that the reg test will pass in future runs of vtr_reg_nightly.
762+
741763
# Adding Tests
742764

743765
Any time you add a feature to VTR you **must** add a test which exercises the feature.

0 commit comments

Comments
 (0)