|
| 1 | +# Verify Script |
| 2 | + |
| 3 | +The verify_odin.sh script is designed for generating regression test results. |
| 4 | + |
| 5 | +./verify_odin.sh [args] |
| 6 | + |
| 7 | +## Arguments |
| 8 | + |
| 9 | +*The tool requires a task to run hence `-t <task directory>` must be passed in |
| 10 | + |
| 11 | +| arg | equivalent form | Following argument | Description | |
| 12 | +|-------|---------------------------|-------------------------------------|------------------------------------------------------------------------------------------------| |
| 13 | +| `-t` | path to directory containing a test |The test types are a task.conf,task_list.conf or a predefined test | |
| 14 | +| `-j` | `-- nb_of_process` | # of processors requested | This allows the user to choose how many processor are used in the computer, the default is 1 | |
| 15 | +| `-d` | `--output_dir` | path to directory | This allows the user to change the directory of the run output | |
| 16 | +| `-C` | `--config` | path to configuration file | This directs the compiler to a configuration file to append the configuration file of the test| |
| 17 | +| `-t` | `--test` | path to directory containing a test | The test types are a task.conf,task_list.conf or a predefined test | |
| 18 | +| `-h` | `--help` | N/A | Prints all possible commands and directives | |
| 19 | +| `-g` | `--generate_bench` | N/A | Generates input and output vectors for test | |
| 20 | +| `-o` | `--generate_output` | N/A | Generates output vector for test given its input vector | |
| 21 | +| `-b` | `--build_config` | path to directory | Generates a configuration file for a given directory | |
| 22 | +| `-c` | `--clean` | N/A | Clean temporary directory | |
| 23 | +| `-f` | `--force_simulate` | N/A | Force the simulation to be executed regardless of the configuration file | |
| 24 | +| N/A | `--override` | N/A | if a configuration file is passed in, override arguments rather than append | |
| 25 | +| N/A | `--dry_run` | N/A | performs a dry run to check the validity of the task and flow | |
| 26 | +| N/A | `--randomize` | N/A | performs a dry run randomly to check the validity of the task and flow | |
| 27 | +| N/A | `--regenerate_expectation`| N/A | regenerates expectation and overrides the expected value only if there's a mismatch | |
| 28 | +| N/A | `--generate_expectation` | N/A | generate the expectation and overrides the expectation file | |
| 29 | + |
| 30 | +## Examples |
| 31 | + |
| 32 | +The following examples are being performed in the ODIN_II directory: |
| 33 | + |
| 34 | +### Generating Results for a New Task |
| 35 | + |
| 36 | +The following commands will generate the results of a new regression test using N processors: |
| 37 | + |
| 38 | +```bash |
| 39 | +make sanitize |
| 40 | +``` |
| 41 | + |
| 42 | +```bash |
| 43 | +./verify_odin.sh --generate_expectation -j N -t <regression_test/benchmark/task/<task_name> |
| 44 | +``` |
| 45 | + |
| 46 | +A synthesis_result.json and a simulation_result.json will be generated in the task's folder. |
| 47 | +The simulation results for each benchmark are only generated if they syntehsize correctly (no exit error), thus if none of the benchmarks synthesize there will be no simulation_result.json generated. |
| 48 | + |
| 49 | +### Regenerating Results for a Changed Test |
| 50 | + |
| 51 | +The following commands will only generate the results of the changes. |
| 52 | +If there are new benchmarks it will add to the results. |
| 53 | +If there are deleted benchmarks or modified benchmarks the results will be updated accordingly. |
| 54 | + |
| 55 | +```bash |
| 56 | +make sanitize |
| 57 | +``` |
| 58 | + |
| 59 | +```bash |
| 60 | +./verify_odin.sh --regenerate_expectation -t <regression_test/benchmark/task/<task_name> |
| 61 | +``` |
| 62 | + |
| 63 | +### Generating Results for a Suite |
| 64 | + |
| 65 | +The following commands generate the results for all the tasks called upon in a suite. |
| 66 | + |
| 67 | +```bash |
| 68 | +make sanitize |
| 69 | +``` |
| 70 | + |
| 71 | +> **NOTE** |
| 72 | +> |
| 73 | +> If the suite calls upon the large test **DO NOT** run `make sanitize`. |
| 74 | +> Instead run `make build`. |
| 75 | +
|
| 76 | +```bash |
| 77 | +./verify_odin.sh --generate_expectation -t <regression_test/benchmark/suite/<suite_name> |
| 78 | +``` |
| 79 | + |
| 80 | +### Checking the configuration file |
| 81 | + |
| 82 | +The following commands will check if a configuration file is being read properly. |
| 83 | + |
| 84 | +```bash |
| 85 | +make build |
| 86 | +``` |
| 87 | + |
| 88 | +```bash |
| 89 | +./verify_odin.sh --dry_run -t <regression_test/benchmark/<path/to/config_file/difrectory> |
| 90 | +``` |
0 commit comments