Skip to content

Commit d86017a

Browse files
Changing vtr_reg_nightly description.
1 parent d377237 commit d86017a

File tree

1 file changed

+21
-42
lines changed

1 file changed

+21
-42
lines changed

README.developers.md

Lines changed: 21 additions & 42 deletions
Original file line numberDiff line numberDiff line change
@@ -183,7 +183,7 @@ Python files are automatically checked using `pylint` to ensure they follow esta
183183

184184
VTR has a variety of tests which are used to check for correctness, performance and Quality of Result (QoR).
185185

186-
There are 4 main regression tests:
186+
There are 4 main regression testing suites:
187187

188188
* `vtr_reg_basic`: ~1 minute serial
189189

@@ -216,54 +216,33 @@ There are 4 main regression tests:
216216

217217
QoR checks in this regression test are primarily 'canary' checks to catch gross degradations in QoR.
218218
Occasionally, changes can cause QoR failures (e.g. due to CAD noise -- particularly on small benchmarks); usually such failures are not a concern if the QoR differences are small.
219+
220+
* `vtr_reg_nightly_test#, #:1-3`:
219221

220-
* `vtr_reg_nightly_test1`: ~4.5 hours with `-j8`
221-
222-
**Goal:** Basic QoR and Performance evaluation.
223-
222+
**Goal:** Basic QoR and Performance evaluation
223+
224224
**Feature Coverage:** Medium
225-
226-
**Benchmarks:** Small-medium size, diverse. Includes:
227-
228-
* ISPD benchmarks
229-
* MCNC20 benchmarks
230-
* VTR benchmarks
231-
225+
232226
**Architectures:** A wider variety of architectures
233-
234-
QoR checks in this regression are aimed at evaluating quality and run-time of the VTR flow.
235-
As a result any QoR failures are a concern and should be investigated and understood.
236-
237-
* `vtr_reg_nightly_test2`: ~6 hours with `-j8`
238-
239-
**Goal:** Basic QoR and Performance evaluation.
240-
241-
**Feature Coverage:** Medium
242-
243-
**Benchmarks:** Small-medium size, diverse. Includes:
244-
227+
228+
**Benchmarks:** Small-medium size, diverse. All include:
229+
245230
* VTR benchmarks
246-
* Titan 'other' benchmarks (smaller than Titan23)
247-
248-
**Architectures:** A wider variety of architectures
231+
* Additional benchmarks for each suite.
249232

250-
QoR checks in this regression are aimed at evaluating quality and run-time of the VTR flow.
233+
QoR checks in these regression suites are aimed at evaluating quality and run-time of the VTR flow.
251234
As a result any QoR failures are a concern and should be investigated and understood.
252235

253-
* `vtr_reg_nightly_test3`: ~5.5 hours with `-j8`
254-
255-
**Goal:** Basic QoR and Performance evaluation.
256-
257-
**Feature Coverage:** Medium
258-
259-
**Benchmarks:** Small-medium size, diverse. Includes:
260-
* VTR benchmarks
261-
262-
**Architectures:** A wider variety of architectures
263-
264-
QoR checks in this regression are aimed at evaluating quality and run-time of the VTR flow.
265-
As a result any QoR failures are a concern and should be investigated and understood.
266-
236+
Note:
237+
238+
These suites comproise a single large suite, `vtr_reg_nightly` and should be run together to test nightly level regression. They are mostly similar in benchmark coverage interms of size and diversity however each suite tests some unique benchmarks in addition to the VTR benchmarks.
239+
240+
| suite | wall-clock time| Additional benchmarks|
241+
|-------|----------------|----------------------|
242+
|vtr_reg_nightly_test1|~4.5 hours with `-j8`|ISPD and MCNC20 |
243+
|vtr_reg_nightly_test2|~6 hours with `-j8`|Titan23 and Titan `other`|
244+
|vtr_reg_nightly_test3|~5.5 hours with `-j8`|none|
245+
267246
* `vtr_reg_weekly`: ~42 hours with `-j4`
268247

269248
**Goal:** Full QoR and Performance evaluation.

0 commit comments

Comments
 (0)