Skip to content

Vtr log changes #1637

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
May 28, 2021
Merged

Vtr log changes #1637

merged 20 commits into from
May 28, 2021

Conversation

sfkhalid
Copy link
Contributor

Made changes to vpr_stdout.log output

Description

  1. Moved the printing of the VPR options to come before the circuit stats printing and the Building complex block graph message
  2. Deleted the RoutingArch lines that used to be printed directly after the options

Related Issue

These changes were suggested in issue #1617

@github-actions github-actions bot added the VPR VPR FPGA Placement & Routing Tool label Jan 18, 2021
@sfkhalid
Copy link
Contributor Author

sfkhalid commented Jan 18, 2021

While making these changes, I noticed that there are some functions in vpr_api.cpp that are written but never called (ex. vpr_show_setup, vpr_setup_vpr). The api code just directly calls the functions (ShowSetup, SetupVPR) that are called within vpr_show_setup and vpr_setup_vpr.
Since these functions are unused, it would probably be best to delete them.

@sfkhalid
Copy link
Contributor Author

@litghost nightly has been taking a long time on this.

@litghost
Copy link
Collaborator

litghost commented Feb 11, 2021

This job is just running normally, right now. Long poles are:

kbuilder 28712 99.9  0.3 381164 353208 ?       R    10:02  43:48 /tmpfs/src/github/vtr-verilog-to-routing/abc/abc -c echo ""; echo "Load Netlist"; echo "============"; read 0_mcml.odin.blif; time; echo ""; echo "Circuit Info"; echo "=========="; print_stats; print_latch; time; echo ""; echo "LUT Costs"; echo "========="; print_lut; time; echo ""; echo "Logic Opt + Techmap"; echo "==================="; strash; ifraig -v; scorr -v; dc2 -v; dch -f; if -K 4 -v; mfs2 -v; print_stats; time; echo ""; echo "Output Netlist"; echo "=============="; write_hie 0_mcml.odin.blif 0_mcml.raw.abc.blif; time;
kbuilder 30058  0.0  0.0   4360   652 ?        S    10:32   0:00 time -v /tmpfs/src/github/vtr-verilog-to-routing/vpr/vpr k4_N8_topology-0.85sL2-0.15gL4-on-cb-off-sb_22nm_22nm.xml LU32PEEng --circuit_file LU32PEEng.pre-vpr.blif --route_chan_width 200 --max_router_iterations 150
kbuilder 30059  100  1.9 2319160 2128196 ?     R    10:32  15:23 /tmpfs/src/github/vtr-verilog-to-routing/vpr/vpr k4_N8_topology-0.85sL2-0.15gL4-on-cb-off-sb_22nm_22nm.xml LU32PEEng --circuit_file LU32PEEng.pre-vpr.blif --route_chan_width 200 --max_router_iterations 150
kbuilder 31177  0.0  0.0   4360   636 ?        S    10:47   0:00 time -v /tmpfs/src/github/vtr-verilog-to-routing/vpr/vpr k4_n4_v7_l1_bidir.xml s38417 --circuit_file s38417.pre-vpr.blif --seed 250 --min_route_chan_width_hint 18
kbuilder 31178 98.9  0.1 299304 133412 ?       R    10:47   0:20 /tmpfs/src/github/vtr-verilog-to-routing/vpr/vpr k4_n4_v7_l1_bidir.xml s38417 --circuit_file s38417.pre-vpr.blif --seed 250 --min_route_chan_width_hint 18

@sfkhalid
Copy link
Contributor Author

@litghost is nightly still running normally here? It has been running for almost 24h now

@litghost
Copy link
Collaborator

@litghost is nightly still running normally here? It has been running for almost 24h now

It has terminated in failure for two reasons. The first was some QoR failures:

++ date
Fri Feb 12 02:52:58 PST 2021
++ uptime
 02:52:58 up 18:05,  0 users,  load average: 1.07, 1.32, 1.68
++ free -h
              total        used        free      shared  buff/cache   available
Mem:           102G        1.9G         22G         16M         77G         99G
Swap:          1.0G         30M        993M
++ sleep 300
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/arm_core.v/common                           		OK (took 467.02 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/stereovision1.v/common                              		OK (took 165.44 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/stereovision0.v/common                              		OK (took 73.60 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/spree.v/common                                      		OK (took 12.31 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/sha.v/common                                        		OK (took 307.19 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/LU8PEEng.v/common                                   		OK (took 774.98 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/diffeq2.v/common                                    		OK (took 9.43 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/diffeq1.v/common                                    		OK (took 8.78 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/ch_intrinsics.v/common                              		OK (took 2.91 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/boundtop.v/common                                   		OK (took 7.44 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/blob_merge.v/common                                 		OK (took 128.34 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/bgm.v/common                                        		OK (took 777.62 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/arm_core.v/common                                   		OK (took 395.01 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/tseng.pre-vpr.blif/common                                             		OK (took 3.59 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/spla.pre-vpr.blif/common                                              		OK (took 17.77 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/seq.pre-vpr.blif/common                                               		OK (took 6.65 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/s38584.1.pre-vpr.blif/common                                          		OK (took 15.80 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/s38417.pre-vpr.blif/common                                            		OK (took 16.66 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/s298.pre-vpr.blif/common                                              		OK (took 3.71 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/pdc.pre-vpr.blif/common                                               		OK (took 27.46 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/misex3.pre-vpr.blif/common                                            		OK (took 4.77 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/frisc.pre-vpr.blif/common                                             		OK (took 13.39 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/ex5p.pre-vpr.blif/common                                              		OK (took 3.98 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/ex1010.pre-vpr.blif/common                                            		OK (took 31.31 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/elliptic.pre-vpr.blif/common                                          		OK (took 12.42 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/dsip.pre-vpr.blif/common                                              		OK (took 6.47 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/diffeq.pre-vpr.blif/common                                            		OK (took 3.85 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/des.pre-vpr.blif/common                                               		OK (took 6.34 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/clma.pre-vpr.blif/common                                              		OK (took 33.26 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/bigkey.pre-vpr.blif/common                                            		OK (took 6.66 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/apex4.pre-vpr.blif/common                                             		OK (took 5.78 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/apex2.pre-vpr.blif/common                                             		OK (took 7.31 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/alu4.pre-vpr.blif/common                                              		OK (took 4.82 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/mkSMAdapter4B.v/common                              		OK (took 31.56 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/mkPktMerge.v/common                                 		OK (took 15.42 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/mcml.v/common                                       		OK (took 5611.61 seconds)
symbiflow:                                 arch.timing.xml/minilitex_ddr_eth_arty.eblif/common                                        		OK (took 636.25 seconds)
symbiflow:                                 arch.timing.xml/picosoc_basys3_full_100.eblif/common                                       		OK (took 241.72 seconds)
vtr_timing_update_diff:                    k6_frac_N10_frac_chain_mem32K_40nm.xml/LU8PEEng.v/common                                   		OK (took 620.39 seconds)
vpr_verify_rr_graph_complex_switch:        complex_switch/k4_N8_topology-0.85sL2-0.15gL4-on-cb-off-sb_22nm_22nm.xml/raygentop.v/common		OK (took 37.04 seconds)
vpr_verify_rr_graph_bidir:                 k4_n4_v7_bidir.xml/elliptic.blif/common                                                    		OK (took 55.18 seconds)
vpr_verify_rr_graph_bidir:                 k4_n4_v7_bidir.xml/dsip.blif/common                                                        		OK (took 33.51 seconds)
vpr_verify_rr_graph:                       hard_fpu_arch_timing.xml/raygentop.v/common                                                		OK (took 180.79 seconds)
vpr_verify_rr_graph:                       k6_N10_mem32K_40nm.xml/raygentop.v/common                                                  		OK (took 22.97 seconds)
vpr_verify_rr_graph:                       k6_frac_N10_mem32K_40nm.xml/raygentop.v/common                                             		OK (took 20.60 seconds)
vpr_verify_rr_graph:                       k6_frac_N10_frac_chain_mem32K_40nm.xml/raygentop.v/common                                  		OK (took 19.41 seconds)
vpr_verify_rr_graph:                       k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/raygentop.v/common                          		OK (took 17.25 seconds)
complex_switch:                            k4_N8_topology-0.85sL2-0.15gL4-on-cb-off-sb_22nm_22nm.xml/stereovision2.v/common           		OK (took 1299.04 seconds)
complex_switch:                            k4_N8_topology-0.85sL2-0.15gL4-on-cb-off-sb_22nm_22nm.xml/or1200.v/common                  		OK (took 32.97 seconds)
complex_switch:                            k4_N8_topology-0.85sL2-0.15gL4-on-cb-off-sb_22nm_22nm.xml/mcml.v/common                    		OK (took 10174.07 seconds)
titan_quick_qor:                           stratixiv_arch.timing.xml/segmentation_stratixiv_arch_timing.blif/common                   		OK (took 1854.92 seconds)
titan_quick_qor:                           stratixiv_arch.timing.xml/stap_qrd_stratixiv_arch_timing.blif/common                       		OK (took 2684.28 seconds)
titan_quick_qor:                           stratixiv_arch.timing.xml/sparcT2_core_stratixiv_arch_timing.blif/common                   		OK (took 3682.35 seconds)
titan_quick_qor:                           stratixiv_arch.timing.xml/mes_noc_stratixiv_arch_timing.blif/common                        		OK (took 7412.28 seconds)
titan_quick_qor:                           stratixiv_arch.timing.xml/LU230_stratixiv_arch_timing.blif/common                          		OK (took 13729.68 seconds)
vtr_func_formal:                           k6_frac_N10_40nm.xml/stereovision3.v/common                                                		OK (took 3.01 seconds)
vtr_func_formal:                           k6_frac_N10_40nm.xml/sha.v/common                                                          		OK (took 2303.62 seconds)
vtr_reg_coffe:                             agilex_arch.xml/matmul_8x8_fp16.v/common                                                   		OK (took 895.02 seconds)
vtr_reg_coffe:                             agilex_arch.xml/or1200.v/common                                                            		OK (took 1083.08 seconds)
vtr_reg_coffe:                             agilex_arch.xml/spree.v/common                                                             		OK (took 873.10 seconds)
vtr_reg_netlist_writer:                    k6_frac_N10_frac_chain_mem32K_40nm.xml/raygentop.v/common                                  		OK (took 22.82 seconds)
vtr_reg_netlist_writer:                    k6_frac_N10_frac_chain_mem32K_40nm.xml/boundtop.v/common                                   		OK (took 7.63 seconds)
vtr_reg_netlist_writer:                    k6_frac_N10_frac_chain_mem32K_40nm.xml/sha.v/common                                        		OK (took 325.49 seconds)
vtr_reg_netlist_writer:                    k6_frac_N10_frac_chain_mem32K_40nm.xml/or1200.v/common                                     		OK (took 55.06 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/stereovision3.v/common                      		OK (took 1.81 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/stereovision2.v/common                      		OK (took 701.90 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/stereovision1.v/common                      		OK (took 314.15 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/stereovision0.v/common                      		OK (took 78.58 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/spree.v/common                              		OK (took 14.57 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/sha.v/common                                		OK (took 330.28 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/raygentop.v/common                          		OK (took 21.22 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/or1200.v/common                             		OK (took 81.88 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/mkSMAdapter4B.v/common                      		OK (took 28.32 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/mkPktMerge.v/common                         		OK (took 17.76 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/mkDelayWorker32B.v/common                   		OK (took 98.54 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/mcml.v/common                               		OK (took 7055.10 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/diffeq2.v/common                            		OK (took 11.84 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/diffeq1.v/common                            		OK (took 9.76 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/ch_intrinsics.v/common                      		OK (took 2.98 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/blob_merge.v/common                         		OK (took 138.98 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/stereovision3.v/common                              		OK (took 1.88 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/stereovision2.v/common                              		OK (took 617.32 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/raygentop.v/common                                  		OK (took 22.05 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/or1200.v/common                                     		OK (took 53.56 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/mkDelayWorker32B.v/common                           		OK (took 86.51 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/LU32PEEng.v/common                                  		OK (took 5924.46 seconds)
Elapsed time: 63938.01 seconds

Parsing test results...
scripts/parse_vtr_task.py -l /tmpfs/src/github/vtr-verilog-to-routing/vtr_flow/tasks/regression_tests/vtr_reg_nightly/task_list.txt

Calculating QoR results...
regression_tests/vtr_reg_nightly/vpr_reg_mcnc...[Pass]
regression_tests/vtr_reg_nightly/vtr_reg_qor_chain...[Pass]
regression_tests/vtr_reg_nightly/vtr_reg_qor_chain_depop...[Pass]
regression_tests/vtr_reg_nightly/vtr_reg_netlist_writer...[Pass]

regression_tests/vtr_reg_nightly/vtr_reg_coffe...[Fail]
[Fail]
agilex_arch.xml/spree.v/common pack_time relative value 0.030759493670886078 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 79.0
[Fail]
agilex_arch.xml/spree.v/common place_time relative value 0.030336157420060125 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 36.59
[Fail]
agilex_arch.xml/spree.v/common min_chan_width_route_time relative value 0.045605452836617356 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 6075.37
[Fail]
agilex_arch.xml/spree.v/common crit_path_route_time relative value 0.07420494699646643 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 14.15

regression_tests/vtr_reg_nightly/vtr_reg_coffe...[Fail]
[Fail]
agilex_arch.xml/or1200.v/common pack_time relative value 0.016648396821046862 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 437.88
[Fail]
agilex_arch.xml/or1200.v/common place_time relative value 0.024406356663118873 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 272.47
[Fail]
agilex_arch.xml/or1200.v/common min_chan_width_route_time relative value 0.045640079997816815 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 10260.28
[Fail]
agilex_arch.xml/or1200.v/common crit_path_route_time relative value 0.055947580645161296 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 39.68

regression_tests/vtr_reg_nightly/vtr_reg_coffe...[Fail]
[Fail]
agilex_arch.xml/matmul_8x8_fp16.v/common pack_time relative value 0.028288774425639766 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 244.62
[Fail]
agilex_arch.xml/matmul_8x8_fp16.v/common place_time relative value 0.02862194717648359 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 180.98
[Fail]
agilex_arch.xml/matmul_8x8_fp16.v/common min_chan_width_route_time relative value 0.043253662080501 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 7102.52
[Fail]
agilex_arch.xml/matmul_8x8_fp16.v/common crit_path_route_time relative value 0.047923907078836654 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 54.67
regression_tests/vtr_reg_nightly/vtr_func_formal...[Pass]
regression_tests/vtr_reg_nightly/titan_quick_qor...[Pass]
regression_tests/vtr_reg_nightly/titan_other...[Pass]
regression_tests/vtr_reg_nightly/vtr_bidir...[Pass]
regression_tests/vtr_reg_nightly/complex_switch...[Pass]
regression_tests/vtr_reg_nightly/vpr_verify_rr_graph...[Pass]
regression_tests/vtr_reg_nightly/vpr_verify_rr_graph_bidir...[Pass]
regression_tests/vtr_reg_nightly/vpr_verify_rr_graph_complex_switch...[Pass]
regression_tests/vtr_reg_nightly/vpr_verify_rr_graph_titan...[Pass]
regression_tests/vtr_reg_nightly/vpr_verify_rr_graph_error_check...[Pass]
regression_tests/vtr_reg_nightly/vtr_timing_update_diff...[Pass]
regression_tests/vtr_reg_nightly/vtr_timing_update_diff_titan...[Pass]
Warning: task includes result for arch.timing.xml/picosoc_basys3_full_100.eblif missing in golden results
Warning: task includes result for arch.timing.xml/picosoc_basys3_full_50.eblif missing in golden results
Warning: task includes result for arch.timing.xml/linux_arty.eblif missing in golden results
Warning: task includes result for arch.timing.xml/minilitex_arty.eblif missing in golden results
Warning: task includes result for arch.timing.xml/minilitex_ddr_arty.eblif missing in golden results
Warning: task includes result for arch.timing.xml/minilitex_ddr_eth_arty.eblif missing in golden results
regression_tests/vtr_reg_nightly/symbiflow...[Pass]

Test 'vtr_reg_nightly' had 12 qor test failures

Test 'vtr_reg_nightly' had 0 run failures

Error: 12 tests failed

The other reason is that the working space for the job exceeded 50 GiB (or 100 GiB). We likely need to cleanup the workspace after running to remove files that consume space. See what was done over here for inspiration: https://github.com/SymbiFlow/symbiflow-arch-defs/blob/45c83a2592923b9a785aa05c9464b5f453ebd151/.github/kokoro/package_results.sh#L12-L48

@sfkhalid
Copy link
Contributor Author

@litghost is nightly still running normally here? It has been running for almost 24h now

It has terminated in failure for two reasons. The first was some QoR failures:

++ date
Fri Feb 12 02:52:58 PST 2021
++ uptime
 02:52:58 up 18:05,  0 users,  load average: 1.07, 1.32, 1.68
++ free -h
              total        used        free      shared  buff/cache   available
Mem:           102G        1.9G         22G         16M         77G         99G
Swap:          1.0G         30M        993M
++ sleep 300
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/arm_core.v/common                           		OK (took 467.02 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/stereovision1.v/common                              		OK (took 165.44 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/stereovision0.v/common                              		OK (took 73.60 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/spree.v/common                                      		OK (took 12.31 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/sha.v/common                                        		OK (took 307.19 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/LU8PEEng.v/common                                   		OK (took 774.98 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/diffeq2.v/common                                    		OK (took 9.43 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/diffeq1.v/common                                    		OK (took 8.78 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/ch_intrinsics.v/common                              		OK (took 2.91 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/boundtop.v/common                                   		OK (took 7.44 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/blob_merge.v/common                                 		OK (took 128.34 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/bgm.v/common                                        		OK (took 777.62 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/arm_core.v/common                                   		OK (took 395.01 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/tseng.pre-vpr.blif/common                                             		OK (took 3.59 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/spla.pre-vpr.blif/common                                              		OK (took 17.77 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/seq.pre-vpr.blif/common                                               		OK (took 6.65 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/s38584.1.pre-vpr.blif/common                                          		OK (took 15.80 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/s38417.pre-vpr.blif/common                                            		OK (took 16.66 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/s298.pre-vpr.blif/common                                              		OK (took 3.71 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/pdc.pre-vpr.blif/common                                               		OK (took 27.46 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/misex3.pre-vpr.blif/common                                            		OK (took 4.77 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/frisc.pre-vpr.blif/common                                             		OK (took 13.39 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/ex5p.pre-vpr.blif/common                                              		OK (took 3.98 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/ex1010.pre-vpr.blif/common                                            		OK (took 31.31 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/elliptic.pre-vpr.blif/common                                          		OK (took 12.42 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/dsip.pre-vpr.blif/common                                              		OK (took 6.47 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/diffeq.pre-vpr.blif/common                                            		OK (took 3.85 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/des.pre-vpr.blif/common                                               		OK (took 6.34 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/clma.pre-vpr.blif/common                                              		OK (took 33.26 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/bigkey.pre-vpr.blif/common                                            		OK (took 6.66 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/apex4.pre-vpr.blif/common                                             		OK (took 5.78 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/apex2.pre-vpr.blif/common                                             		OK (took 7.31 seconds)
vpr_reg_mcnc:                              k6_frac_N10_40nm.xml/alu4.pre-vpr.blif/common                                              		OK (took 4.82 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/mkSMAdapter4B.v/common                              		OK (took 31.56 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/mkPktMerge.v/common                                 		OK (took 15.42 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/mcml.v/common                                       		OK (took 5611.61 seconds)
symbiflow:                                 arch.timing.xml/minilitex_ddr_eth_arty.eblif/common                                        		OK (took 636.25 seconds)
symbiflow:                                 arch.timing.xml/picosoc_basys3_full_100.eblif/common                                       		OK (took 241.72 seconds)
vtr_timing_update_diff:                    k6_frac_N10_frac_chain_mem32K_40nm.xml/LU8PEEng.v/common                                   		OK (took 620.39 seconds)
vpr_verify_rr_graph_complex_switch:        complex_switch/k4_N8_topology-0.85sL2-0.15gL4-on-cb-off-sb_22nm_22nm.xml/raygentop.v/common		OK (took 37.04 seconds)
vpr_verify_rr_graph_bidir:                 k4_n4_v7_bidir.xml/elliptic.blif/common                                                    		OK (took 55.18 seconds)
vpr_verify_rr_graph_bidir:                 k4_n4_v7_bidir.xml/dsip.blif/common                                                        		OK (took 33.51 seconds)
vpr_verify_rr_graph:                       hard_fpu_arch_timing.xml/raygentop.v/common                                                		OK (took 180.79 seconds)
vpr_verify_rr_graph:                       k6_N10_mem32K_40nm.xml/raygentop.v/common                                                  		OK (took 22.97 seconds)
vpr_verify_rr_graph:                       k6_frac_N10_mem32K_40nm.xml/raygentop.v/common                                             		OK (took 20.60 seconds)
vpr_verify_rr_graph:                       k6_frac_N10_frac_chain_mem32K_40nm.xml/raygentop.v/common                                  		OK (took 19.41 seconds)
vpr_verify_rr_graph:                       k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/raygentop.v/common                          		OK (took 17.25 seconds)
complex_switch:                            k4_N8_topology-0.85sL2-0.15gL4-on-cb-off-sb_22nm_22nm.xml/stereovision2.v/common           		OK (took 1299.04 seconds)
complex_switch:                            k4_N8_topology-0.85sL2-0.15gL4-on-cb-off-sb_22nm_22nm.xml/or1200.v/common                  		OK (took 32.97 seconds)
complex_switch:                            k4_N8_topology-0.85sL2-0.15gL4-on-cb-off-sb_22nm_22nm.xml/mcml.v/common                    		OK (took 10174.07 seconds)
titan_quick_qor:                           stratixiv_arch.timing.xml/segmentation_stratixiv_arch_timing.blif/common                   		OK (took 1854.92 seconds)
titan_quick_qor:                           stratixiv_arch.timing.xml/stap_qrd_stratixiv_arch_timing.blif/common                       		OK (took 2684.28 seconds)
titan_quick_qor:                           stratixiv_arch.timing.xml/sparcT2_core_stratixiv_arch_timing.blif/common                   		OK (took 3682.35 seconds)
titan_quick_qor:                           stratixiv_arch.timing.xml/mes_noc_stratixiv_arch_timing.blif/common                        		OK (took 7412.28 seconds)
titan_quick_qor:                           stratixiv_arch.timing.xml/LU230_stratixiv_arch_timing.blif/common                          		OK (took 13729.68 seconds)
vtr_func_formal:                           k6_frac_N10_40nm.xml/stereovision3.v/common                                                		OK (took 3.01 seconds)
vtr_func_formal:                           k6_frac_N10_40nm.xml/sha.v/common                                                          		OK (took 2303.62 seconds)
vtr_reg_coffe:                             agilex_arch.xml/matmul_8x8_fp16.v/common                                                   		OK (took 895.02 seconds)
vtr_reg_coffe:                             agilex_arch.xml/or1200.v/common                                                            		OK (took 1083.08 seconds)
vtr_reg_coffe:                             agilex_arch.xml/spree.v/common                                                             		OK (took 873.10 seconds)
vtr_reg_netlist_writer:                    k6_frac_N10_frac_chain_mem32K_40nm.xml/raygentop.v/common                                  		OK (took 22.82 seconds)
vtr_reg_netlist_writer:                    k6_frac_N10_frac_chain_mem32K_40nm.xml/boundtop.v/common                                   		OK (took 7.63 seconds)
vtr_reg_netlist_writer:                    k6_frac_N10_frac_chain_mem32K_40nm.xml/sha.v/common                                        		OK (took 325.49 seconds)
vtr_reg_netlist_writer:                    k6_frac_N10_frac_chain_mem32K_40nm.xml/or1200.v/common                                     		OK (took 55.06 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/stereovision3.v/common                      		OK (took 1.81 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/stereovision2.v/common                      		OK (took 701.90 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/stereovision1.v/common                      		OK (took 314.15 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/stereovision0.v/common                      		OK (took 78.58 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/spree.v/common                              		OK (took 14.57 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/sha.v/common                                		OK (took 330.28 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/raygentop.v/common                          		OK (took 21.22 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/or1200.v/common                             		OK (took 81.88 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/mkSMAdapter4B.v/common                      		OK (took 28.32 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/mkPktMerge.v/common                         		OK (took 17.76 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/mkDelayWorker32B.v/common                   		OK (took 98.54 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/mcml.v/common                               		OK (took 7055.10 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/diffeq2.v/common                            		OK (took 11.84 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/diffeq1.v/common                            		OK (took 9.76 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/ch_intrinsics.v/common                      		OK (took 2.98 seconds)
vtr_reg_qor_chain_depop:                   k6_frac_N10_frac_chain_depop50_mem32K_40nm.xml/blob_merge.v/common                         		OK (took 138.98 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/stereovision3.v/common                              		OK (took 1.88 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/stereovision2.v/common                              		OK (took 617.32 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/raygentop.v/common                                  		OK (took 22.05 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/or1200.v/common                                     		OK (took 53.56 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/mkDelayWorker32B.v/common                           		OK (took 86.51 seconds)
vtr_reg_qor_chain:                         k6_frac_N10_frac_chain_mem32K_40nm.xml/LU32PEEng.v/common                                  		OK (took 5924.46 seconds)
Elapsed time: 63938.01 seconds

Parsing test results...
scripts/parse_vtr_task.py -l /tmpfs/src/github/vtr-verilog-to-routing/vtr_flow/tasks/regression_tests/vtr_reg_nightly/task_list.txt

Calculating QoR results...
regression_tests/vtr_reg_nightly/vpr_reg_mcnc...[Pass]
regression_tests/vtr_reg_nightly/vtr_reg_qor_chain...[Pass]
regression_tests/vtr_reg_nightly/vtr_reg_qor_chain_depop...[Pass]
regression_tests/vtr_reg_nightly/vtr_reg_netlist_writer...[Pass]

regression_tests/vtr_reg_nightly/vtr_reg_coffe...[Fail]
[Fail]
agilex_arch.xml/spree.v/common pack_time relative value 0.030759493670886078 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 79.0
[Fail]
agilex_arch.xml/spree.v/common place_time relative value 0.030336157420060125 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 36.59
[Fail]
agilex_arch.xml/spree.v/common min_chan_width_route_time relative value 0.045605452836617356 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 6075.37
[Fail]
agilex_arch.xml/spree.v/common crit_path_route_time relative value 0.07420494699646643 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 14.15

regression_tests/vtr_reg_nightly/vtr_reg_coffe...[Fail]
[Fail]
agilex_arch.xml/or1200.v/common pack_time relative value 0.016648396821046862 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 437.88
[Fail]
agilex_arch.xml/or1200.v/common place_time relative value 0.024406356663118873 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 272.47
[Fail]
agilex_arch.xml/or1200.v/common min_chan_width_route_time relative value 0.045640079997816815 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 10260.28
[Fail]
agilex_arch.xml/or1200.v/common crit_path_route_time relative value 0.055947580645161296 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 39.68

regression_tests/vtr_reg_nightly/vtr_reg_coffe...[Fail]
[Fail]
agilex_arch.xml/matmul_8x8_fp16.v/common pack_time relative value 0.028288774425639766 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 244.62
[Fail]
agilex_arch.xml/matmul_8x8_fp16.v/common place_time relative value 0.02862194717648359 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 180.98
[Fail]
agilex_arch.xml/matmul_8x8_fp16.v/common min_chan_width_route_time relative value 0.043253662080501 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 7102.52
[Fail]
agilex_arch.xml/matmul_8x8_fp16.v/common crit_path_route_time relative value 0.047923907078836654 outside of range [0.1,10.0], above absolute threshold 2.0 and not equal to golden value: 54.67
regression_tests/vtr_reg_nightly/vtr_func_formal...[Pass]
regression_tests/vtr_reg_nightly/titan_quick_qor...[Pass]
regression_tests/vtr_reg_nightly/titan_other...[Pass]
regression_tests/vtr_reg_nightly/vtr_bidir...[Pass]
regression_tests/vtr_reg_nightly/complex_switch...[Pass]
regression_tests/vtr_reg_nightly/vpr_verify_rr_graph...[Pass]
regression_tests/vtr_reg_nightly/vpr_verify_rr_graph_bidir...[Pass]
regression_tests/vtr_reg_nightly/vpr_verify_rr_graph_complex_switch...[Pass]
regression_tests/vtr_reg_nightly/vpr_verify_rr_graph_titan...[Pass]
regression_tests/vtr_reg_nightly/vpr_verify_rr_graph_error_check...[Pass]
regression_tests/vtr_reg_nightly/vtr_timing_update_diff...[Pass]
regression_tests/vtr_reg_nightly/vtr_timing_update_diff_titan...[Pass]
Warning: task includes result for arch.timing.xml/picosoc_basys3_full_100.eblif missing in golden results
Warning: task includes result for arch.timing.xml/picosoc_basys3_full_50.eblif missing in golden results
Warning: task includes result for arch.timing.xml/linux_arty.eblif missing in golden results
Warning: task includes result for arch.timing.xml/minilitex_arty.eblif missing in golden results
Warning: task includes result for arch.timing.xml/minilitex_ddr_arty.eblif missing in golden results
Warning: task includes result for arch.timing.xml/minilitex_ddr_eth_arty.eblif missing in golden results
regression_tests/vtr_reg_nightly/symbiflow...[Pass]

Test 'vtr_reg_nightly' had 12 qor test failures

Test 'vtr_reg_nightly' had 0 run failures

Error: 12 tests failed

The other reason is that the working space for the job exceeded 50 GiB (or 100 GiB). We likely need to cleanup the workspace after running to remove files that consume space. See what was done over here for inspiration: https://github.com/SymbiFlow/symbiflow-arch-defs/blob/45c83a2592923b9a785aa05c9464b5f453ebd151/.github/kokoro/package_results.sh#L12-L48

Ok thanks, I'll look into that

@vaughnbetz
Copy link
Contributor

@sfkhalid should we merge this?

@sfkhalid
Copy link
Contributor Author

Yes, I think we can, it just changes the order of some things in the vpr stdout log file. It only shows the same QoR failures as the ones on other branches. But I think we'll have to update the branch first.

@vaughnbetz
Copy link
Contributor

@sfkhalid: this failed with error code 127 in vtr_reg_nightly; so did another recent test. I merged the change to bring back compressed result files and relaunched CI (plus I merged a few other PRs over the last day or two). Sarah can you take a look into this and see what's failing? Is it a build script failure due to the bringing back compressed files? Something else? It looks like there may be a hanging / running out of time / other issue with vtr_reg_nightly so it's important we figure it out quickly and get it fixed.

@vaughnbetz
Copy link
Contributor

Seyed's PR has a similar failure, but nothing passed before it just seemed to hang. Maybe there is an issue with kokoro and it is hung? Restarting CI to see if the issue is resolved.

@sfkhalid
Copy link
Contributor Author

@vaughnbetz Nightly has passed on this pull request and on my more recent PR #1741.

@sfkhalid
Copy link
Contributor Author

When I ran nightly locally on this branch there were 4 QoR failures in titan_quick_qor. I updated the local branch to make sure it was the same as the remote, so I am not sure where the differences are coming from.

QoR_failures

@vaughnbetz
Copy link
Contributor

OK, glad this was a transient error! The QoR failures are all really the same failure: placement stretched the critical path on one design by ~50%, which is more than the allowed 40%. WNS matches the critical path delay (largely reporting the same thing) and routing values follow the placement values as the router can't fix a bad placement. So the local failure is probably just a very unlucky compile/seed noise.

@vaughnbetz vaughnbetz merged commit d06bc71 into master May 28, 2021
@vaughnbetz vaughnbetz deleted the vtr_log_changes branch May 28, 2021 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
libvtrutil VPR VPR FPGA Placement & Routing Tool
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants