Skip to content

Generating new golden results (titan_quick_qor, vtr_reg_coffe) + implicit ram connections checking #1689

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Mar 19, 2021
Merged

Generating new golden results (titan_quick_qor, vtr_reg_coffe) + implicit ram connections checking #1689

merged 2 commits into from
Mar 19, 2021

Conversation

sdamghan
Copy link
Member

@sdamghan sdamghan commented Mar 15, 2021 •
edited
Loading

Description

Related Issue/PR

PR #1658

Motivation and Context

How Has This Been Tested?

  1. make test (ODIN-II)
  2. Basic Regression Tests
  3. Strong Regression Test
  4. Basic Valgrind Memory Tests (also, Valgrind memory tests for all other regression suits)
  5. Sanitized Basic Regression Tests
  6. Odin-II Micro Tests

Types of changes

  • Bug fix (change which fixes an issue)
  • New feature (change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My change requires a change to the documentation
  • I have updated the documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed

@github-actions github-actions bot added the Odin Odin II Logic Synthesis Tool: Unsorted item label Mar 15, 2021
@sdamghan
Copy link
Member Author

@vaughnbetz
This PR includes the new golden results for titan_quick_qor and vtr_reg_coffe, would you please have a look?
It worth noting that I received failure for vpr in the following case:

titan_quick_qor:        stratixiv_arch.timing.xml/LU230_stratixiv_arch_timing.blif/common        		Error: Executable vpr failed
	full command:  /usr/bin/env time -v /home/casauser/Desktop/Most-Updated-VTR/Main/vpr/vpr stratixiv_arch.timing.xml LU230_stratixiv_arch_timing --circuit_file LU230_stratixiv_arch_timing.pre-vpr.blif --route_chan_width 300 --max_router_iterations 400 --router_lookahead map --initial_pres_fac 1.0 --router_profiler_astar_fac 1.5 --sdc_file /home/casauser/Desktop/Most-Updated-VTR/Main/vtr_flow/benchmarks/titan_blif/LU230_stratixiv_arch_timing.sdc
	returncode  :  137
	log file    : **please see the attachment**

failed: Executable vpr failed (took 3470.59 seconds)

vpr.out.txt

@vaughnbetz
Copy link
Contributor

It looks like that run was killed with a signal (-9). That doesn't look right -- it seems Ctrl+C was hit or the task was otherwise interrupted somehow? Can you try re-running that regtest as we don't want to update the golden results with a failure that looks odd / like a machine issue.
For CPU times, is your machine producing cpu times that are close to what the CI machines are producing? We don't normalize the cpu times, so it's good to check a few circuits to make sure the cpu time of your system is in the right range (should happen naturally if this is a relatively recent desktop or server).

@sdamghan sdamghan force-pushed the implicit_ram_connections branch from 9db6ef8 to ab9ad23 Compare March 19, 2021 04:24
@sdamghan
Copy link
Member Author

sdamghan commented Mar 19, 2021 •
edited
Loading

@vaughnbetz
I have just pushed another update. This time all benchmarks went well without any error. I have also attached a comparison table for VTR elapsed time. Generally, titan quick qor benchmarks cpu time decreased almost 30% however this amount for vtr reg coffe benchmarks is in average more than 96% decrement which is absolutely considerable. I have also noticed that in previous golden results for vtr reg coffe, sanitizer was active, so at least I know in Odin having sanitizer ON makes a huge difference in terms of RAM usage. Please let me know your thought.

NewGoldenResults.zip

sdamghan
sdamghan commented Mar 19, 2021
View reviewed changes
vtr_flow/tasks/regression_tests/vtr_reg_nightly/vtr_reg_coffe/config/golden_results.txt
@@ -1,4 +1,4 @@
arch circuit script_params vtr_flow_elapsed_time error odin_synth_time max_odin_mem abc_depth abc_synth_time abc_cec_time abc_sec_time max_abc_mem ace_time max_ace_mem num_clb num_io num_memories num_mult vpr_status vpr_revision vpr_build_info vpr_compiler vpr_compiled hostname rundir max_vpr_mem num_primary_inputs num_primary_outputs num_pre_packed_nets num_pre_packed_blocks num_netlist_clocks num_post_packed_nets num_post_packed_blocks device_width device_height device_grid_tiles device_limiting_resources device_name pack_time placed_wirelength_est place_time place_quench_time placed_CPD_est placed_setup_TNS_est placed_setup_WNS_est placed_geomean_nonvirtual_intradomain_critical_path_delay_est place_delay_matrix_lookup_time place_quench_timing_analysis_time place_quench_sta_time place_total_timing_analysis_time place_total_sta_time min_chan_width routed_wirelength min_chan_width_route_success_iteration logic_block_area_total logic_block_area_used min_chan_width_routing_area_total min_chan_width_routing_area_per_tile min_chan_width_route_time min_chan_width_total_timing_analysis_time min_chan_width_total_sta_time crit_path_routed_wirelength crit_path_route_success_iteration crit_path_total_nets_routed crit_path_total_connections_routed crit_path_total_heap_pushes crit_path_total_heap_pops critical_path_delay geomean_nonvirtual_intradomain_critical_path_delay setup_TNS setup_WNS hold_TNS hold_WNS crit_path_routing_area_total crit_path_routing_area_per_tile router_lookahead_computation_time crit_path_route_time crit_path_total_timing_analysis_time crit_path_total_sta_time
agilex_arch.xml spree.v common 11829.46 21.69 469644 15 1.09 -1 -1 39156 -1 -1 52 45 6 -1 success v8.0.0-3054-g8e6da26b9-dirty release IPO VTR_ASSERT_LEVEL=2 sanitizers GNU 7.5.0 on Linux-4.15.0-124-generic x86_64 2021-01-08T20:29:51 jupiter0 /export/aman/vtr_aman/vtr-verilog-to-routing/vtr_flow/tasks 5297340 45 32 1272 1229 1 802 136 178 82 14596 -1 mylayout 79.00 8887 36.59 0.24 7.63003 -5828.16 -7.63003 7.63003 3851.46 0.0870093 0.0791016 14.1198 12.7913 74 19778 40 6.8487e+08 2.93092e+06 1.00893e+08 6912.39 6075.37 58.2347 54.3008 16908 14 4153 11286 3182091 652840 10.1949 10.1949 -7867.19 -10.1949 0 0 1.24410e+08 8523.59 352.60 14.15 3.49607 3.31772
Copy link
Member Author

@sdamghan sdamghan Mar 19, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vaughnbetz , I meant this section:
release IPO VTR_ASSERT_LEVEL=2 sanitizers

@vaughnbetz
Copy link
Contributor

Thanks Seyed. The vtr_reg_coffe tests are turned off for now since their golden results were quite messed up. Perhaps having the sanitizers on is why, so updating their qor results sounds good.
It looks like your CPU is faster than the old cpu results. The acceptable CPU ranges are quite broad (0.1x - 10x) to accommodate machine differences though, so it's OK. Let's merge it.

@vaughnbetz vaughnbetz merged commit da48e84 into verilog-to-routing:master Mar 19, 2021
@sdamghan sdamghan deleted the implicit_ram_connections branch November 19, 2021 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
Odin Odin II Logic Synthesis Tool: Unsorted item
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@sdamghan @vaughnbetz