Move rr_graph_reader & rr_graph_writer to librrgraph. #2101

oscarcheng105 · 2022-07-21T00:12:07Z

Description

In this PR rr_graph_reader and rr_graph_writer from vpr/src/route are moved to librrgraph. We also avoid dependencies from vpr and try moving the vpr dependencies to the according libraries.

MOVED/ADDED FILES

The following Files/DataStructures/Functions from vpr are moved/added to librrgraph to avoid vpr dependencies:

rr_graph_reader.h/.cpp
rr_graph_writer.h/.cpp
rr_graph_uxsdcxx_serializer.h
uxsdcxx generator
rr_graph.xsd
rr_metadata.h/.cpp
rr_rc_data.h/.cpp
check_rr_graph.h/.cpp
rr_graph_utils.h/.cpp
rr_graph_storage_utils.h
describe_rr_node.h/.cpp
get_parallel_segs.h/.cpp
alloc_and_load_rr_indexed_data.h/.cpp

The following Files/Data-structures/Functions from vpr are moved/added to libarchfpga to avoid vpr dependencies:

device_grid.h/.cpp
histogram.h/.cpp
graph_type.h
chan_width.h
route_type.h
cost_indices.h
base_cost_type.h
unified_to_parallel_seg_index.h

The following Files/DataStructures/Functions from vpr are moved to libvtrutil to avoid vpr dependencies:

vpr_error.h/.cpp

MAJOR CODE CHANGES

The parameters of the following functions are modified. Data Structures in DeviceContext are now passed in directly as parameters to avoid dependency on DeviceContext (VprContext):

load_rr_file
write_rr_graph
rr_node_metadata
rr_edge_metadata
add_rr_node_metadata
add_rr_edge_metadata
find_create_rr_rc_data
check_rr_graph
check_rr_node
describe_rr_node
alloc_and_load_rr_indexed_data

*Parameters of subroutines local to the module are also changed accordingly regarding the parameter change from above.

The following private variables are added to data structures in rr_graph_uxsdcxx_serializer.h to accommodate the parameter changes:

std::vector<t_rr_rc_data>* rr_rc_data added to RrGraphSerializer
const int virtual_clock_network_root_idx added to RrGraphSerializer
const bool echo_enabled added to RrGraphSerializer
const char* echo_file_name added to RrGraphSerializer
MetadataStorage<int>* rr_node_metadata_ added to MetadataBind
MetadataStorage<std::tuple<int, int, short>>* rr_edge_metadata_ added to MetadataBind

Declaration of the following Data-structures/Functions are removed from vpr, re-declared in the according library, and made as dependency again to files in vpr that reference them:

DeviceGrid removed from vpr/src/route, moved to libarchfpga
histogram removed from vpr/src/util, moved to libarchfpga
t_unified_to_parallel_seg_index removed from rr_graph2.h, moved to libarchfpga
get_parallel_segs removed from rr_graph2.h, moved to libarchfpga
e_route_type removed from vpr_types.h, moved to libarchfpga
e_base_cost_type removed from vpr_types.h, moved to libarchfpga
e_cost_indices removed from vpr_types.h, moved to libarchfpga
t_chan_width removed from vpr_types.h, moved to libarchfpga
describe_rr_node removed from rr_graph.h, moved to librrgraph
alloc_and_load_rr_indexed_data removed from rr_graph_indexed_data.h, moved to librrgraph
e_graph_type (aka t_graph_type) removed from rr_graph.h, moved to librrgraph
rr_graph_util (vpr/src/route) merged to librrgraph

OTHER CODE CHANGE

VTR_ENABLE_CAPNPROTO added to librrgraph/CMakeLists.txt
generate_rr_graph_serializers moved from vpr/CMakeLists.txt to `librrgraph/CMakeLists.txt

UPDATES

Files in Librrgraph are categorized into three folders:

io for files regarding read/write purposes.
utils for utility functions.
base for rest of the files in librrgraph.

The following data structures are combined into rr_graph_type.h (librrgraph/base) for builder/general usage:

e_graph_type in graph_type.h
e_route_type in route_type.h
t_chan_width in chan_width.h
t_unified_to_parallel_seg_index.h in unified_to_parallel_seg_index.h

The following data structures are combined into rr_graph_cost.h (librrgraph/base) for router usage:

e_base_cost_type in base_cost_type.h
e_cost_indices in cost_indices.h

Related Issue

Motivation and Context

As devised by the mentor #2000

How Has This Been Tested?

The changes passed the two regression tests vtr_reg_basic and vtr_reg_strong.

Types of changes

Bug fix (change which fixes an issue)
New feature (change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

My change requires a change to the documentation
I have updated the documentation accordingly
I have added tests to cover my changes
All new and existing tests passed

…graph(), and check_rr_node().

…), write_rr_graph(), and find_create_rr_rc_data().

tangxifan · 2022-07-21T00:22:05Z

Notes

doc build is already failling on the current master. This PR has no touch on doc.
There are 6.5k lines of code changes but most of them are moving the files around. Note that all the files under vpr/src/route/gen are auto generated which contain 6k lines of codes. Actual code changes are small

tangxifan · 2022-07-21T00:27:25Z

Todo list on this PR:

Need confirmation from @vaughnbetz and others (who do related work around rr_graph) on the code changes **especially on the migration of data structures from vpr/src to libs
Should agree with another ongoing PRs FPGA interchange: add RR graph generation #1999 Add Intra-Cluster Pins & Connections to RR Graph #2066 on these changes. Resolve any conflicts we may have.
PR Moving the rrgraph_read/ rrgraph_write to libs/rrgraph #2000 Can be closed as its code changes have already been covered in this PR.

tangxifan · 2022-07-21T00:27:37Z

@oscarcheng105 Please fix the code format errors: https://github.com/verilog-to-routing/vtr-verilog-to-routing/runs/7440223859?check_suite_focus=true

tangxifan · 2022-07-21T17:19:49Z

Memo @oscarcheng105

Should also address Remove some out of date TO-DO and comments in rr_graph.cpp #1995 in a later PR.

vaughnbetz · 2022-07-21T17:44:18Z

Needs a make format.

libs/librrgraph/src/base/check_rr_graph.cpp

libs/librrgraph/src/base/check_rr_graph.h

libs/librrgraph/src/base/rr_graph_writer.cpp

libs/librrgraph/src/base/rr_graph_uxsdcxx_serializer.h

libs/librrgraph/src/base/rr_graph_writer.cpp

libs/librrgraph/src/base/rr_graph_writer.h

tangxifan

@oscarcheng105 Just some comments to categorize source files.
Also, to group header files, we need your information about what are the client functions of these data types/structures.
There are five major clients of RRGraph:

Placer
Router
routing resource graph builder
GUI (drawer)
Timing analyzer

Suggest to follow this thread to group headers. We can go through them one by one, once you can provide details.

NOTES

The following is a list of files that references the migrated data-structures mentioned previously:

e_graph_type/t_graph_type in graph_type.h
- First Layer
  - rr_graph_uxsdcxx_serializer.h (librrgraph)
  - rr_graph_reader.h (librrgraph)
  - check_rr_graph_.h (librrgraph)
  - rr_graph.h (vpr/src/route)
- Second Layer
  - rr_graph_uxsdcxx_serializer.h -> rr_graph_writer.h (librrgraph)
  - rr_graph.h -> place_and_route.h (vpr/src/base)
  - rr_graph.h -> vpr_api.cpp (vpr/src/base)
  - rr_graph.h -> route_common.cpp (vpr/src/route)
  - rr_graph.h -> router_delay_profiling.cpp (vpr/src/route)
  - rr_graph.h -> time_place_lookup.cpp (vpr/src/place)
  - rr_graph.h -> main.cpp (utils/route_diag/src)
- Third Layer
  - place_and_route.cpp -> place.cpp (vpr/src/place)
  - place_and_route.cpp -> test_connection_router.cpp (vpr/test)
t_unified_to_parallel_seg_index in unified_to_parallel_seg_index.h
- First Layer
  - get_parallel_segs.h (librrgraph)
  - rr_graph_uxsdcxx_serializer.h (librrgraph)
  - rr_graph2.h (vpr/src/route)
- Second Layer
  - rr_graph2.h -> clock_network_builders.h (vpr/src/route)
  - rr_graph2.h -> rr_graph.cpp (vpr/src/route)
- Third Layer
  - clock_network_builders.h -> rr_graph_clock.h (vpr/src/route)
e_route_type in route_type.h
- First Layer
  - check_rr_graph.h (librrgraph)
  - vpr_types.h (vpr/src/base)
- Second Layer
  - vpr_types.h -> read_options.h (vpr/src/base)
  - vpr_types.h -> stats.cpp (vpr/src/base)
  - vpr_types.h -> check_route.cpp (vpr/src/route)
  - vpr_types.h -> draw_types.h (vpr/src/draw)
- Fourth Layer
  - draw_types.h -> draw_global -> draw.h (vpr/src/draw)
e_base_cost_type in base_cost_type.h
- First Layer
  - rr_graph_reader.h (librrgraph)
  - rr_graph_uxsdcxx_serializer.h (librrgraph)
  - vpr_types.h (vpr/src/base)
- Second Layer
  - rr_graph_uxsdcxx_serializer.h -> rr_graph_writer.h (librrgraph)
  - vpr_types.h -> read_options.h (vpr/src/base)
  - vpr_types.h -> rr_graph.cpp (vpr/src/route)
  - vpr_types.h -> rr_graph_indexed_data.cpp (vpr/src/route)
e_cost_indices in cost_indices.h
- First Layer
  - rr_graph_uxsdcxx_serializer.h (librrgraph)
  - vpr_types.h (vpr/src/base)
- Second Layer
  - vpr_types.h -> rr_graph.h (vpr/src/route)
  - vpr_types.h -> rr_graph_indexed_data.cpp (vpr/src/route)
  - vpr_types.h -> router_lookahead_map.cpp (vpr/src/route)
  - vpr_types.h -> router_lookahead.h (vpr/src/route)
  - vpr_types.h -> route_timing.h (vpr/src/route)
- Third Layer
  - rr_graph.h -> router_lookahead_extended_map.cpp (vpr/src/route)
- Fourth Layer
  - vpr_types.h -> rr_graph2.h -> clock_connection_builders.cpp (vpr/src/route)
t_chan_width in chan_width.h
- First Layer
  - check_rr_graph.h (librrgrpah)
  - rr_graph_reader.h (librrgraph)
  - rr_graph_writer.h (librrgraph)
  - rr_graph_uxsdcxx_serializer.h (librrgraph)
  - vpr_types.h (vpr/src/base)
- Second Layer
  - vpr_types.h -> place_and_route.h (vpr/src/base)
  - vpr_types.h -> vpr_api.h (vpr/src/base)
  - vpr_types.h -> vpr_context.h (vpr/srce/base)
  - vpr_types.h -> timing_place_lookup.cpp (vpr/src/place)
  - vpr_types.h -> build_switchblocks.h (vpr/src/route)
  - vpr_types.h -> cb_metrics.cpp (vpr/src/route)
  - vpr_types.h -> route_common.cpp (vpr/src/route)
  - vpr_types.h -> router_delay_profiling.h (vpr/src/route)
  - vpr_types.h -> rr_graph.h (vpr/src/route)
  - vpr_types.h -> rr_graph2.cpp (vpr/src/route)
  - vpr_types.h -> rr_graph_sbox.cpp (vpr/src/route)
- Third Layer
  - rr_graph.h -> rr_graph_clock.cpp (vpr/src/route)
- Fourth Layer
  - rr_graph.h -> main.cpp (utils/route_diag/src)

libs/libarchfpga/src/graph_type.h

libs/librrgraph/src/base/alloc_and_load_rr_indexed_data.h

libs/librrgraph/src/base/describe_rr_node.h

libs/librrgraph/src/base/rr_graph_reader.cpp

…zer, find_create_rr_rc_data.

…nd_load_rr_indexed_data.cpp to librrgraph/src/util

vaughnbetz · 2022-07-28T17:50:11Z

@acomodi: we suggest you review this in case it interacts with the interchange format; also a big restructuring so worth reviewing in any case.

vaughnbetz · 2022-07-28T17:50:53Z

To do: please add vtr and titan QoR data to sh[ow no degradations in runtime, or unexpected QoR changes. See the developer guide and check both the vtr and titan benchmarks.

tangxifan · 2022-07-28T17:55:46Z

@oscarcheng105

Templates to provide QoR test summary: #1972 (comment)

Documentation about test: https://docs.verilogtorouting.org/en/latest/README.developers/#tests

oscarcheng105 · 2022-07-29T20:50:47Z

@vaughnbetz @tangxifan
QoR tests are completed with Master and Feature Branch and quick summary is given below. QoR comparison files are also attached.
vtr_reg_qor_chain_depop test:
Flow run time 0.37% on average.
critical path delay 0% on average.
Peak memory usage -0.03% on average.
titan_quick_qor test:
flow run time 0.11% on average.
critical path delay 0% on average.
Peak memory usage 0% on average.
vtr_reg_qor_chain_depop.xlsx
titan_quick_qor_master.xlsx

tangxifan · 2022-07-29T22:02:47Z

libs/librrgraph/src/base/check_rr_graph.cpp


-static bool has_adjacent_channel(const t_rr_node& node, const DeviceGrid& grid);
+static bool has_adjacent_channel(const RRGraphView& rr_graph, const DeviceGrid& grid, const t_rr_node& node);


Just a mark here. This function need a refactoring. We should use RRNodeId rather than t_rr_node which is now an internal data of RRGraph

libs/librrgraph/src/base/check_rr_graph.cpp

tangxifan · 2022-07-29T22:13:29Z

libs/librrgraph/src/base/check_rr_graph.cpp

@@ -494,13 +497,13 @@ void check_rr_node(int inode, enum e_route_type route_type, const DeviceContext&
            //Don't worry about disconnect PINs which have no adjacent channels (i.e. on the device perimeter)
            bool check_for_out_edges = true;
            if (rr_type == IPIN || rr_type == OPIN) {
-                if (!has_adjacent_channel(rr_graph.rr_nodes()[inode], device_ctx.grid)) {
+                if (!has_adjacent_channel(rr_graph, grid, rr_graph.rr_nodes()[inode])) {


Just a mark, rr_nodes() just expose the internal node storage directly to client function, which should be avoid. I suggest to rework this function later.

vtr-verilog-to-routing/libs/librrgraph/src/base/rr_graph_view.h

Lines 442 to 445 in 7989197

/** @brief Return the node-level storage structure for queries from client functions */

inline const t_rr_graph_storage& rr_nodes() const {

return node_storage_;

}

libs/librrgraph/src/base/rr_graph_type.h

libs/librrgraph/src/io/rr_graph_writer.cpp

libs/librrgraph/src/io/rr_graph_writer.h

libs/librrgraph/src/utils/describe_rr_node.h

utils/fasm/src/fasm.cpp

tangxifan · 2022-07-29T22:36:37Z

vpr/src/draw/search_bar.cpp

@@ -138,9 +138,11 @@ void search_and_highlight(GtkWidget* /*widget*/, ezgl::application* app) {
 bool highlight_rr_nodes(int hit_node) {
    t_draw_state* draw_state = get_draw_state_vars();

+    //TODO: fixed sized char array may cause overflow.


@vaughnbetz I am not sure why there is a fixed-length message string here. But it looks dangerous to me. Just a reminder here. See if you do need to fix it.

…-to-routing into librrgraph_vpr

tangxifan

LGTM

@vaughnbetz : I have gone through another round of code view. I am satisfied with the code changes. QoR results have been uploaded as well, showing almost zero overhead on critical metric. Waiting for your decision.

vaughnbetz · 2022-08-11T14:44:22Z

Thanks! Merged.

oscarcheng105 added 9 commits July 18, 2022 10:24

Files moved to Librrgraph and Files added to Libarchfpga

e785f1c

Merge branch 'verilog-to-routing:master' into master

377186b

Fix input parameters for load_rr_graph(), write_rr_graph(), check_rr_…

93a7dc6

…graph(), and check_rr_node().

Fix input parameters of add_rr_edge_metadata(), add_rr_node_metadata(…

b42102a

…), write_rr_graph(), and find_create_rr_rc_data().

[vpr] move VTR_ENABLE_CAPNPROTO to librrgraph CMakeList.txt

5db7988

[vpr] move cmake dependencies in vpr/src/route to librrgraph/src/base/

0bb613f

Merge branch 'verilog-to-routing:master' into librrgraph_vpr

3cccda7

[vpr] clean up dead code

e428f8a

[vpr] clean up dead code 2

a30ca69

oscarcheng105 requested a review from tangxifan July 21, 2022 00:12

github-actions bot added libarchfpga Library for handling FPGA Architecture descriptions libvtrutil VPR VPR FPGA Placement & Routing Tool labels Jul 21, 2022

[vpr] format fix

7840120

tangxifan mentioned this pull request Jul 21, 2022

Moving the rrgraph_read/ rrgraph_write to libs/rrgraph #2000

Closed

7 tasks

Yitian4Debug reviewed Jul 21, 2022

View reviewed changes

tangxifan requested changes Jul 22, 2022

View reviewed changes

oscarcheng105 added 8 commits July 22, 2022 17:00

[vpr] move files to corresponding folder in librrgraph/src

3200ba9

[vpr] fix parameters of check_rr_graph, check_rr_node, RrGraphSeriali…

916c6fe

…zer, find_create_rr_rc_data.

[vpr] move histogram to libarchfpga, merge rr_graph_util, add alloc_a…

ff83a48

…nd_load_rr_indexed_data.cpp to librrgraph/src/util

Merge branch 'verilog-to-routing:master' into librrgraph_vpr

77971b0

[vpr] move describe_rr_node.cpp to librrgraph/src/utils

bfcb7ec

[vpr] add get_parallel_segs.cpp to librrgraph

c3ff424

[vpr] format fix

ca391d3

[vpr] format

70cc0b4

oscarcheng105 added 2 commits July 26, 2022 10:36

[vpr] format fix 2

9682216

[vpr] categorize files to rr_graph_type & rr_graph_cost in librrgraph

2280445

vaughnbetz requested a review from acomodi July 28, 2022 17:49

Merge branch 'verilog-to-routing:master' into librrgraph_vpr

2aa7ce6

tangxifan requested changes Jul 29, 2022

View reviewed changes

oscarcheng105 added 2 commits July 29, 2022 16:17

[vpr] code cleanup

4a4eb96

Merge branch 'librrgraph_vpr' of github.com:oscarcheng105/vtr-verilog…

783fa6b

…-to-routing into librrgraph_vpr

tangxifan approved these changes Aug 1, 2022

View reviewed changes

oscarcheng105 added 2 commits August 2, 2022 13:34

[vpr] doc update

e900c68

Merge branch 'verilog-to-routing:master' into librrgraph_vpr

1b8d060

vaughnbetz merged commit 3afe583 into verilog-to-routing:master Aug 11, 2022

oscarcheng105 mentioned this pull request Sep 2, 2022

[WIP] Librrgraph unit test #2150

Open

7 tasks


		static bool has_adjacent_channel(const t_rr_node& node, const DeviceGrid& grid);
		static bool has_adjacent_channel(const RRGraphView& rr_graph, const DeviceGrid& grid, const t_rr_node& node);

	/** @brief Return the node-level storage structure for queries from client functions */
	inline const t_rr_graph_storage& rr_nodes() const {
	return node_storage_;
	}

Move rr_graph_reader & rr_graph_writer to librrgraph. #2101

Move rr_graph_reader & rr_graph_writer to librrgraph. #2101

Uh oh!

Conversation

oscarcheng105 commented Jul 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

MOVED/ADDED FILES

MAJOR CODE CHANGES

OTHER CODE CHANGE

UPDATES

Related Issue

Motivation and Context

How Has This Been Tested?

Types of changes

Checklist:

Uh oh!

tangxifan commented Jul 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Notes

Uh oh!

tangxifan commented Jul 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tangxifan commented Jul 21, 2022

Uh oh!

tangxifan commented Jul 21, 2022

Uh oh!

vaughnbetz commented Jul 21, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tangxifan left a comment • edited by oscarcheng105 Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

NOTES

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vaughnbetz commented Jul 28, 2022

Uh oh!

vaughnbetz commented Jul 28, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tangxifan commented Jul 28, 2022

Uh oh!

oscarcheng105 commented Jul 29, 2022

Uh oh!

tangxifan Jul 29, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

tangxifan Jul 29, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tangxifan Jul 29, 2022

Choose a reason for hiding this comment

Uh oh!

tangxifan left a comment

Choose a reason for hiding this comment

Uh oh!

vaughnbetz commented Aug 11, 2022

Uh oh!

Uh oh!

oscarcheng105 commented Jul 21, 2022 •

edited

Loading

tangxifan commented Jul 21, 2022 •

edited

Loading

tangxifan commented Jul 21, 2022 •

edited

Loading

tangxifan left a comment •

edited by oscarcheng105

Loading

vaughnbetz commented Jul 28, 2022 •

edited

Loading