Parallel router: baseline #2337

duck2 · 2023-06-12T07:06:44Z

Description

Add a spatial partitioning-based parallel router. This version is very simple, it builds a PartitionTree which recursively partitions the netlist into groups (PartitionTreeNodes) of non-overlapping nets, then uses a thread pool to route the groups.

Related Issue

Motivation and Context

More CPUs, runtime go down

How Has This Been Tested?

Ran vtr_reg_strong and titan_quick_qor with --router_algorithm parallel.

Types of changes

Bug fix (change which fixes an issue)
New feature (change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

My change requires a change to the documentation
I have updated the documentation accordingly
I have added tests to cover my changes
All new and existing tests passed

duck2 · 2023-06-22T05:12:44Z

Now this PR contains a number of cleanups and API changes:

RRNodeId-ified many files
Removed dead code left from breadth first router
Moved common route_timing.cpp functions into the header (they used to be static)
ConnectionRouter fns now take a can_grow_bb parameter and return an additional value retry_with_full_bb
- Reason: Parallel router threads cannot simply grow the bounding box and retry routing (data race), they need a flag bubbled up from the connection router to try doing that on the next iteration
timing_driven_route_net and callees return NetResultFlags instead of bool

Should format, squash and rebase. I'll run vtr_reg_strong with --router_algorithm parallel to check for bugs.

Oops: there's no TBB in the runners

duck2 · 2023-06-22T10:33:46Z

I think the tatum nondeterminism still exists with VPR_USE_TBB=1 and TATUM_USE_TBB=0.

duck2 · 2023-06-24T17:42:53Z

I think the tatum nondeterminism was not due to a data race (spent a lot of time on it with Intel Inspector) but the placer behavior depending on the ordering of modified_pins.

Parallel execution in SetupSlackCrit::update_slacks_and_criticalities gives a different ordering each time. Now the modified pins are in a std::set which is a sorted container. This seems to fix it at the first glance? Let's see if it also works with TATUM_USE_TBB=1

duck2 · 2023-06-25T04:20:31Z

a lot of failures due to memory and runtime increase?

duck2 · 2023-07-07T10:31:37Z

Turns out NetPinTimingInvalidator doesn't do anything when incremental STA is not enabled, but it does keep a cache to avoid doing nothing several times. It's a lot of trouble to keep that cache in a multithreaded and generic way and keep the performance. Added a NoopInvalidator for the general case of non-incremental timing analysis, this should help with the place time failures?

duck2 · 2023-07-08T14:00:30Z

I would add script_params_add=--router_algorithm parallel to some tests in vtr_reg_strong, but this PR already touched too many files. Better to update golden results for the problematic cases (place_time 0.88 in titan_quick_qor but fails for some FIR circuits in nightly1_odin?), upload QoR results and then ask for reviews.

utils/route_diag/src/main.cpp

duck2 · 2023-07-10T18:41:02Z

The strong_odin failure doesn't make a lot of sense: the input cmdline has --route_chan_width 26 but the test fails because min_chan_width was 18 in the golden results?

vaughnbetz · 2023-07-10T19:15:06Z

Maybe it's a two phase test where it runs the router in find minimum channel width mode, and then re-runs at a channel width of 26?

duck2 · 2023-07-10T21:06:24Z

Indeed it was a bug I accidentally introduced while rebasing.

duck2 · 2023-07-11T13:48:56Z

I can't reproduce the place time failures on wintermute. Probably better to disable tbb for CI and figure out another way to integrate tbb tests.

duck2 · 2023-07-24T16:07:02Z

libs/EXTERNAL/libtatum/libtatum/tatum/graph_walkers/SerialIncrWalker.hpp

+         * may be called concurrently */
+#ifdef TATUM_USE_TBB
+        tbb::concurrent_vector<EdgeId> invalidated_edges_;
+        tatum::util::linear_map<EdgeId, size_t> edge_invalidated_;


this can be a char (to save memory) (explain in the comments why that's thread safe)

vaughnbetz

Looks good. Biggest feedback: comment functions (in header, or at static declaration) as much as you can.
I agree making a unit test for the partition tree is good, but that can wait for another PR.

vpr/src/route/connection_based_routing.h

vpr/src/route/connection_router.h

vpr/src/route/route_timing.h

vpr/src/route/router_lookahead_extended_map.cpp

vpr/src/timing/slack_evaluation.cpp

vaughnbetz · 2023-07-24T17:29:21Z

Please summarize and link to your QoR data. QoR results master vs. this branch with parallel off, and again with parallel on.

vpr/src/route/connection_router.cpp

duck2 · 2023-07-31T11:54:24Z

Looks like there are a bunch of build failures in the test CI too

duck2 · 2023-07-31T16:48:15Z

E: Failed to fetch http://security.ubuntu.com/ubuntu/pool/main/o/openjdk-lts/openjdk-11-jre-headless_11.0.20%2b8-1ubuntu1%7e22.04_amd64.deb 404 Not Found [IP: 185.125.190.36 80]

🤦

This commit has a bunch of cleanups piggybacked onto the baseline parallel router: * Use RRNodeId where possible * Remove dead code and docs left from breadth-first router * Fix nondeterminism in parallel tatum * Fix performance penalty from NetPinTimingInvalidator

vaughnbetz · 2023-07-31T19:15:29Z

Seems like this is working (CI already fixed by you @duck2)?

duck2 · 2023-08-01T01:26:12Z

Yes, the ducktape seems to make it work.

Yosys didn't get built in Test / R: Basic with NO_GRAPHICS (push) but it worked in Test / R: Basic with NO_GRAPHICS (pull_request)?? retrying that one (edit: worked after a retry)

vaughnbetz · 2023-08-01T15:36:37Z

Thanks @duck2 .

duck2 force-pushed the par_router branch from 63fab3e to a0b5dd2 Compare June 13, 2023 02:13

duck2 force-pushed the par_router branch 2 times, most recently from 574bcbf to 87b2b50 Compare June 22, 2023 05:19

github-actions bot added VPR VPR FPGA Placement & Routing Tool libvtrutil external_libs infra Project Infrastructure labels Jun 22, 2023

duck2 force-pushed the par_router branch from cca6f6d to 0e5a4a8 Compare June 22, 2023 07:28

duck2 force-pushed the par_router branch from 701f839 to f8540a3 Compare June 24, 2023 17:37

duck2 force-pushed the par_router branch from 1cbe10f to 7920edf Compare July 8, 2023 13:52

duck2 commented Jul 8, 2023

View reviewed changes

utils/route_diag/src/main.cpp Outdated Show resolved Hide resolved

duck2 marked this pull request as ready for review July 10, 2023 16:00

duck2 requested review from amin1377 and vaughnbetz July 10, 2023 16:00

duck2 changed the title ~~[WIP] Parallel router: baseline~~ Parallel router: baseline Jul 10, 2023

duck2 force-pushed the par_router branch 2 times, most recently from ca0d135 to d151a28 Compare July 11, 2023 13:53

github-actions bot removed the infra Project Infrastructure label Jul 11, 2023

duck2 force-pushed the par_router branch 2 times, most recently from f662632 to ae98395 Compare July 11, 2023 17:31

duck2 force-pushed the par_router branch from ae98395 to a8a1c50 Compare July 21, 2023 17:16

duck2 commented Jul 24, 2023

View reviewed changes

vaughnbetz reviewed Jul 24, 2023

View reviewed changes

duck2 commented Jul 29, 2023

View reviewed changes

vpr/src/route/connection_router.cpp Outdated Show resolved Hide resolved

duck2 force-pushed the par_router branch from b58cf3a to 66e92f3 Compare July 31, 2023 11:40

github-actions bot added the ABC ABC Logic Optimization & Technology Mapping Tool label Jul 31, 2023

duck2 force-pushed the par_router branch from dc71299 to 33e9014 Compare July 31, 2023 16:52

duck2 merged commit f2783e2 into master Aug 1, 2023

duck2 deleted the par_router branch August 1, 2023 02:35

Parallel router: baseline #2337

Parallel router: baseline #2337

Uh oh!

Conversation

duck2 commented Jun 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Types of changes

Checklist:

Uh oh!

duck2 commented Jun 22, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

duck2 commented Jun 22, 2023

Uh oh!

duck2 commented Jun 24, 2023

Uh oh!

duck2 commented Jun 25, 2023

Uh oh!

duck2 commented Jul 7, 2023

Uh oh!

duck2 commented Jul 8, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

duck2 commented Jul 10, 2023

Uh oh!

vaughnbetz commented Jul 10, 2023

Uh oh!

duck2 commented Jul 10, 2023

Uh oh!

duck2 commented Jul 11, 2023

Uh oh!

duck2 Jul 24, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vaughnbetz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vaughnbetz commented Jul 24, 2023

Uh oh!

Uh oh!

duck2 commented Jul 31, 2023

Uh oh!

duck2 commented Jul 31, 2023

Uh oh!

vaughnbetz commented Jul 31, 2023

Uh oh!

duck2 commented Aug 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vaughnbetz commented Aug 1, 2023

Uh oh!

Uh oh!

duck2 commented Jun 12, 2023 •

edited

Loading

duck2 commented Jun 22, 2023 •

edited

Loading

duck2 commented Jul 8, 2023 •

edited

Loading

duck2 Jul 24, 2023 •

edited

Loading

duck2 commented Aug 1, 2023 •

edited

Loading