Merge branch 'master' into improve-nightly-test

Author: AlexandreSinger

README.developers.md

@@ -301,17 +301,29 @@ For the very large runs, you can submit your runs on a large cluster. A template
 a Slurm-managed cluster can be found under vtr_flow/tasks/slurm/
 
 ## Continuous integration (CI)
+
+### Automatic (Github runner) CI tests
+
 For the following tests, you can use remote servers instead of running them locally. Once the changes are pushed into the 
 remote repository, or a PR is created, the [Test Workflow](https://github.com/verilog-to-routing/vtr-verilog-to-routing/blob/master/.github/workflows/test.yml)
 will be triggered. Many tests are included in the workflow, including:
-* [vtr_reg_nightly_test1-N](#vtr_reg_nightly_test1-N)
+* [vtr_reg_nightly_test1-N](#vtr_reg_nightly_test1-n)
 * [vtr_reg_strong](#vtr_reg_strong)
 * [vtr_reg_basic](#vtr_reg_basic)
 * odin_reg_strong
 * parmys_reg_basic
 
 instructions on how to gather QoR results of CI runs can be found [here](#example-extracting-qor-data-from-ci-runs).
 
+### Manual Nightly Tests
+
+You can use remote servers to run the [vtr_reg_nightly_test1-7](#vtr_reg_nightly_test1-n) tests. These tests are triggered manually by going to the GitHub Actions menu, selecting the NightlyTestManual workflow and selecting run workflow on the branch you want to test. Once you do that, the [Nightly Test Manual Workflow](https://github.com/verilog-to-routing/vtr-verilog-to-routing/blob/master/.github/workflows/nightly_test_manual.yml) will be triggered. This run will take approximately 15 hours to complete and will cancel all other workflow runs for the same branch.
+
+<img src="https://raw.githubusercontent.com/verilog-to-routing/vtr-verilog-to-routing/master/doc/src/dev/run_ci_manual/select_actions.png" alt="Select GitHub Actions menu" width="60%"/>
+<br/>
+<img src="https://raw.githubusercontent.com/verilog-to-routing/vtr-verilog-to-routing/master/doc/src/dev/run_ci_manual/select_workflow.png" alt="Select the NightlyTestManual workflow" width="30%"/>
+<img src="https://raw.githubusercontent.com/verilog-to-routing/vtr-verilog-to-routing/master/doc/src/dev/run_ci_manual/run_workflow.png" alt="Run the Workflow" width="30%"/>
+
 #### Re-run CI Tests
 In the case that you want to re-run the CI tests, due to certain issues such as infrastructure failure,
 go to the "Action" tab and find your workflow under Test Workflow.

doc/_doxygen/vpr.dox

@@ -6,7 +6,7 @@ EXTRACT_ALL            = YES
 EXTRACT_PRIVATE        = YES
 EXTRACT_STATIC         = YES
 WARN_IF_UNDOCUMENTED   = NO
-INPUT                  = ../../vpr
+INPUT                  = ../../vpr ../../libs/libarchfpga/
 RECURSIVE              = YES
 GENERATE_HTML          = NO
 GENERATE_LATEX         = NO

doc/src/dev/run_ci_manual/run_workflow.png

@@ -89,6 +89,8 @@ VPR runs all stages of (pack, place, route, and analysis) if none of :option:`--
     as such, the :option:`--pack` and :option:`--place` options should not be set when this option is set.
     This flow requires that the device has a fixed size and some of the primitive blocks are fixed somewhere on the device grid.
 
+    .. seealso:: See :ref:`analytical_placement_options` for the options for this flow.
+
     .. seealso:: See :ref:`Fixed FPGA Grid Layout <fixed_arch_grid_layout>` and :option:`--device` for how to fix the device size.
 
     .. seealso:: See :ref:`VPR Placement Constraints <placement_constraints>` for how to fix primitive blocks in a design to the device grid.
@@ -1163,6 +1165,40 @@ The following options are only used when FPGA device and netlist contain a NoC r
 
     **Default:** ``vpr_noc_placement_output.txt``
 
+
+.. _analytical_placement_options:
+
+Analytical Placement Options
+^^^^^^^^^^^^^^^
+Instead of Packing atoms into clusters and placing the clusters into valid tile
+sites on the FPGA, Analytical Placement uses analytical techniques to place atoms
+on the FPGA device by relaxing the constraints on where they can be placed. This
+atom-level placement is then legalized into a clustered placement and passed into
+the router in VPR.
+
+Analytical Placement is generally split into three stages:
+
+* Global Placement: Uses analytical techniques to place atoms on the FPGA grid.
+
+* Full Legalization: Legalizes a flat (atom) placement into legal clusters placed on the FPGA grid.
+
+* Detailed Placement: While keeping the clusters legal, performs optimizations on the clustered placement.
+
+.. warning::
+
+    Analytical Placement is experimental and under active development.
+
+.. option:: --ap_full_legalizer {naive | appack}
+
+    Controls which Full Legalizer to use in the AP Flow.
+
+    * ``naive`` Use a Naive Full Legalizer which will try to create clusters exactly where their atoms are placed.
+
+    * ``appack`` Use APPack, which takes the Packer in VPR and uses the flat atom placement to create better clusters.
+
+    **Default:** ``appack``
+
+
 .. _router_options:
 
 Router Options
@@ -1179,7 +1215,7 @@ VPR uses a negotiated congestion algorithm (based on Pathfinder) to perform rout
     This means that during the routing stage, all nets, both intra- and inter-cluster, are routed directly from one primitive pin to another primitive pin.
     This increases routing time but can improve routing quality by re-arranging LUT inputs and exposing additional optimization opportunities in architectures with local intra-cluster routing that is not a full crossbar.
 
-    **Default:** ``OFF`
+    **Default:** ``off``
 
 .. option:: --max_router_iterations <int>
 

doc/src/dev/run_ci_manual/select_actions.png

@@ -19,8 +19,12 @@ struct t_grid_tile {
     const t_metadata_dict* meta = nullptr;
 };
 
-///@brief DeviceGrid represents the FPGA fabric. It is used to get information about different layers and tiles.
-// TODO: All of the function that use helper functions of this class should pass the layer_num to the functions, and the default value of layer_num should be deleted eventually.
+
+//TODO: All of the functions that use helper functions of this class should pass the layer_num to the functions, and the default value of layer_num should be deleted eventually.
+/**
+ * @class DeviceGrid
+ * @brief Represents the FPGA fabric. It is used to get information about different layers and tiles.
+ */
 class DeviceGrid {
   public:
     DeviceGrid() = default;

doc/src/dev/run_ci_manual/select_workflow.png

@@ -6,7 +6,7 @@ numpy
 scipy
 # Python linter and formatter
 click==8.0.2 # Our version of black needs an older version of click (https://stackoverflow.com/questions/71673404/importerror-cannot-import-name-unicodefun-from-click)
-black==21.4b0
+black==24.3.0
 pylint==2.7.4
 
 # Surelog

doc/src/vpr/command_line_usage.rst

@@ -68,8 +68,7 @@ void run_analytical_placement_flow(t_vpr_setup& vpr_setup) {
     const UserPlaceConstraints& constraints = g_vpr_ctx.floorplanning().constraints;
 
     // Run the prepacker
-    Prepacker prepacker;
-    prepacker.init(atom_nlist, device_ctx.logical_block_types);
+    const Prepacker prepacker(atom_nlist, device_ctx.logical_block_types);
 
     // Create the ap netlist from the atom netlist using the result from the
     // prepacker.
@@ -80,7 +79,8 @@ void run_analytical_placement_flow(t_vpr_setup& vpr_setup) {
 
     // Run the Global Placer
     std::unique_ptr<GlobalPlacer> global_placer = make_global_placer(e_global_placer::SimPL,
-                                                                     ap_netlist);
+                                                                     ap_netlist,
+                                                                     prepacker);
     PartialPlacement p_placement = global_placer->place();
 
     // Verify that the partial placement is valid before running the full
@@ -93,17 +93,15 @@ void run_analytical_placement_flow(t_vpr_setup& vpr_setup) {
                                   device_ctx.grid.get_num_layers()));
 
     // Run the Full Legalizer.
-    FullLegalizer full_legalizer(ap_netlist,
-                                 vpr_setup,
-                                 device_ctx.grid,
-                                 device_ctx.arch,
-                                 atom_nlist,
-                                 prepacker,
-                                 device_ctx.logical_block_types,
-                                 vpr_setup.PackerRRGraph,
-                                 device_ctx.arch->models,
-                                 device_ctx.arch->model_library,
-                                 vpr_setup.PackerOpts);
-    full_legalizer.legalize(p_placement);
+    const t_ap_opts& ap_opts = vpr_setup.APOpts;
+    std::unique_ptr<FullLegalizer> full_legalizer = make_full_legalizer(ap_opts.full_legalizer_type,
+                                                                        ap_netlist,
+                                                                        atom_nlist,
+                                                                        prepacker,
+                                                                        vpr_setup,
+                                                                        *device_ctx.arch,
+                                                                        device_ctx.grid,
+                                                                        device_ctx.logical_block_types);
+    full_legalizer->legalize(p_placement);
 }
 

libs/libarchfpga/src/device_grid.h

@@ -0,0 +1,20 @@
+/**
+ * @file
+ * @author  Alex Singer
+ * @date    February 2025
+ * @brief   Enumerations used by the Analytical Placement Flow.
+ */
+
+#pragma once
+
+/**
+ * @brief The type of a Full Legalizer.
+ *
+ * The Analytical Placement flow may implement different Full Legalizers. This
+ * enum can select between these different Full Legalizers.
+ */
+enum class e_ap_full_legalizer {
+    Naive,  ///< The Naive Full Legalizer, which clusters atoms placed in the same tile and tries to place them in that tile according to the flat placement.
+    APPack  ///< The APPack Full Legalizer, which uses the flat placement to improve the Packer and Placer.
+};
+

requirements.txt

@@ -9,13 +9,13 @@
 #include <string>
 #include "netlist_fwd.h"
 #include "netlist_utils.h"
-#include "vpr_types.h"
+#include "prepack.h"
 #include "vtr_assert.h"
 
 /*
  * Blocks
  */
-const t_pack_molecule* APNetlist::block_molecule(const APBlockId id) const {
+PackMoleculeId APNetlist::block_molecule(const APBlockId id) const {
     VTR_ASSERT_SAFE(valid_block_id(id));
 
     return block_molecules_[id];
@@ -37,19 +37,19 @@ const APFixedBlockLoc& APNetlist::block_loc(const APBlockId id) const {
 /*
  * Mutators
  */
-APBlockId APNetlist::create_block(const std::string& name, const t_pack_molecule* mol) {
+APBlockId APNetlist::create_block(const std::string& name, PackMoleculeId molecule_id) {
     APBlockId blk_id = Netlist::create_block(name);
 
     // Initialize the data
-    block_molecules_.insert(blk_id, mol);
+    block_molecules_.insert(blk_id, molecule_id);
     block_mobilities_.insert(blk_id, APBlockMobility::MOVEABLE);
     block_locs_.insert(blk_id, APFixedBlockLoc());
 
     // Check post-conditions: size
     VTR_ASSERT(validate_block_sizes());
 
     // Check post-conditions: values
-    VTR_ASSERT(block_molecule(blk_id) == mol);
+    VTR_ASSERT(block_molecule(blk_id) == molecule_id);
     VTR_ASSERT(block_mobility(blk_id) == APBlockMobility::MOVEABLE);
 
     return blk_id;

vpr/src/analytical_place/analytical_placement_flow.cpp

@@ -23,9 +23,7 @@
 #include <string>
 #include "netlist.h"
 #include "ap_netlist_fwd.h"
-
-// Forward declarations
-class t_pack_molecule;
+#include "prepack.h"
 
 /**
  * @brief Struct to store fixed block location information
@@ -83,7 +81,7 @@ class APNetlist : public Netlist<APBlockId, APPortId, APPinId, APNetId> {
      */
 
     /// @brief Returns the molecule that this block represents.
-    const t_pack_molecule* block_molecule(const APBlockId id) const;
+    PackMoleculeId block_molecule(const APBlockId id) const;
 
     /// @brief Returns the mobility of this block.
     APBlockMobility block_mobility(const APBlockId id) const;
@@ -104,7 +102,7 @@ class APNetlist : public Netlist<APBlockId, APPortId, APPinId, APNetId> {
      *  @param name The unique name of the block
      *  @param mol  The molecule the block represents
      */
-    APBlockId create_block(const std::string& name, const t_pack_molecule* mol);
+    APBlockId create_block(const std::string& name, PackMoleculeId molecule_id);
 
     /**
      * @brief Fixes a block at the given location
@@ -182,7 +180,7 @@ class APNetlist : public Netlist<APBlockId, APPortId, APPinId, APNetId> {
 
 private: // Private Data
     /// @brief Molecule of each block
-    vtr::vector_map<APBlockId, const t_pack_molecule*> block_molecules_;
+    vtr::vector_map<APBlockId, PackMoleculeId> block_molecules_;
     /// @brief Type of each block
     vtr::vector_map<APBlockId, APBlockMobility> block_mobilities_;
     /// @brief Location of each block (if fixed).