Merge pull request #1762 from verilog-to-routing/slurm-submission-script

Add a template submission script for SLURM-managed clusters

Author: MohamedElgammal

README.developers.md

@@ -280,6 +280,9 @@ You can also run multiple regression tests together:
 #Run both the basic and strong regression, with up to 4 tests in parallel
 $ ./run_reg_test.py vtr_reg_basic vtr_reg_strong -j4
 ```
+## Running in a large cluster using SLURM
+For the very large runs, you can submit your runs on a large cluster. A template of submission script to 
+a Slurm-managed cluster can be found under vtr_flow/tasks/slurm/
 
 ## Odin Functionality Tests
 

vtr_flow/scripts/slurm/README.md

@@ -0,0 +1,21 @@
+submission_template.sh is a template used to submit jobs using SLURM on a large cluster
+- This script is tested on computecanada cluster (Niagara) system
+
+To use this script:
+- First, generate all the shell scripts to run a task using:
+    * run_vtr_task.py -system scripts (Sec. 3.9.3 in the documentation)
+    * this will create a new run directory under the specified task (e.g. run001)
+
+- Second, edit the submission_template.sh script to match your run
+    * All lines starting with #SBATCH go to SLURM
+    * Edit all these line depending on the number of cores you want, number of parallel jobs you want,
+      the job name, your email and the job time limit
+    * Edit the relative path to your task (path_to_task) and the specific run directory where the 
+      generated shell scripts are (dir)
+
+- Third, submit these jobs to SLURM using the following command
+    * sbatch submission_template.sh
+
+You can check the state of the submitted job (or all the jobs that you submitted) using
+squeue --me
+

vtr_flow/scripts/slurm/submission_template.sh

@@ -0,0 +1,40 @@
+#! /bin/bash
+
+#======================================================================================
+#Please Make a copy of this template in the same directory and edit it to fit your run.
+#Please do not submit your copy of this template to the repo.
+#======================================================================================
+
+
+#The following section controls the parameters of your run, feel free to edit:
+#=============================================================================
+#SBATCH --nodes=1                                       # Number of nodes in our pool
+#SBATCH --ntasks=80                                     # Number of parallel tasks
+#SBATCH --ntasks-per-node=40                            # maximum number of tasks simultaneously running on a node
+#SBATCH --job-name=vtr_test                             # Job name
+#SBATCH --mail-type=FAIL,END                            # When to send an email
+#SBATCH [email protected]               # The user's email
+#SBATCH --output=parallel_%j.log                        # Determine the output log file; (%j) is the jobid
+#SBATCH --error=error_%j.log                            # Determine the error log file
+#SBATCH --time=10:00:00                                 # The job time limit in hh:mm:ss
+
+#You can also overwrite the values of some of these paramters using environment variables
+#========================================================================================
+# - SBATCH_JOB_NAME instead of --job-name
+# - SBATCH_TIMELIMIT instead of --time
+
+#Preload the required environment modules
+module load gcc/9.2.0
+module load python/3.8.5
+
+#Choose the run directory 
+path_to_task="../../tasks/regression_tests/vtr_reg_nightly/vtr_reg_qor_chain_large"
+dir="run001"
+
+
+for script in ${path_to_task}/${dir}/*/*/common*/vtr_flow.sh
+do
+        echo $script
+        $script &
+done
+wait