Merge pull request #1914 from byuccl/venv

Refactor build/dependency documentation + new markdown parser (myst-parser)

Author: vaughnbetz

.gitignore

@@ -104,6 +104,7 @@ ODIN_II/usefull_tools/**/track_completed
 #Python
 #
 *.pyc
+/.venv
 
 #
 #Vim

BUILDING.md

@@ -1,255 +1,49 @@
-# Building VTR #
+# Building VTR
 
 
-## Overview ##
+## Setting up Your Environment
 
-VTR uses [CMake](https://cmake.org) as it's build system.
 
-CMake provides a portable cross-platform build systems with many useful features.
+VTR requires several system packages.  From the top-level directory, run the following script to install the required packages on a modern Debian or Ubuntu system:
 
-## Tested Compilers ##
-VTR requires a C++-14 compliant compiler.
-It is tested against the default compilers of all Debian and Ubuntu releases within their standard support lifetime. Currently, those are the following:
+    ./install_apt_packages.sh
 
-* GCC/G++: 7, 8, 9, 10, 11
-* Clang/Clang++: 6, 7, 10
 
-Other compilers may work but are untested (your milage may vary).
+You will also need several Python packages.  You can optionally install and activate a Python virtual environment so that you do not need to modify your system Python installation:
 
-## Unix-like ##
-For unix-like systems we provide a wrapper Makefile which supports the traditional `make` and `make clean` commands, but calls CMake behind the scenes.
+    make env
+    source .venv/bin/activate
 
-### Dependencies ###
+Then to install the Python packages:
 
-For the basic tools you need:
- * Bison & Flex
- * cmake, make
- * A modern C++ compiler supporting C++14 (such as GCC >= 4.9 or clang >= 3.6)
+    pip install -r requirements.txt
 
-For the VPR GUI you need:
- * Cairo
- * FreeType
- * Xft (libXft + libX11)
- * fontconfig
- * libgtk-3-dev
+**Note:** If you chose to install the Python virtual environment, you will need to remember to activate it on any new terminal window you use, before you can run the VTR flow or regressions tests (`source .venv/bin/activate`).
 
-For the [regression testing and benchmarking](README.developers.md#running-tests) you will need:
- * Perl + List::MoreUtils
- * Python
- * time
+## Building
 
-It is also recommended you install the following development tools:
- * git
- * ctags
- * gdb
- * valgrind
- * clang-format-7
+From the top-level, run:
 
-For Docs generation you will need:
- * Doxygen
- * python-sphinx
- * python-sphinx-rtd-theme
- * python-recommonmark
+    make
 
-#### Debian & Ubuntu ####
+   which will build all the required tools.
 
-The following should be enough to get the tools, VPR GUI and tests going on a modern Debian or Ubuntu system:
+The complete VTR flow has been tested on 64-bit Linux systems.
+The flow should work in other platforms (32-bit Linux, Windows with cygwin) but this is untested.
 
-```shell
-apt-get install \
-	build-essential \
-	flex \
-	bison \
-	cmake \
-	fontconfig \
-	libcairo2-dev \
-	libfontconfig1-dev \
-	libx11-dev \
-	libxft-dev \
-	libgtk-3-dev \
-	perl \
-	liblist-moreutils-perl \
-	python \
-	time
-```
+*Full information about building VTR, including setting up required system packages and Python packages, can be found in [Optional Build Information](doc/src/vtr/optional_build_info.md) page.*
 
-For documentation generation these additional packages are required:
+Please [let us know](doc/src/contact.md) your experience with building VTR so that we can improve the experience for others.
 
-```shell
-apt-get install \
-	doxygen \
-	python-sphinx \
-	python-sphinx-rtd-theme \
-	python-recommonmark
-```
+The tools included official VTR releases have been tested for compatibility.
+If you download a different version of those tools, then those versions may not be mutually compatible with the VTR release.
 
+## Verifying Installation
 
-For development the following additional packages are useful:
+To verfiy that VTR has been installed correctly run::
 
-```shell
-apt-get install \
-	git \
-	valgrind \
-	gdb \
-	ctags
-```
+    ./vtr_flow/scripts/run_vtr_task.py basic_flow
 
-#### Using Nix ####
+The expected output is::
 
-Although the recommended platform is Debian or Ubuntu, Nix can be used to build VTR on other platforms, such as MacOS.
-
-If you don't have [Nix](https://nixos.org/nix/), you can [get it](https://nixos.org/nix/download.html) with:
-
-```shell
-$ curl -L https://nixos.org/nix/install | sh
-```
-
-These commands will set up dependencies for Linux and MacOS and build VTR:
-
-```shell
-#In the VTR root
-$ nix-shell dev/nix/shell.nix
-$ make
-```
-
-### Building using the Makefile wrapper ###
-Run `make` from the root of the VTR source tree
-
-```shell
-#In the VTR root
-$ make
-...
-[100%] Built target vpr
-```
-
-
-#### Specifying the build type ####
-You can specify the build type by passing the `BUILD_TYPE` parameter.
-
-For instance to create a debug build (no optimization and debug symbols):
-
-```shell
-#In the VTR root
-$ make BUILD_TYPE=debug
-...
-[100%] Built target vpr
-```
-
-
-#### Passing parameters to CMake ####
-You can also pass parameters to CMake.
-
-For instance to set the CMake configuration variable `VTR_ENABLE_SANITIZE` on:
-
-```shell
-#In the VTR root
-$ make CMAKE_PARAMS="-DVTR_ENABLE_SANITIZE=ON"
-...
-[100%] Built target vpr
-```
-
-Both the `BUILD_TYPE` and `CMAKE_PARAMS` can be specified concurrently:
-```shell
-#In the VTR root
-$ make BUILD_TYPE=debug CMAKE_PARAMS="-DVTR_ENABLE_SANITIZE=ON"
-...
-[100%] Built target vpr
-```
-
-
-### Using CMake directly ###
-You can also use cmake directly.
-
-First create a build directory under the VTR root:
-
-```shell
-#In the VTR root
-$ mkdir build
-$ cd build
-
-#Call cmake pointing to the directory containing the root CMakeLists.txt
-$ cmake ..
-
-#Build
-$ make
-```
-
-#### Changing configuration on the command line ####
-You can change the CMake configuration by passing command line parameters.
-
-For instance to set the configuration to debug:
-
-```shell
-#In the build directory
-$ cmake . -DCMAKE_BUILD_TYPE=debug
-
-#Re-build
-$ make
-```
-
-#### Changing configuration interactively with ccmake ####
-You can also use `ccmake` to to modify the build configuration.
-
-```shell
-#From the build directory
-$ ccmake . #Make some configuration change
-
-#Build
-$ make
-```
-
-## Other platforms ##
-
-CMake supports a variety of operating systems and can generate project files for a variety of build systems and IDEs.
-While VTR is developed primarily on Linux, it should be possible to build on different platforms (your milage may vary).
-See the [CMake documentation](https://cmake.org) for more details about using cmake and generating project files on other platforms and build systems (e.g. Eclipse, Microsoft Visual Studio).
-
-
-### Microsoft Windows ###
-
-*NOTE: VTR support on Microsoft Windows is considered experimental*
-
-#### Cygwin ####
-[Cygwin](https://www.cygwin.com/) provides a POSIX (i.e. unix-like) environment for Microsoft Windows.
-
-From within the cygwin terminal follow the Unix-like build instructions listed above.
-
-Note that the generated executables will rely upon Cygwin (e.g. `cygwin1.dll`) for POSIX compatibility.
-
-#### Cross-compiling from Linux to Microsoft Windows with MinGW-W64 ####
-It is possible to cross-compile from a Linux host system to generate Microsoft Windows executables using the [MinGW-W64](https://mingw-w64.org) compilers.
-These can usually be installed with your Linux distribution's package manager (e.g. `sudo apt-get install mingw-w64` on Debian/Ubuntu).
-
-Unlike Cygwin, MinGW executables will depend upon the standard Microsoft Visual C++ run-time.
-
-To build VTR using MinGW:
-```shell
-#In the VTR root
-$ mkdir build_win64
-$ cd build_win64
-
-#Run cmake specifying the toolchain file to setup the cross-compilation environment
-$ cmake .. -DCMAKE_TOOLCHAIN_FILE ../cmake/toolchains/mingw-linux-cross-compile-to-windows.cmake
-
-#Building will produce Windows executables
-$ make
-```
-
-Note that by default the MS Windows target system will need to dynamically link to the `libgcc` and `libstdc++` DLLs.
-These are usually found under /usr/lib/gcc on the Linux host machine.
-
-See the [toolchain file](cmake/toolchains/mingw-linux-cross-compile-to-windows.cmake) for more details.
-
-#### Microsoft Visual Studio ####
-CMake can generate a Microsft Visual Studio project, enabling VTR to be built with the Microsoft Visual C++ (MSVC) compiler.
-
-##### Installing additional tools #####
-VTR depends on some external unix-style tools during it's buid process; in particular the `flex` and `bison` parser generators.
-
-One approach is to install these tools using [MSYS2](http://www.msys2.org/), which provides up-to-date versions of many unix tools for MS Windows.
-
-To ensure CMake can find the `flex` and `bison` executables you must ensure that they are available on your system path.
-For instance, if MSYS2 was installed to `C:\msys64` you would need to ensure that `C:\msys64\usr\bin` was included in the system PATH environment variable.
-
-##### Generating the Visual Studio Project #####
-CMake (e.g. the `cmake-gui`) can then be configured to generate the MSVC project.
+    k6_N10_memSize16384_memData64_40nm_timing/ch_intrinsics...OK

CMakeLists.txt

@@ -451,3 +451,9 @@ list(APPEND DIRS_TO_FORMAT_PY "${CMAKE_CURRENT_SOURCE_DIR}/vpr")
 list(APPEND DIRS_TO_FORMAT_PY "${CMAKE_CURRENT_SOURCE_DIR}/vtr_flow")
 
 include(AutoPyFormat)
+
+
+#
+# Python Environment setup
+#
+include(PyEnv)

CONTRIBUTING.md

@@ -154,7 +154,7 @@ The following are general tips for making your pull requests easy to review (and
 * Keep changes small
 
     Large change sets are difficult and time-consuming to review.
-    If a change set is becoming too large, consider splitting it into smaller pieces; you'll probably want to [file an issue](#filling-feature-requests) to discuss things first.
+    If a change set is becoming too large, consider splitting it into smaller pieces; you'll probably want to [file an issue](#filling-enhancement-requests) to discuss things first.
 
 * Do one thing only
 

README.developers.md

@@ -966,7 +966,7 @@ $ make
 this turns on more extensive assertion checking and re-builds VTR.
 
 ## GDB Pretty Printers
-To make it easier to debug some of VTR's data structures with [GDB](www.gnu.org/gdb).
+To make it easier to debug some of VTR's data structures with [GDB](https://www.sourceware.org/gdb/).
 
 ### STL Pretty Printers
 
@@ -1145,7 +1145,7 @@ make CMAKE_PARAMS="-DVTR_IPO_BUILD=off" -j8 vpr
 # External Subtrees
 VTR includes some code which is developed in external repositories, and is integrated into the VTR source tree using [git subtrees](https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree).
 
-To simplify the process of working with subtrees we use the [`dev/external_subtrees.py`](./dev/external_subtrees.py) script.
+To simplify the process of working with subtrees we use the [`dev/external_subtrees.py`](https://github.com/verilog-to-routing/vtr-verilog-to-routing/blob/master/dev/external_subtrees.py) script.
 
 For instance, running `./dev/external_subtrees.py --list` from the VTR root it shows the subtrees:
 ```
@@ -1159,15 +1159,15 @@ Component: libtatum        Path: libs/EXTERNAL/libtatum         URL: https://git
 Code included in VTR by subtrees should *not be modified within the VTR source tree*.
 Instead changes should be made in the relevant up-stream repository, and then synced into the VTR tree.
 
-### Updating an existing Subtree
+## Updating an existing Subtree
 1. From the VTR root run: `./dev/external_subtrees.py $SUBTREE_NAME`, where `$SUBTREE_NAME` is the name of an existing subtree.
 
     For example to update the `libtatum` subtree:
     ```shell
     ./dev/external_subtrees.py --update libtatum
     ```
 
-### Adding a new Subtree
+## Adding a new Subtree
 
 To add a new external subtree to VTR do the following:
 
@@ -1207,7 +1207,7 @@ To add a new external subtree to VTR do the following:
     The first will squash all the upstream changes, the second will merge those changes into the current branch.
 
 
-### Subtree Rational
+## Subtree Rational
 
 VTR uses subtrees to allow easy tracking of upstream dependencies.
 
@@ -1221,7 +1221,7 @@ See [here](https://blogs.atlassian.com/2013/05/alternatives-to-git-submodule-git
 # Finding Bugs with Coverity
 [Coverity Scan](https://scan.coverity.com) is a static code analysis service which can be used to detect bugs.
 
-### Browsing Defects
+## Browsing Defects
 To view defects detected do the following:
 
 1. Get a coverity scan account
@@ -1231,7 +1231,7 @@ To view defects detected do the following:
 2. Browse the existing defects through the coverity web interface
 
 
-### Submitting a build
+## Submitting a build
 To submit a build to coverity do the following:
 
 1. [Download](https://scan.coverity.com/download) the coverity build tool
@@ -1264,7 +1264,7 @@ Note that we explicitly asked for gcc and g++, the coverity build tool defaults
 
 Once the build has been analyzed you can browse the latest results through the coverity web interface
 
-### No files emitted
+## No files emitted
 If you get the following warning from cov-build:
 
     [WARNING] No files were emitted.
@@ -1303,5 +1303,5 @@ The following outlines the procedure to following when making an official VTR re
  * GitHub will automatically create a release based on the tag
  * Add the new change log entry to the [GitHub release description](https://github.com/verilog-to-routing/vtr-verilog-to-routing/releases)
  * Update the [ReadTheDocs configuration](https://readthedocs.org/projects/vtr/versions/) to build and serve documentation for the relevant tag (e.g. `v8.0.0`)
- * Send a release announcement email to the [vtr-announce]([email protected]) mailing list (make sure to thank all contributors!)
+ * Send a release announcement email to the [vtr-announce](mailto:[email protected]) mailing list (make sure to thank all contributors!)
 

cmake/modules/PyEnv.cmake

@@ -0,0 +1,7 @@
+# Create a python virtual environment using venv and install all necessary packages
+# wheel should be installed before the requirements.txt list to prevent errors.
+add_custom_target(env
+    COMMAND python3 -m venv ../.venv
+    COMMAND . ../.venv/bin/activate && pip3 install wheel
+    COMMAND . ../.venv/bin/activate && pip3 install -r ../requirements.txt
+)