-
Notifications
You must be signed in to change notification settings - Fork 416
Refactor build/dependency documentation + new markdown parser (myst-parser) #1914
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -104,6 +104,7 @@ ODIN_II/usefull_tools/**/track_completed | |
| #Python | ||
| # | ||
| *.pyc | ||
| /.venv | ||
|
|
||
| # | ||
| #Vim | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,255 +1,49 @@ | ||
| # Building VTR # | ||
| # Building VTR | ||
|
|
||
|
|
||
| ## Overview ## | ||
| ## Setting up Your Environment | ||
|
|
||
| VTR uses [CMake](https://cmake.org) as it's build system. | ||
|
|
||
| CMake provides a portable cross-platform build systems with many useful features. | ||
| VTR requires several system packages. From the top-level directory, run the following script to install the required packages on a modern Debian or Ubuntu system: | ||
|
|
||
| ## Tested Compilers ## | ||
| VTR requires a C++-14 compliant compiler. | ||
| It is tested against the default compilers of all Debian and Ubuntu releases within their standard support lifetime. Currently, those are the following: | ||
| ./install_apt_packages.sh | ||
|
|
||
| * GCC/G++: 7, 8, 9, 10, 11 | ||
| * Clang/Clang++: 6, 7, 10 | ||
|
|
||
| Other compilers may work but are untested (your milage may vary). | ||
| You will also need several Python packages. You can optionally install and activate a Python virtual environment so that you do not need to modify your system Python installation: | ||
|
|
||
| ## Unix-like ## | ||
| For unix-like systems we provide a wrapper Makefile which supports the traditional `make` and `make clean` commands, but calls CMake behind the scenes. | ||
| make env | ||
| source .venv/bin/activate | ||
|
|
||
| ### Dependencies ### | ||
| Then to install the Python packages: | ||
|
|
||
| For the basic tools you need: | ||
| * Bison & Flex | ||
| * cmake, make | ||
| * A modern C++ compiler supporting C++14 (such as GCC >= 4.9 or clang >= 3.6) | ||
| pip install -r requirements.txt | ||
|
|
||
| For the VPR GUI you need: | ||
| * Cairo | ||
| * FreeType | ||
| * Xft (libXft + libX11) | ||
| * fontconfig | ||
| * libgtk-3-dev | ||
| **Note:** If you chose to install the Python virtual environment, you will need to remember to activate it on any new terminal window you use, before you can run the VTR flow or regressions tests (`source .venv/bin/activate`). | ||
|
|
||
| For the [regression testing and benchmarking](README.developers.md#running-tests) you will need: | ||
| * Perl + List::MoreUtils | ||
| * Python | ||
| * time | ||
| ## Building | ||
|
|
||
| It is also recommended you install the following development tools: | ||
| * git | ||
| * ctags | ||
| * gdb | ||
| * valgrind | ||
| * clang-format-7 | ||
| From the top-level, run: | ||
|
|
||
| For Docs generation you will need: | ||
| * Doxygen | ||
| * python-sphinx | ||
| * python-sphinx-rtd-theme | ||
| * python-recommonmark | ||
| make | ||
|
|
||
| #### Debian & Ubuntu #### | ||
| which will build all the required tools. | ||
|
|
||
| The following should be enough to get the tools, VPR GUI and tests going on a modern Debian or Ubuntu system: | ||
| The complete VTR flow has been tested on 64-bit Linux systems. | ||
| The flow should work in other platforms (32-bit Linux, Windows with cygwin) but this is untested. | ||
|
|
||
| ```shell | ||
| apt-get install \ | ||
| build-essential \ | ||
| flex \ | ||
| bison \ | ||
| cmake \ | ||
| fontconfig \ | ||
| libcairo2-dev \ | ||
| libfontconfig1-dev \ | ||
| libx11-dev \ | ||
| libxft-dev \ | ||
| libgtk-3-dev \ | ||
| perl \ | ||
| liblist-moreutils-perl \ | ||
| python \ | ||
| time | ||
| ``` | ||
| *Full information about building VTR, including setting up required system packages and Python packages, can be found in [Optional Build Information](doc/src/vtr/optional_build_info.md) page.* | ||
|
|
||
| For documentation generation these additional packages are required: | ||
| Please [let us know](doc/src/contact.md) your experience with building VTR so that we can improve the experience for others. | ||
|
|
||
| ```shell | ||
| apt-get install \ | ||
| doxygen \ | ||
| python-sphinx \ | ||
| python-sphinx-rtd-theme \ | ||
| python-recommonmark | ||
| ``` | ||
| The tools included official VTR releases have been tested for compatibility. | ||
| If you download a different version of those tools, then those versions may not be mutually compatible with the VTR release. | ||
|
|
||
| ## Verifying Installation | ||
|
|
||
| For development the following additional packages are useful: | ||
| To verfiy that VTR has been installed correctly run:: | ||
|
|
||
| ```shell | ||
| apt-get install \ | ||
| git \ | ||
| valgrind \ | ||
| gdb \ | ||
| ctags | ||
| ``` | ||
| ./vtr_flow/scripts/run_vtr_task.py basic_flow | ||
|
|
||
| #### Using Nix #### | ||
| The expected output is:: | ||
|
|
||
| Although the recommended platform is Debian or Ubuntu, Nix can be used to build VTR on other platforms, such as MacOS. | ||
|
|
||
| If you don't have [Nix](https://nixos.org/nix/), you can [get it](https://nixos.org/nix/download.html) with: | ||
|
|
||
| ```shell | ||
| $ curl -L https://nixos.org/nix/install | sh | ||
| ``` | ||
|
|
||
| These commands will set up dependencies for Linux and MacOS and build VTR: | ||
|
|
||
| ```shell | ||
| #In the VTR root | ||
| $ nix-shell dev/nix/shell.nix | ||
| $ make | ||
| ``` | ||
|
|
||
| ### Building using the Makefile wrapper ### | ||
| Run `make` from the root of the VTR source tree | ||
|
|
||
| ```shell | ||
| #In the VTR root | ||
| $ make | ||
| ... | ||
| [100%] Built target vpr | ||
| ``` | ||
|
|
||
|
|
||
| #### Specifying the build type #### | ||
| You can specify the build type by passing the `BUILD_TYPE` parameter. | ||
|
|
||
| For instance to create a debug build (no optimization and debug symbols): | ||
|
|
||
| ```shell | ||
| #In the VTR root | ||
| $ make BUILD_TYPE=debug | ||
| ... | ||
| [100%] Built target vpr | ||
| ``` | ||
|
|
||
|
|
||
| #### Passing parameters to CMake #### | ||
| You can also pass parameters to CMake. | ||
|
|
||
| For instance to set the CMake configuration variable `VTR_ENABLE_SANITIZE` on: | ||
|
|
||
| ```shell | ||
| #In the VTR root | ||
| $ make CMAKE_PARAMS="-DVTR_ENABLE_SANITIZE=ON" | ||
| ... | ||
| [100%] Built target vpr | ||
| ``` | ||
|
|
||
| Both the `BUILD_TYPE` and `CMAKE_PARAMS` can be specified concurrently: | ||
| ```shell | ||
| #In the VTR root | ||
| $ make BUILD_TYPE=debug CMAKE_PARAMS="-DVTR_ENABLE_SANITIZE=ON" | ||
| ... | ||
| [100%] Built target vpr | ||
| ``` | ||
|
|
||
|
|
||
| ### Using CMake directly ### | ||
| You can also use cmake directly. | ||
|
|
||
| First create a build directory under the VTR root: | ||
|
|
||
| ```shell | ||
| #In the VTR root | ||
| $ mkdir build | ||
| $ cd build | ||
|
|
||
| #Call cmake pointing to the directory containing the root CMakeLists.txt | ||
| $ cmake .. | ||
|
|
||
| #Build | ||
| $ make | ||
| ``` | ||
|
|
||
| #### Changing configuration on the command line #### | ||
| You can change the CMake configuration by passing command line parameters. | ||
|
|
||
| For instance to set the configuration to debug: | ||
|
|
||
| ```shell | ||
| #In the build directory | ||
| $ cmake . -DCMAKE_BUILD_TYPE=debug | ||
|
|
||
| #Re-build | ||
| $ make | ||
| ``` | ||
|
|
||
| #### Changing configuration interactively with ccmake #### | ||
| You can also use `ccmake` to to modify the build configuration. | ||
|
|
||
| ```shell | ||
| #From the build directory | ||
| $ ccmake . #Make some configuration change | ||
|
|
||
| #Build | ||
| $ make | ||
| ``` | ||
|
|
||
| ## Other platforms ## | ||
|
|
||
| CMake supports a variety of operating systems and can generate project files for a variety of build systems and IDEs. | ||
| While VTR is developed primarily on Linux, it should be possible to build on different platforms (your milage may vary). | ||
| See the [CMake documentation](https://cmake.org) for more details about using cmake and generating project files on other platforms and build systems (e.g. Eclipse, Microsoft Visual Studio). | ||
|
|
||
|
|
||
| ### Microsoft Windows ### | ||
|
|
||
| *NOTE: VTR support on Microsoft Windows is considered experimental* | ||
|
|
||
| #### Cygwin #### | ||
| [Cygwin](https://www.cygwin.com/) provides a POSIX (i.e. unix-like) environment for Microsoft Windows. | ||
|
|
||
| From within the cygwin terminal follow the Unix-like build instructions listed above. | ||
|
|
||
| Note that the generated executables will rely upon Cygwin (e.g. `cygwin1.dll`) for POSIX compatibility. | ||
|
|
||
| #### Cross-compiling from Linux to Microsoft Windows with MinGW-W64 #### | ||
| It is possible to cross-compile from a Linux host system to generate Microsoft Windows executables using the [MinGW-W64](https://mingw-w64.org) compilers. | ||
| These can usually be installed with your Linux distribution's package manager (e.g. `sudo apt-get install mingw-w64` on Debian/Ubuntu). | ||
|
|
||
| Unlike Cygwin, MinGW executables will depend upon the standard Microsoft Visual C++ run-time. | ||
|
|
||
| To build VTR using MinGW: | ||
| ```shell | ||
| #In the VTR root | ||
| $ mkdir build_win64 | ||
| $ cd build_win64 | ||
|
|
||
| #Run cmake specifying the toolchain file to setup the cross-compilation environment | ||
| $ cmake .. -DCMAKE_TOOLCHAIN_FILE ../cmake/toolchains/mingw-linux-cross-compile-to-windows.cmake | ||
|
|
||
| #Building will produce Windows executables | ||
| $ make | ||
| ``` | ||
|
|
||
| Note that by default the MS Windows target system will need to dynamically link to the `libgcc` and `libstdc++` DLLs. | ||
| These are usually found under /usr/lib/gcc on the Linux host machine. | ||
|
|
||
| See the [toolchain file](cmake/toolchains/mingw-linux-cross-compile-to-windows.cmake) for more details. | ||
|
|
||
| #### Microsoft Visual Studio #### | ||
| CMake can generate a Microsft Visual Studio project, enabling VTR to be built with the Microsoft Visual C++ (MSVC) compiler. | ||
|
|
||
| ##### Installing additional tools ##### | ||
| VTR depends on some external unix-style tools during it's buid process; in particular the `flex` and `bison` parser generators. | ||
|
|
||
| One approach is to install these tools using [MSYS2](http://www.msys2.org/), which provides up-to-date versions of many unix tools for MS Windows. | ||
|
|
||
| To ensure CMake can find the `flex` and `bison` executables you must ensure that they are available on your system path. | ||
| For instance, if MSYS2 was installed to `C:\msys64` you would need to ensure that `C:\msys64\usr\bin` was included in the system PATH environment variable. | ||
|
|
||
| ##### Generating the Visual Studio Project ##### | ||
| CMake (e.g. the `cmake-gui`) can then be configured to generate the MSVC project. | ||
| k6_N10_memSize16384_memData64_40nm_timing/ch_intrinsics...OK | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -966,7 +966,7 @@ $ make | |
| this turns on more extensive assertion checking and re-builds VTR. | ||
|
|
||
| ## GDB Pretty Printers | ||
| To make it easier to debug some of VTR's data structures with [GDB](www.gnu.org/gdb). | ||
| To make it easier to debug some of VTR's data structures with [GDB](https://www.sourceware.org/gdb/). | ||
|
|
||
| ### STL Pretty Printers | ||
|
|
||
|
|
@@ -1145,7 +1145,7 @@ make CMAKE_PARAMS="-DVTR_IPO_BUILD=off" -j8 vpr | |
| # External Subtrees | ||
| VTR includes some code which is developed in external repositories, and is integrated into the VTR source tree using [git subtrees](https://www.atlassian.com/blog/git/alternatives-to-git-submodule-git-subtree). | ||
|
|
||
| To simplify the process of working with subtrees we use the [`dev/external_subtrees.py`](./dev/external_subtrees.py) script. | ||
| To simplify the process of working with subtrees we use the [`dev/external_subtrees.py`](https://github.com/verilog-to-routing/vtr-verilog-to-routing/blob/master/dev/external_subtrees.py) script. | ||
|
|
||
| For instance, running `./dev/external_subtrees.py --list` from the VTR root it shows the subtrees: | ||
| ``` | ||
|
|
@@ -1159,15 +1159,15 @@ Component: libtatum Path: libs/EXTERNAL/libtatum URL: https://git | |
| Code included in VTR by subtrees should *not be modified within the VTR source tree*. | ||
| Instead changes should be made in the relevant up-stream repository, and then synced into the VTR tree. | ||
|
|
||
| ### Updating an existing Subtree | ||
| ## Updating an existing Subtree | ||
| 1. From the VTR root run: `./dev/external_subtrees.py $SUBTREE_NAME`, where `$SUBTREE_NAME` is the name of an existing subtree. | ||
|
|
||
| For example to update the `libtatum` subtree: | ||
| ```shell | ||
| ./dev/external_subtrees.py --update libtatum | ||
| ``` | ||
|
|
||
| ### Adding a new Subtree | ||
| ## Adding a new Subtree | ||
|
|
||
| To add a new external subtree to VTR do the following: | ||
|
|
||
|
|
@@ -1207,7 +1207,7 @@ To add a new external subtree to VTR do the following: | |
| The first will squash all the upstream changes, the second will merge those changes into the current branch. | ||
|
|
||
|
|
||
| ### Subtree Rational | ||
| ## Subtree Rational | ||
|
|
||
| VTR uses subtrees to allow easy tracking of upstream dependencies. | ||
|
|
||
|
|
@@ -1221,7 +1221,7 @@ See [here](https://blogs.atlassian.com/2013/05/alternatives-to-git-submodule-git | |
| # Finding Bugs with Coverity | ||
| [Coverity Scan](https://scan.coverity.com) is a static code analysis service which can be used to detect bugs. | ||
|
|
||
| ### Browsing Defects | ||
| ## Browsing Defects | ||
| To view defects detected do the following: | ||
|
|
||
| 1. Get a coverity scan account | ||
|
|
@@ -1231,7 +1231,7 @@ To view defects detected do the following: | |
| 2. Browse the existing defects through the coverity web interface | ||
|
|
||
|
|
||
| ### Submitting a build | ||
| ## Submitting a build | ||
| To submit a build to coverity do the following: | ||
|
|
||
| 1. [Download](https://scan.coverity.com/download) the coverity build tool | ||
|
|
@@ -1264,7 +1264,7 @@ Note that we explicitly asked for gcc and g++, the coverity build tool defaults | |
|
|
||
| Once the build has been analyzed you can browse the latest results through the coverity web interface | ||
|
|
||
| ### No files emitted | ||
| ## No files emitted | ||
| If you get the following warning from cov-build: | ||
|
|
||
| [WARNING] No files were emitted. | ||
|
|
@@ -1303,5 +1303,5 @@ The following outlines the procedure to following when making an official VTR re | |
| * GitHub will automatically create a release based on the tag | ||
| * Add the new change log entry to the [GitHub release description](https://github.com/verilog-to-routing/vtr-verilog-to-routing/releases) | ||
| * Update the [ReadTheDocs configuration](https://readthedocs.org/projects/vtr/versions/) to build and serve documentation for the relevant tag (e.g. `v8.0.0`) | ||
| * Send a release announcement email to the [vtr-announce]([email protected]) mailing list (make sure to thank all contributors!) | ||
| * Send a release announcement email to the [vtr-announce](mailto:[email protected]) mailing list (make sure to thank all contributors!) | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| # Create a python virtual environment using venv and install all necessary packages | ||
| # wheel should be installed before the requirements.txt list to prevent errors. | ||
| add_custom_target(env | ||
| COMMAND python3 -m venv ../.venv | ||
| COMMAND . ../.venv/bin/activate && pip3 install wheel | ||
| COMMAND . ../.venv/bin/activate && pip3 install -r ../requirements.txt | ||
| ) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can / should the source .venv/bin/activate command be done by make env? (one less command to run).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately there's no way to affect the environment of the caller's process (the user's terminal) from within a Makefile command. https://stackoverflow.com/a/7508273
What I've seen done in other projects (including several Symbiflow projects) is to activate the environment from within a Makefile target, which would have effect for only that target. Since we don't run the VTR flow using Makefile targets (but rather have users directly invoke the Python scripts) that doesn't really help.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, that's a good reason :).