Skip to content

Native Windows Compile Fixes For vpr #1582

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 19, 2020

Conversation

cr1901
Copy link
Contributor

@cr1901 cr1901 commented Nov 3, 2020

Description

This PR adds some fixes so that mingw-w64 gcc can successfully compiler vpr. Only vpr is known to compile as of this PR.

Specifically:

  • Use _aligned_malloc and _aligned_free on Windows instead of
  • Change long to uintptr_t or intptr_t based on context in libvqm.
  • Do not include sys/mman.h header in libvtrcapnproto.

Related Issue

This PR should close #1317, but includes additional fixes so that compilation succeeds using gcc targeting native windows exes.

Motivation and Context

Per @vaughnbetz in #1317, getting a Windows compile without Cygwin is desired, but hasn't worked in a few months.

How Has This Been Tested?

See below in Checklist for how I tested changes. long to uintptr_t or intptr_t should be a no-op on LP64 systems like Linux (and thus have no effect on the compiled code). However, I am unable to test immediately on Linux.

Environment:

  • CMake command line:

    cmake -G"Ninja" -DCMAKE_INSTALL_PREFIX=/mingw64 -DWITH_ABC=OFF -DWITH_ODIN=OFF ..
    
  • System:

    William@DESKTOP-H0PMN4M MINGW64 ~/Projects/FPGA/vtr-verilog-to-routing
    $ uname -a
    MINGW64_NT-10.0-18363 DESKTOP-H0PMN4M 3.0.7-338.x86_64 2019-07-11 10:58 UTC x86_64 Msys
    
  • Compiler:

    William@DESKTOP-H0PMN4M MINGW64 ~/Projects/FPGA/vtr-verilog-to-routing
    $ g++ -v
    Using built-in specs.
    COLLECT_GCC=C:\msys64\mingw64\bin\g++.exe
    COLLECT_LTO_WRAPPER=C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/9.2.0/lto-wrapper.exe
    Target: x86_64-w64-mingw32
    Configured with: ../gcc-9.2.0/configure --prefix=/mingw64 --with-local-prefix=/mingw64/local --build=x86_64-w64-mingw32 --host=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --with-native-system-header-dir=/mingw64/x86_64-w64-mingw32/include --libexecdir=/mingw64/lib --enable-bootstrap --with-arch=x86-64 --with-tune=generic --enable-languages=c,lto,c++,fortran,ada,objc,obj-c++ --enable-shared --enable-static --enable-libatomic --enable-threads=posix --enable-graphite --enable-fully-dynamic-string --enable-libstdcxx-filesystem-ts=yes --enable-libstdcxx-time=yes --disable-libstdcxx-pch --disable-libstdcxx-debug --disable-isl-version-check --enable-lto --enable-libgomp --disable-multilib --enable-checking=release --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --enable-plugin -- with-libiconv --with-system-zlib --with-gmp=/mingw64 --with-mpfr=/mingw64 --with-mpc=/mingw64 --with-isl=/mingw64 --with-pkgversion='Rev2, Built by MSYS2 project' --with-bugurl=https://sourceforge.net/projects/msys2 --with-gnu-as --with-gnu-ld
    Thread model: posix
    gcc version 9.2.0 (Rev2, Built by MSYS2 project)
    

Types of changes

  • Bug fix (change which fixes an issue)
  • New feature (change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

  • My change requires a change to the documentation
  • I have updated the documentation accordingly
  • I have added tests to cover my changes
  • All new and existing tests passed

First three are not applicable. As for the tests...

I have only compiled vpr, as that is at present the only part of vtr that I plan to work on. When I try running python run_reg_test.py vtr_reg_basic, the test hangs without any output, most likely because I only installed vpr and not the rest (fwiw, abc is known to compile fine on Windows).

However, when I run test_vpr, and test_fasm, I get the following output:

test_vpr:

William@DESKTOP-H0PMN4M MINGW64 ~/Projects/FPGA/vtr-verilog-to-routing/build/vpr
$ ./test_vpr.exe

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_vpr.exe is a Catch v1.11.0 host application.
Run with -? for options

-------------------------------------------------------------------------------
read_arch_metadata
-------------------------------------------------------------------------------
../vpr/test/test_vpr.cpp:18
...............................................................................

../vpr/test/test_vpr.cpp:18: FAILED:
due to unexpected exception with message:
  Failed to open file

VPR FPGA Placement and Routing.
Version: 8.1.0-dev+0ff5a3d89-dirty
Revision: v8.0.0-2892-g0ff5a3d89-dirty
Compiled: 2020-11-03T01:40:31
Compiler: GNU 9.2.0 on Windows-10.0.18363 AMD64
Build Info: Release IPO VTR_ASSERT_LEVEL=2

University of Toronto
verilogtorouting.org
[email protected]
This is free open source code under MIT license.

VPR was run with the following command-line:
test_vpr test_read_arch_metadata.xml wire.eblif --route_chan_width 100


Architecture file: test_read_arch_metadata.xml
Circuit name: wire

Loading Architecture Description
Loading Architecture Description took 0.00 seconds (max_rss 0.0 MiB, delta_rss +0.0 MiB)
-------------------------------------------------------------------------------
read_rr_graph_metadata
-------------------------------------------------------------------------------
../vpr/test/test_vpr.cpp:114
...............................................................................

../vpr/test/test_vpr.cpp:114: FAILED:
due to unexpected exception with message:
  Failed to open file

Loading router wire lookahead map
Loading router wire lookahead map took 0.00 seconds (max_rss 0.0 MiB, delta_rss +0.0 MiB) VPR FPGA Placement and Routing.
Version: 8.1.0-dev+0ff5a3d89-dirty
Revision: v8.0.0-2892-g0ff5a3d89-dirty
Compiled: 2020-11-03T01:40:31
Compiler: GNU 9.2.0 on Windows-10.0.18363 AMD64
Build Info: Release IPO VTR_ASSERT_LEVEL=2

University of Toronto
verilogtorouting.org
[email protected]
This is free open source code under MIT license.

VPR was run with the following command-line:
test_vpr ../../vtr_flow/arch/timing/k6_frac_N10_mem32K_40nm.xml wire.eblif --route_chan_width 100


Architecture file: ../../vtr_flow/arch/timing/k6_frac_N10_mem32K_40nm.xml
Circuit name: wire

Loading Architecture Description
Loading Architecture Description took 0.03 seconds (max_rss 0.0 MiB, delta_rss +0.0 MiB)
Building complex block graph
Warning 1: io[0].clock[0] unconnected pin in architecture.
Building complex block graph took 0.04 seconds (max_rss 0.0 MiB, delta_rss +0.0 MiB)
Load circuit
Load circuit took 0.00 seconds (max_rss 0.0 MiB, delta_rss +0.0 MiB)
-------------------------------------------------------------------------------
connection_router
-------------------------------------------------------------------------------
../vpr/test/test_connection_router.cpp:110
...............................................................................

../vpr/test/test_connection_router.cpp:110: FAILED:
due to unexpected exception with message:
  Failed to open file

===============================================================================
test cases:    19 |    16 passed | 3 failed
assertions: 59875 | 59872 passed | 3 failed

test_fasm:

William@DESKTOP-H0PMN4M MINGW64 ~/Projects/FPGA/vtr-verilog-to-routing/build/utils/fasm
$ ./test_fasm.exe
VPR FPGA Placement and Routing.
Version: 8.1.0-dev+0ff5a3d89-dirty
Revision: v8.0.0-2892-g0ff5a3d89-dirty
Compiled: 2020-11-03T01:40:31
Compiler: GNU 9.2.0 on Windows-10.0.18363 AMD64
Build Info: Release IPO VTR_ASSERT_LEVEL=2

University of Toronto
verilogtorouting.org
[email protected]
This is free open source code under MIT license.

VPR was run with the following command-line:
test_vpr test_fasm_arch.xml wire.eblif --route_chan_width 100


Architecture file: test_fasm_arch.xml
Circuit name: wire

Loading Architecture Description
Loading Architecture Description took 0.00 seconds (max_rss 0.0 MiB, delta_rss +0.0 MiB)

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test_fasm.exe is a Catch v1.11.0 host application.
Run with -? for options

-------------------------------------------------------------------------------
fasm_integration_test
-------------------------------------------------------------------------------
../utils/fasm/test/test_fasm.cpp:171
...............................................................................

../utils/fasm/test/test_fasm.cpp:171: FAILED:
due to unexpected exception with message:
  Failed to open file

===============================================================================
test cases:    14 |    13 passed | 1 failed
assertions: 11377 | 11376 passed | 1 failed

Error 1: fasm placeholder 'prefix' not defined!

Looks like most things are working, though it appears that the test binaries don't like me running them directly from the build directory. How should I go about testing with these binaries properly?

As posix_memalign does not exist on Windows, we can use the above
functions as a workaround until C++20 support matures.
Use uintptr_t in parser file- the warning is outdated. For all other
conversions, intptr_t is chosen as the equivalent of long used as a
pointer on LP64, and the equivalent of long long on LLP64.
sys/mman.h is not required to compiler libvtrcapnproto. To keep
changes minimally invasive, headers such as unistd.h are retained.
These headers may only exist as part of the MinGW-w64 compiler on
Windows and not MSVC.
@@ -360,7 +361,7 @@ t_pin_def *add_pin(char *name, int left, int right, t_pin_def_type type, t_parse

//Append the element, the position is one less thatn the size of the array
//which is returned by append_array_element
size_t position = append_array_element( (long) pin, pin_list) - 1;
size_t position = append_array_element( (intptr_t) pin, pin_list) - 1;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you should change long to intptr_t. We're using id-based structures in the code I looked at below, where we are inserting things into vectors, setting integer ids and such. We don't need a pointer for that; just an integer of sufficient size. So long is appropriate for most cases, and for some smaller ids we use shorts.

Copy link
Contributor Author

@cr1901 cr1901 Nov 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

long => int64_t then? The problem with long on Windows is that it's not the same size as a pointer on 64-bit windows (long is 32 bit). So all the conversions from pointer to an integer will truncate (and cause compiler errors). What about a typedef such as id_t to convey meaning?

As for the compiler errors, I'll take a look. At least for the one you showed, looks like I forgot a header...

Also, Idk if signedness actually matters, but I've been using signed numbers when possible to match where the code uses signed numbers.

Copy link
Contributor Author

@cr1901 cr1901 Nov 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vaughnbetz Friendly ping... what should I do instead of changing long to intptr_t? Keeping long here will not work because long is not 64-bits long, even on 64-bit Windows. So I need to change the type to something that will be the same size as a pointer (since we cast between pointer and int) on all platforms vpr runs on. Unless you prefer a platform-specific typedef, intptr_t seems to be the most appropriate type.

However, I am unable to test immediately on Linux.

I am now able to test on Linux :), so I'll make sure I can compile locally before pushing new changes. Should I rebase to remove the intptr_t changes from history (if applicable), or leave as-is and keep committing on-top? Gonna guess rebase is appropriate since I broke the build, but unsure.

@vaughnbetz
Copy link
Contributor

There are a lot of build errors (see the CI above; here is an example): 77%] Building CXX object libs/libvqm/CMakeFiles/libvqm.dir/vqm_dll.cpp.o
In file included from /home/travis/build/verilog-to-routing/vtr-verilog-to-routing/libs/libvqm/vqm_dll.cpp:19:0:
/home/travis/build/verilog-to-routing/vtr-verilog-to-routing/libs/libvqm/vqm_parser.y:39:2: error: ‘uintptr_t’ does not name a type
uintptr_t value;
^
libs/libvqm/CMakeFiles/libvqm.dir/build.make:86: recipe for target 'libs/libvqm/CMakeFiles/libvqm.dir/vqm_dll.cpp.o' failed

I also think this PR makes many unnecessary changes of long to intptr_t that aren't needed and obscure meaning.

To do a quick test on windows you could manually run vpr using the commands at https://docs.verilogtorouting.org/en/latest/quickstart/

For Linux, you would need to pass the full CI, which is run automatically on each PR (and has been run above but had failures).

@vaughnbetz
Copy link
Contributor

OK, I stand corrected on intptr_t. I talked to Kevin Murray today and he said libvqm is weird; it is indeed using long for pointers. Sorry, I didn't catch that your long->intptr_t changes were all in libvqm. We don't use long for pointers in the main vpr code base, so I thought you were changing ids to intptr_t. So the libvqm changes look good; go for them.

The other changes outside libvqm look limited and reasonble. So if you can get CI green, and confirm manually that vpr builds & functions on windows then we can merge this.

Longer term, we should get Windows builds going on Travis CI. If you also want to take a crack at that feel free, but antmicro will take putting Windows builds into CI if you get them working, so you don't have to do that if you don't know travis setup.

This is required because it is not guaranteed that stdint.h will be
in scope by the C++ compiler by the time the parser data types are
included. The parser union, in particular, needs uintptr_t.
@cr1901
Copy link
Contributor Author

cr1901 commented Nov 8, 2020

Results of running the quickstart on Windows match the expected output, so I think VPR is working fine. Linux compile works fine now too. Now waiting on CI...

@vaughnbetz
Copy link
Contributor

Looks good except you need to run 'make format' to format the code to specification, and then check in the updated files (it'll only be the ones you changed). That's the only CI failure I see.

@cr1901
Copy link
Contributor Author

cr1901 commented Nov 8, 2020

@vaughnbetz I only have clang-format for Clang 10 installed; the build system hardcodes clang-format-7. Running clang-format manually, there are a few more files touched besides the one I modified, but not many. Is this okay? It's kinda difficult for me to install Clang 7.

Copy link
Contributor

@mithro mithro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cr1901 - You should be able to grab the required patch out of the Travis CI output?

@cr1901
Copy link
Contributor Author

cr1901 commented Nov 9, 2020

Clearly, there's something different about clang-format 10's defaults that breaks the vtr build :(.

For future reference, could someone with access to clang-format-7 run the following command at the root of the vtr source directory and paste the output? clang-format appears to be smart enough to search for .clang-format when dumping its config.

clang-format-7 --dump-config > config-7.cfg

EDIT: @mithro You are correct, there is a diff. I somehow managed to miss that even after your comment :P!

@vaughnbetz
Copy link
Contributor

CI looks stuck; forcing a kokoro rerun.

@cr1901
Copy link
Contributor Author

cr1901 commented Nov 18, 2020

CI continues to look stuck :(...

@mithro
Copy link
Contributor

mithro commented Nov 18, 2020

I kicked the kokoro build off again.

@vaughnbetz
Copy link
Contributor

Kokoro still has no output ... strange. Seems to be working on other PRs.

Copy link
Contributor

@mithro mithro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is only the nightly which appears to fail. I say we merge and then revert if it causes master to go red?

@vaughnbetz vaughnbetz merged commit e4266ef into verilog-to-routing:master Nov 19, 2020
@vaughnbetz
Copy link
Contributor

OK, done. Here's hoping!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

VTR cannot build on Cygwin due to 'posix_memalign' not declared
4 participants