Add support for building aarch64 wheel #108
Conversation
…nly if it applies
jcfr
force-pushed
the
add-support-for-building-aarch64-wheel
branch
from
4508e1d to
6655a4a
Compare
scikit-ci.yml
Outdated
| @@ -26,6 +26,9 @@ before_install: | |||
| # SETUP_CMAKE_ARGS | |||
| if arch in ["x86", "x64"]: | |||
| os.environ["SETUP_CMAKE_ARGS"] = "-DOPENSSL_ROOT_DIR:PATH=/usr/local/ssl " + os.environ["SETUP_CMAKE_ARGS"] | |||
| # SETUP_BDIST_WHEEL_ARGS | |||
| if arch == "aarch64": | |||
| os.environ["SETUP_BDIST_WHEEL_ARGS"] = "--plat-name %s" % image_name | |||
There was a problem hiding this comment.
Once dockcross/dockcross#420 is integrated, this should be changed to:
os.environ["SETUP_BDIST_WHEEL_ARGS"] = "--plat-name %s" % os.environ["AUDITWHEEL_PLAT"]
There was a problem hiding this comment.
@jcfr, dockcross/dockcross#420 PR is merged now.
There was a problem hiding this comment.
Great, thanks for the follow up.
The next issue to address is having openssl libraries and header compiled for aarch64, this will address this error:
/work/src/Utilities/cmcurl/lib/md5.c:88:25: fatal error: openssl/md5.h: No such file or directory
#include <openssl/md5.h>
^
Since the dockcross image host is x86_64 and the toolchain aarch64, I do not think we can install aarch64 dependencies using the host the yum package manager, we have to compile them
There was a problem hiding this comment.
I also wonder if we can download relevant openssl packages from http://mirror.centos.org/centos/8/BaseOS/aarch64/os/Packages/ and manually extract them ?
There was a problem hiding this comment.
For normal cross-compilation of openssl, I am using below code:
export CROSS_COMPILE=aarch64-linux-gnu- && \
export PATH=$PATH:/tmp/toolchain/gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu/bin && \
cd /tmp/ && \
git clone https://github.com/openssl/openssl && \
cd openssl && \
./Configure linux-aarch64 --cross-compile-prefix=${CROSS_COMPILE} --prefix=/tmp/openssl/build shared && \
make depend && \
make -j5 && \
make install && \
cp -r /tmp/openssl/build/include/* /tmp/toolchain/gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu/aarch64-linux-gnu/libc/usr/include/ && \
cp -r /tmp/openssl/build/lib/* /tmp/toolchain/gcc-arm-8.3-2019.03-x86_64-aarch64-linux-gnu/lib/gcc/aarch64-linux-gnu/8.3.0/
In dockcross, I think we need to update CROSS_COMPILE=aarch64-unknown-linux-gnueabi- and copy libraries to /usr/xcc/aarch64-unknown-linux-gnueabi/aarch64-unknown-linux-gnueabi/lib folder.
Maybe we can copy from http://mirror.centos.org/centos/8/BaseOS/aarch64/os/Packages , but need to check on what all packages are required.
There was a problem hiding this comment.
Thanks for the follow up 🙏
In dockcross, I think we need to update CROSS_COMPILE=aarch64-unknown-linux-gnueabi-
Could you check that OpenSSL can be compiled in dockcross ? Once we have the receipt, it will be easy to finalize this PR.
There was a problem hiding this comment.
@jcfr ,
I am able to successfully build openssl in dockcross/manylinux2014-aarch64 docker container by following below steps:
yum install perl-IPC-Cmd.noarch perl-IPC-Run.noarch perl-IPC-Run3.noarch
git clone https://github.com/openssl/openssl && \
cd openssl
./Configure linux-aarch64 --cross-compile-prefix= --prefix=/tmp/openssl/build shared
make depend
make -j5
make install
cp -r /tmp/openssl/build/lib/* /usr/xcc/aarch64-unknown-linux-gnueabi/aarch64-unknown-linux-gnueabi/lib64/
cp -r /tmp/openssl/build/include/* /usr/xcc/aarch64-unknown-linux-gnueabi/aarch64-unknown-linux-gnueabi/include/
Please let me know if any more info is required.
There was a problem hiding this comment.
Thanks for the update, looking at it now
|
Remaining error: Traceback (most recent call last):
File "/opt/python/cp37-cp37m/bin/auditwheel", line 8, in <module>
sys.exit(main())
File "/opt/_internal/cpython-3.7.7/lib/python3.7/site-packages/auditwheel/main.py", line 35, in main
main_repair.configure_parser(sub_parsers)
File "/opt/_internal/cpython-3.7.7/lib/python3.7/site-packages/auditwheel/main_repair.py", line 25, in configure_parser
default=highest_policy)
File "/opt/_internal/cpython-3.7.7/lib/python3.7/argparse.py", line 1359, in add_argument
action = action_class(**kwargs)
File "/opt/_internal/cpython-3.7.7/lib/python3.7/site-packages/auditwheel/tools.py", line 126, in __init__
raise argparse.ArgumentError(self, msg % args)
argparse.ArgumentError: argument --plat: invalid choice: 'manylinux2014_aarch64' from environment variable 'AUDITWHEEL_PLAT' (choose from 'linux_x86_64', 'manylinux1_x86_64', 'manylinux2010_x86_64', 'manylinux2014_x86_64')Issue has been discussed in pypa/auditwheel#244 (comment) Summary is:
For the sake of building the cmake wheel, I will implement a workaround. That said, we should look into addressing this and ensure architecture and processor associated with |
jcfr
force-pushed
the
add-support-for-building-aarch64-wheel
branch
from
3996f94 to
f8b2d6e
Compare
See #96 Co-authored-by: odidev <[email protected]>
This commit explicit sets the path to the strip executable to ensure the cmake aarch64 executable can be stripped. See #96 Co-authored-by: odidev <[email protected]>
This commit allows the build to progress using images missing: - fix for sudo (see dockcross/dockcross#421) - auditwheel environment variables (see dockcross/dockcross#420)
Since the wheel is properly named by default and CMake doesn't have any external dependencies beside of openssl, this commit skips the repair.
jcfr
force-pushed
the
add-support-for-building-aarch64-wheel
branch
from
1ecd483 to
59e9f9d
Compare
|
@odidev Et voila, wheel for aarch64 is now available on pypi:
|
|
@jcfr , Thanks for uploading cmake aarch64 wheel. |
No description provided.