Skip to content

Add GitHub actions setup for CI instead of Travis & Appveyour (where possible) #470

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 43 commits into from
Apr 30, 2021
Merged
Show file tree
Hide file tree
Changes from 39 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
093137b
Add initial config for github actions
sergregory Apr 5, 2021
eb06701
Allow actions for all branches and PRs to master by default
sergregory Apr 5, 2021
69297d8
Fix typo
sergregory Apr 5, 2021
46a4ca9
Do not set docker image info for multibuild, as we do it in github ac…
sergregory Apr 5, 2021
db300f2
Do not run github action in docker as multiduild will do it for us
sergregory Apr 5, 2021
86b0ece
Remove error steps - somehow I've missed them when cleaned for the build
sergregory Apr 5, 2021
89082e5
Merge all build logic into one step as somehow build wheel is not kno…
sergregory Apr 5, 2021
4b1a401
Debug: try to change directory to get inside the repo sources
sergregory Apr 5, 2021
e550802
Adjust env variables to enable build
sergregory Apr 6, 2021
4a1452e
Specify docker image to use in github builds
sergregory Apr 6, 2021
a6c746d
Enable back all python versions in build matrix
sergregory Apr 6, 2021
940e5b1
Remove numpy as build dependency as prebuilt docker should handle it …
sergregory Apr 6, 2021
60e6a36
Make linux build matrix wider - enable contrib, headless/gui, source/…
sergregory Apr 7, 2021
5c33d11
Fix return code setting for skipped tests in case of sdist
sergregory Apr 7, 2021
219c4a1
Build source distribution with only one version of python - as we do …
sergregory Apr 7, 2021
665b09b
Try to fix config
sergregory Apr 7, 2021
4b6885d
Try to fix config
sergregory Apr 7, 2021
c1c51f8
Enable OSX builds
sergregory Apr 7, 2021
af32e40
Quickfix to enable OSX build
sergregory Apr 7, 2021
fbebaa6
Temporary disable linux and all pythons execpt one to reduce the buil…
sergregory Apr 7, 2021
a271af1
Disable cache stage that was implemented for travis
sergregory Apr 7, 2021
169df1f
Do not unlink python2 for github osx builds
sergregory Apr 7, 2021
d2e3540
Fix ffmpeg bottle build command
sergregory Apr 7, 2021
093968e
Disable travis cache stage
sergregory Apr 7, 2021
db31b23
Changed build_wheels to Windows platform
Apr 14, 2021
591c15a
Try new pipeline
asenyaev Apr 16, 2021
f56fb78
Try windows with a new configuration
asenyaev Apr 16, 2021
a1be420
Try bash on Windows
asenyaev Apr 16, 2021
834e9b3
Added ubuntu and macos into pipeline
asenyaev Apr 16, 2021
8a4a967
Fix if condition for windows
asenyaev Apr 16, 2021
c7db704
Turned on all python versions, uploading artifacts
asenyaev Apr 16, 2021
bfa06a9
Merge pull request #4 from asenyaev/asen/windows_python
sergregory Apr 16, 2021
b3b8d09
Fixed Windows and created Release job
asenyaev Apr 21, 2021
e411b08
Merge pull request #5 from asenyaev/asen/fix_release
sergregory Apr 22, 2021
b8b409f
Add missed source distribution build
sergregory Apr 22, 2021
d512b45
Move x86 python builds to github actions. Travis does only aarch64 now
sergregory Apr 22, 2021
176a99b
Split linux and OSX builds to exclude x86 OSX builds from build matrix
sergregory Apr 22, 2021
aceb2a3
Reduce architectures to x64 for both OSX and Ubuntu as x86 is support…
sergregory Apr 22, 2021
d376197
Add PyPI secrets usage
sergregory Apr 23, 2021
f316fb8
Publish packages built in PRs to testPyPI
sergregory Apr 28, 2021
7483e1b
Push to test PyPI on each package build. Remove unneded build on push…
sergregory Apr 29, 2021
377e3cf
Apply review comments
sergregory Apr 29, 2021
6945db0
Combine all release steps into one job
sergregory Apr 30, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
366 changes: 366 additions & 0 deletions .github/workflows/build_wheels.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,366 @@
name: Build PYPI wheels for opencv-python

on:
push:
branches:
- all
pull_request:
branches:
- master
release:
types: [published, edited]


jobs:
build-windows-x86_64:
runs-on: ${{ matrix.os }}
defaults:
run:
shell: powershell

strategy:
fail-fast: false
matrix:
os: [windows-latest]
python-version: [3.6, 3.7, 3.8, 3.9]
platform: [x86, x64]
with_contrib: [0, 1]
without_gui: [0, 1]
build_sdist: [0]

env:
ACTIONS_ALLOW_UNSECURE_COMMANDS: true
SDIST: ${{ matrix.build_sdist || 0}}
ENABLE_HEADLESS: ${{ matrix.without_gui }}
ENABLE_CONTRIB: ${{ matrix.with_contrib }}

steps:
- name: Checkout
uses: actions/checkout@v2
with:
submodules: true
fetch-depth: 0

- name: Update submodules
run: |
git submodule update --remote

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
architecture: ${{ matrix.platform }}

- name: Setup MSBuild.exe
uses: warrenbuckley/Setup-MSBuild@v1

- name: build script
run: |
python --version
python -m pip install --upgrade pip
python -m pip install --upgrade setuptools
set "CI_BUILD=1" && python -m pip wheel --wheel-dir=%cd%\wheelhouse . --verbose
shell: cmd

- name: before test
run: |
cd ${{ github.workspace }}/tests
$env:PYTHONWARNINGS = "ignore:::pip._internal.cli.base_command"
&python -m pip install --user --no-warn-script-location (ls "../wheelhouse/opencv*.whl")
if ($LastExitCode -ne 0) {throw $LastExitCode}
shell: powershell

- name: run test
run: |
cd ${{ github.workspace }}/tests
python -m unittest test
shell: cmd

- name: saving artifacts
uses: actions/upload-artifact@v2
with:
name: wheels
path: wheelhouse/opencv*.whl

build:
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash

strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest]
python-version: [3.6, 3.7, 3.8, 3.9]
platform: [x64]
with_contrib: [0, 1]
without_gui: [0, 1]
build_sdist: [0]

env:
ACTIONS_ALLOW_UNSECURE_COMMANDS: true
REPO_DIR: .
BUILD_COMMIT: master
PROJECT_SPEC: opencv-python
MB_PYTHON_VERSION: ${{ matrix.python-version }}
TRAVIS_PYTHON_VERSION: ${{ matrix.python-version }}
MB_ML_VER: 2014
NP_TEST_DEP: numpy
TRAVIS_BUILD_DIR: ${{ github.workspace }}
CONFIG_PATH: travis_config.sh
DOCKER_IMAGE: quay.io/skvark/manylinux2014_${PLAT}
USE_CCACHE: 1
UNICODE_WIDTH: 32
SDIST: ${{ matrix.build_sdist || 0}}
ENABLE_HEADLESS: ${{ matrix.without_gui }}
ENABLE_CONTRIB: ${{ matrix.with_contrib }}

steps:
- name: Checkout
uses: actions/checkout@v2
with:
submodules: true
fetch-depth: 0

- name: Update submodules
run: |
git submodule update --remote

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
architecture: ${{ matrix.platform }}

- name: Setup Environment variables
run: |
if [ "macos-latest" == "${{ matrix.os }}" ]; then echo "TRAVIS_OS_NAME=osx" >> $GITHUB_ENV; else echo "TRAVIS_OS_NAME=${{ matrix.os }}" >> $GITHUB_ENV; fi
if [ "schedule" == "${{ github.event_name }}" ]; then echo "TRAVIS_EVENT_TYPE=cron" >> $GITHUB_ENV; else echo "TRAVIS_EVENT_TYPE=${{ github.event_name }}" >> $GITHUB_ENV; fi
if [ "schedule" == "${{ github.event_name }}" ]; then echo "BUILD_COMMIT=master" >> $GITHUB_ENV; else echo "BUILD_COMMIT=$BUILD_COMMIT" >> $GITHUB_ENV; fi
if [ "x64" == "${{ matrix.platform }}" ]; then echo "PLAT=x86_64" >> $GITHUB_ENV; fi
if [ "x86" == "${{ matrix.platform }}" ]; then echo "PLAT=i686" >> $GITHUB_ENV; fi
echo "BUILD_DEPENDS=$(echo $NP_BUILD_DEP)" >> $GITHUB_ENV;
echo "TEST_DEPENDS=$(echo $NP_TEST_DEP)" >> $GITHUB_ENV;

- name: before install
run: |
set -e
if [[ $SDIST == 0 ]]; then
# Check out and prepare the source
# Multibuild doesn't have releases, so --depth would break eventually (see
# https://superuser.com/questions/1240216/server-does-not-allow-request-for-unadvertised)
git submodule update --init multibuild
source multibuild/common_utils.sh
# https://github.com/matthew-brett/multibuild/issues/116
if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then export ARCH_FLAGS=" "; fi
source multibuild/travis_steps.sh
# This sets -x
# source travis_multibuild_customize.sh
echo $ENABLE_CONTRIB > contrib.enabled
echo $ENABLE_HEADLESS > headless.enabled
echo "end"
# Not interested in travis internal scripts' output
fi
set +x
# Build and package
set -x
ls
if [[ $SDIST == 1 ]]; then
python -m pip install --upgrade pip
python -m pip install scikit-build
python setup.py sdist
else
build_wheel $REPO_DIR $PLAT
fi
set +x
# Install and run tests
set -x
if [[ $SDIST == 1 ]]; then
echo "skipping tests because of sdist"
rc=0
else
install_run $PLAT && rc=$? || rc=$?
fi
set +x
#otherwise, Travis logic terminates prematurely
#https://travis-ci.community/t/shell-session-update-command-not-found-in-build-log-causes-build-to-fail-if-trap-err-is-set/817
trap ERR
test "$rc" -eq 0
- name: saving artifacts
uses: actions/upload-artifact@v2
with:
name: wheels
path: wheelhouse/opencv*.whl


build_sdist:
runs-on: ${{ matrix.os }}
defaults:
run:
shell: bash

strategy:
fail-fast: false
matrix:
os: [ubuntu-latest]
python-version: [3.8]
platform: [x64]
with_contrib: [0, 1]
without_gui: [0, 1]
build_sdist: [1]

env:
ACTIONS_ALLOW_UNSECURE_COMMANDS: true
REPO_DIR: .
BUILD_COMMIT: master
PROJECT_SPEC: opencv-python
PLAT: x86_64
MB_PYTHON_VERSION: ${{ matrix.python-version }}
TRAVIS_PYTHON_VERSION: ${{ matrix.python-version }}
MB_ML_VER: 2014
NP_TEST_DEP: numpy
TRAVIS_BUILD_DIR: ${{ github.workspace }}
CONFIG_PATH: travis_config.sh
DOCKER_IMAGE: quay.io/skvark/manylinux2014_${PLAT}
USE_CCACHE: 1
UNICODE_WIDTH: 32
SDIST: ${{ matrix.build_sdist || 0}}
ENABLE_HEADLESS: ${{ matrix.without_gui || 0 }}
ENABLE_CONTRIB: ${{ matrix.with_contrib || 0}}

steps:
- name: Checkout
uses: actions/checkout@v2
with:
submodules: true
fetch-depth: 0

- name: Update submodules
run: |
git submodule update --remote

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: ${{ matrix.python-version }}
architecture: ${{ matrix.platform }}

- name: Setup Environment variables
run: |
if [ "macos-latest" == "${{ matrix.os }}" ]; then echo "TRAVIS_OS_NAME=osx" >> $GITHUB_ENV; else echo "TRAVIS_OS_NAME=${{ matrix.os }}" >> $GITHUB_ENV; fi
if [ "schedule" == "${{ github.event_name }}" ]; then echo "TRAVIS_EVENT_TYPE=cron" >> $GITHUB_ENV; else echo "TRAVIS_EVENT_TYPE=${{ github.event_name }}" >> $GITHUB_ENV; fi
if [ "schedule" == "${{ github.event_name }}" ]; then echo "BUILD_COMMIT=master" >> $GITHUB_ENV; else echo "BUILD_COMMIT=$BUILD_COMMIT" >> $GITHUB_ENV; fi
echo "BUILD_DEPENDS=$(echo $NP_BUILD_DEP)" >> $GITHUB_ENV;
echo "TEST_DEPENDS=$(echo $NP_TEST_DEP)" >> $GITHUB_ENV;

- name: before install
run: |
set -e
# Build and package
set -x
python -m pip install --upgrade pip
python -m pip install scikit-build
python setup.py sdist
set +x
# Install and run tests
set -x
echo "skipping tests because of sdist"
- name: saving artifacts
uses: actions/upload-artifact@v2
with:
name: wheels
path: dist/opencv*.tar.gz


release_opencv_python:
if: startsWith(github.ref, 'refs/tags/v')
needs: [build, build-windows-x86_64, build_sdist]
runs-on: ubuntu-latest
environment: opencv-python-release
defaults:
run:
shell: bash
steps:
- uses: actions/download-artifact@v2
with:
name: wheels
path: wheelhouse/

- name: Upload wheels ${{ matrix.os }}
env:
# PYPI repository credentials
TWINE_USERNAME: ${{ secrets.OPENCV_PYTHON_USERNAME }}
TWINE_PASSWORD: ${{ secrets.OPENCV_PYTHON_PASSWORD }}
run: |
python -m pip install twine
python -m twine upload -u ${Env:TWINE_USERNAME} -p ${Env:TWINE_PASSWORD} --skip-existing wheelhouse/opencv_python-*

release_opencv_contrib_python:
if: startsWith(github.ref, 'refs/tags/v')
needs: [build, build-windows-x86_64, build_sdist]
runs-on: ubuntu-latest
environment: opencv-python-release
defaults:
run:
shell: bash
steps:
- uses: actions/download-artifact@v2
with:
name: wheels
path: wheelhouse/

- name: Upload wheels ${{ matrix.os }}
env:
# PYPI repository credentials
TWINE_USERNAME: ${{ secrets.OPENCV_CONTRIB_PYTHON_USERNAME }}
TWINE_PASSWORD: ${{ secrets.OPENCV_CONTRIB_PYTHON_PASSWORD }}
run: |
python -m pip install twine
python -m twine upload -u ${Env:TWINE_USERNAME} -p ${Env:TWINE_PASSWORD} --skip-existing wheelhouse/opencv_contrib_python-*

release_opencv_python_headless:
if: startsWith(github.ref, 'refs/tags/v')
needs: [build, build-windows-x86_64, build_sdist]
runs-on: ubuntu-latest
environment: opencv-python-release
defaults:
run:
shell: bash
steps:
- uses: actions/download-artifact@v2
with:
name: wheels
path: wheelhouse/

- name: Upload wheels ${{ matrix.os }}
env:
# PYPI repository credentials
TWINE_USERNAME: ${{ secrets.OPENCV_PYTHON_HEADLESS_USERNAME }}
TWINE_PASSWORD: ${{ secrets.OPENCV_PYTHON_HEADLESS_PASSWORD }}
run: |
python -m pip install twine
python -m twine upload -u ${Env:TWINE_USERNAME} -p ${Env:TWINE_PASSWORD} --skip-existing wheelhouse/opencv_python_headless-*

release_opencv_contrib_python_headless:
if: startsWith(github.ref, 'refs/tags/v')
needs: [build, build-windows-x86_64, build_sdist]
runs-on: ubuntu-latest
environment: opencv-python-release
defaults:
run:
shell: bash
steps:
- uses: actions/download-artifact@v2
with:
name: wheels
path: wheelhouse/

- name: Upload wheels ${{ matrix.os }}
env:
# PYPI repository credentials
TWINE_USERNAME: ${{ secrets.OPENCV_CONTRIB_PYTHON_HEADLESS_USERNAME }}
TWINE_PASSWORD: ${{ secrets.OPENCV_CONTRIB_PYTHON_HEADLESS_PASSWORD }}
run: |
python -m pip install twine
python -m twine upload -u ${Env:TWINE_USERNAME} -p ${Env:TWINE_PASSWORD} --skip-existing wheelhouse/opencv_contrib_python_headless-*
Loading