Manual build using provided Docker images fails

[PavloFesenko]

Expected behaviour

I'm trying to build opencv-python-headless for Python 3.10 with only several modules (core, imgproc, videoio) using the provided Docker image with OpenCV dependencies, and using the provided instructions for manual builds.

I spin up the following Docker container quay.io/opencv-ci/opencv-python-manylinux2014-x86-64:20231006 and execute the following code in the container shell:

cd ~
git clone --recursive https://github.com/opencv/opencv-python.git
cd ~/opencv-python
git checkout -b 78
export CMAKE_ARGS="-DBUILD_LIST=core,imgproc,videoio" ENABLE_HEADLESS=1
python3.10 -m pip wheel . --verbose > out.txt 2>&1

Actual behaviour

The build fails with the error below and the full output of pip wheel . --verbose can be viewed in out.txt.

  Copying files from CMake output
  Traceback (most recent call last):
    File "/opt/_internal/cpython-3.10.13/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/opt/_internal/cpython-3.10.13/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/opt/_internal/cpython-3.10.13/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 251, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/tmp/pip-build-env-v332t_75/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 230, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "/tmp/pip-build-env-v332t_75/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir
      self.run_setup()
    File "/tmp/pip-build-env-v332t_75/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 267, in run_setup
      super(_BuildMetaLegacyBackend,
    File "/tmp/pip-build-env-v332t_75/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 158, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 535, in <module>
      main()
    File "setup.py", line 267, in main
      skbuild.setup(
    File "/tmp/pip-build-env-v332t_75/overlay/lib/python3.10/site-packages/skbuild/setuptools_wrap.py", line 706, in setup
      _classify_installed_files(
    File "setup.py", line 409, in _classify_installed_files_override
      with open(config_py, 'w') as opencv_init_config:
  FileNotFoundError: [Errno 2] No such file or directory: '_skbuild/linux-x86_64-3.10/cmake-install/python/cv2/config-3.py'
  error: subprocess-exited-with-error
  
  × Building wheel for opencv-python-headless (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.

I also tried adding -DPYTHON3_LIMITED_API=ON to CMAKE_ARGS but the result is the same.

And I noticed that there is no folder python in _skbuild/linux-x86_64-3.10/cmake-install, let alone any config files.

Steps to reproduce

• see the code above
• Windows 10 using Docker Desktop 4.25.0
• manylinux2014-x86-64
• opencv-python-headless 4.8.1.78

Since the error is related to scikit-build, maybe @henryiii could also help? Thanks a lot to everyone! 🙏

Issue submission checklist

• This is not a generic OpenCV usage question (looking for help for coding, other usage questions, homework etc.)
• I have read the README of this repository and understand that this repository provides only an automated build toolchain for OpenCV Python packages (there is no actual OpenCV code here)
• The issue is related to the build scripts in this repository, to the pre-built binaries or is a feature request (such as "please enable this additional dependency")
• I'm using the latest version of opencv-python

[asmorkalov]

From log:

  --   Python 3:
  --     Interpreter:                 /usr/local/bin/python3.10 (ver 3.10.13)
  --     Libraries:                   NO
  --     numpy:                       /tmp/pip-build-env-v332t_75/overlay/lib/python3.10/site-packages/numpy/core/include (ver 1.21.2)
  --     install path:                -

it means that python3-dev package was not found by some reason. Most probably, you need to set some extra environment variables to fix it. I'll take a look on CI setup and return back.

[PavloFesenko]

I have managed to build a wheel without errors by using an extended set of modules CMAKE_ARGS="-DBUILD_LIST=core,dnn,features2d,flann,gapi,imgcodecs,imgproc,python3,videoio" since some of them depend on each other and somehow this dependencies aren't resolved automatically. If some of these modules aren't explicitly mentioned, I get Exception: Not found: 'python/cv2/py.typed' which is related to missing type stubs, for example:

  /home/ci/opencv-python/opencv/modules/python/src2/typing_stubs_generator.py:52: UserWarning: Typing stubs generation has failed.
  Traceback (most recent call last):
    File "/home/ci/opencv-python/opencv/modules/python/src2/typing_stubs_generation/nodes/type_node.py", line 291, in resolve
      self.value.resolve(root)
    File "/home/ci/opencv-python/opencv/modules/python/src2/typing_stubs_generation/nodes/type_node.py", line 583, in resolve
      raise TypeResolutionError('Failed to resolve "{}" exposed as "{}"'.format(
  typing_stubs_generation.nodes.type_node.TypeResolutionError: Failed to resolve "Feature2D" exposed as "Feature2D"

Unfortunately, after installing the wheel (can be downloaded as opencv_python_headless-4.8.1+7cfd1ee-cp310-cp310-linux_x86_64.zip) and trying to import it in Python, I get the libopenblas error below although libopenblas was successfully detected during the wheel build as I'm using the provided Docker image quay.io/opencv-ci/opencv-python-manylinux2014-x86-64:20231006 (see the full build logs in out.txt):

>>> import cv2
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pavlo/miniforge3/envs/test/lib/python3.10/site-packages/cv2/__init__.py", line 181, in <module>
    bootstrap()
  File "/home/pavlo/miniforge3/envs/test/lib/python3.10/site-packages/cv2/__init__.py", line 153, in bootstrap
    native_module = importlib.import_module("cv2")
  File "/home/pavlo/miniforge3/envs/test/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: libopenblas.so.0: cannot open shared object file: No such file or directory

The check by auditwheel didn't reveal any problems but it also didn't mention libopenblas in its list of shared libraries. 🤔 @asmorkalov, @asenyaev, @skvark, any ideas what could be the problem?

opencv_python_headless-4.8.1+7cfd1ee-cp310-cp310-linux_x86_64.whl is
consistent with the following platform tag: "linux_x86_64".

The wheel references external versioned symbols in these
system-provided shared libraries: librt.so.1 with versions
{'GLIBC_2.2.5'}, libgcc_s.so.1 with versions {'GCC_3.4', 'GCC_3.0'},
libswscale.so.6 with versions {'LIBSWSCALE_6'}, libpthread.so.0 with
versions {'GLIBC_2.3.2', 'GLIBC_2.2.5'}, libz.so.1 with versions
{'ZLIB_1.2.3.4', 'ZLIB_1.2.0', 'ZLIB_1.2.9'}, libm.so.6 with versions
{'GLIBC_2.2.5', 'GLIBC_2.29'}, libc.so.6 with versions {'GLIBC_2.7',
'GLIBC_2.6', 'GLIBC_2.4', 'GLIBC_2.3', 'GLIBC_2.2.5', 'GLIBC_2.14',
'GLIBC_2.11', 'GLIBC_2.3.4'}, libavutil.so.57 with versions
{'LIBAVUTIL_57'}, libpng16.so.16 with versions {'PNG16_0'},
libavcodec.so.59 with versions {'LIBAVCODEC_59'}, libavformat.so.59
with versions {'LIBAVFORMAT_59'}, libdl.so.2 with versions
{'GLIBC_2.2.5'}, libstdc++.so.6 with versions {'GLIBCXX_3.4.14',
'CXXABI_1.3.5', 'GLIBCXX_3.4', 'CXXABI_1.3.3', 'GLIBCXX_3.4.18',
'GLIBCXX_3.4.19', 'GLIBCXX_3.4.17', 'GLIBCXX_3.4.15', 'GLIBCXX_3.4.9',
'GLIBCXX_3.4.13', 'GLIBCXX_3.4.11', 'CXXABI_1.3'}

This constrains the platform tag to "manylinux_2_31_x86_64". In order
to achieve a more compatible tag, you would need to recompile a new
wheel from source on a system with earlier versions of these
libraries, such as a recent manylinux image.