Use pyproject.toml as Python version source
#136
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Python and Python-based tools are used in the development and maintenance of the project. A standardized version of Python is used for these operations. Poetry is used for Python package dependencies management. Poetry installs dependencies into a virtual environment. For this reason, the Poetry configuration includes a Python version number, which must match the correct version of Python for the project. This configuration is stored in the `pyproject.toml` file. Python is installed in the GitHub Actions runner environments using the "actions/setup-python" action, which also must be configured to install the correct version of Python. Previously the version number for use by the actions/setup-python action was defined in each workflow. This meant that we had multiple copies of the Python version information, all of which had to be kept in sync. Fortunately, support for using `pyproject.toml` as the source of version information for the "actions/setup-python" action was recently added. This means it is now possible for all components of the project infrastructure to get the Python version from a single source.
The version of Python used by the project infrastructure is controlled via the pyproject.yaml metadata file. Previously
the file specified that any version >=3.9.0 and <4.0.0 could be used, since it had been validated with 3.9 and
theoretically no breaking changes should be introduced without a major version bump.
However, there was a breaking change in Python version 3.10, which caused an incompatibility with the version of the
project's pyyaml Python package dependency. This caused the `poetry install` command to fail when a version of Python
greater than 3.9.x was used:
```
Installing dependencies from lock file
Package operations: 4 installs, 0 updates, 0 removals
- Installing pathspec (0.9.0)
- Installing pyyaml (5.4.1)
PEP517 build of a dependency failed
Backend subprocess exited when trying to invoke get_requires_for_build_wheel
| Command '['/tmp/tmp4jjabujb/.venv/bin/python', '/opt/hostedtoolcache/Python/3.13.2/x64/lib/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py', 'get_requires_for_build_wheel', '/tmp/tmprk6qet9s']' returned non-zero exit status 1.
|
| running egg_info
| writing lib3/PyYAML.egg-info/PKG-INFO
| writing dependency_links to lib3/PyYAML.egg-info/dependency_links.txt
| writing top-level names to lib3/PyYAML.egg-info/top_level.txt
| Traceback (most recent call last):
| File "/opt/hostedtoolcache/Python/3.13.2/x64/lib/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
| main()
| ~~~~^^
| File "/opt/hostedtoolcache/Python/3.13.2/x64/lib/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in main
| json_out["return_val"] = hook(**hook_input["kwargs"])
| ~~~~^^^^^^^^^^^^^^^^^^^^^^^^
| File "/opt/hostedtoolcache/Python/3.13.2/x64/lib/python3.13/site-packages/pyproject_hooks/_in_process/_in_process.py", line 143, in get_requires_for_build_wheel
| return hook(config_settings)
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/build_meta.py", line 334, in get_requires_for_build_wheel
| return self._get_build_requires(config_settings, requirements=[])
| ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/build_meta.py", line 304, in _get_build_requires
| self.run_setup()
| ~~~~~~~~~~~~~~^^
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/build_meta.py", line 320, in run_setup
| exec(code, locals())
| ~~~~^^^^^^^^^^^^^^^^
| File "<string>", line 271, in <module>
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/__init__.py", line 117, in setup
| return distutils.core.setup(**attrs)
| ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/_distutils/core.py", line 186, in setup
| return run_commands(dist)
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
| dist.run_commands()
| ~~~~~~~~~~~~~~~~~^^
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 1002, in run_commands
| self.run_command(cmd)
| ~~~~~~~~~~~~~~~~^^^^^
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/dist.py", line 999, in run_command
| super().run_command(command)
| ~~~~~~~~~~~~~~~~~~~^^^^^^^^^
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/_distutils/dist.py", line 1021, in run_command
| cmd_obj.run()
| ~~~~~~~~~~~^^
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/command/egg_info.py", line 312, in run
| self.find_sources()
| ~~~~~~~~~~~~~~~~~^^
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/command/egg_info.py", line 320, in find_sources
| mm.run()
| ~~~~~~^^
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/command/egg_info.py", line 543, in run
| self.add_defaults()
| ~~~~~~~~~~~~~~~~~^^
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/command/egg_info.py", line 581, in add_defaults
| sdist.add_defaults(self)
| ~~~~~~~~~~~~~~~~~~^^^^^^
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/command/sdist.py", line 109, in add_defaults
| super().add_defaults()
| ~~~~~~~~~~~~~~~~~~~~^^
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/_distutils/command/sdist.py", line 245, in add_defaults
| self._add_defaults_ext()
| ~~~~~~~~~~~~~~~~~~~~~~^^
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/_distutils/command/sdist.py", line 330, in _add_defaults_ext
| self.filelist.extend(build_ext.get_source_files())
| ~~~~~~~~~~~~~~~~~~~~~~~~~~^^
| File "<string>", line 201, in get_source_files
| File "/tmp/tmp4jjabujb/.venv/lib/python3.13/site-packages/setuptools/_distutils/cmd.py", line 131, in __getattr__
| raise AttributeError(attr)
| AttributeError: cython_sources
Note: This error originates from the build backend, and is likely not a problem with poetry but one of the following issues with pyyaml (5.4.1)
- not supporting PEP 517 builds
- not specifying PEP 517 build requirements correctly
- the build requirements are incompatible with your operating system or Python version
- the build requirements are missing system dependencies (eg: compilers, libraries, headers).
You can verify this by running pip wheel --no-cache-dir --use-pep517 "pyyaml (==5.4.1)".
exit status 1
```
Although the incompatibility has already been fixed in pyyaml, this is a transitive dependency and so it might be
difficult to update the version used by the project.
So the chosen solution is to adjust the Python version constraints to require a version less than 3.10.0 be used. This
will still allow use of any Python version in the 3.9.x series to make the infrastructure more friendly to local use by
contributors, while ensuring that a compatible version is used.
per1234
added
type: enhancement
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Python and Python-based tools are used in the development and maintenance of the project. A standardized version of Python is used for these operations.
Poetry is used for Python package dependencies management. Poetry installs dependencies into a virtual environment. For this reason, the Poetry configuration includes a Python version number, which must match the correct version of Python for the project. This configuration is stored in the
pyproject.tomlfile.Python is installed in the GitHub Actions runner environments using the actions/setup-python action, which also must be configured to install the correct version of Python. Previously the version number for use by the actions/setup-python action was defined in each workflow. This meant that we had multiple copies of the Python version information, all of which had to be kept in sync.
Fortunately, support for using
pyproject.tomlas the source of version information for the actions/setup-python action was recently added (actions/setup-python@0d5da6a). This means it is now possible for all components of the project infrastructure to get the Python version from a single source.