Skip to content

Commit cd498b5

Browse files
authored
Build: install all the latest Python "core requirements" (#10508)
* Build: install all the latest Python "core requirements" I created a new feature flag called `INSTALL_LATEST_CORE_REQUIREMENTS` that does not depend on any other feature flag. Its goal is to always install the latest Python packages. We are trying to move away from pinning the dependencies on our side and telling users to do that on their side if they want to have reproducible builds over time. I think this is the best outcome for new projects since they will immediately use the latest versions of everything instead of old (and maybe broken) dependencies. I propose to set a particular date for this feature flag and make new projects to start building with all the latest dependencies. This will, at least, stop making the problem bigger. Later, we can decide how to communicate to old projects that we are going to install the latest requirements and if they don't want that, they should pin their dependencies. We can follow our new and shiny deprecation path we have built and tested in the last month. Related readthedocs/meta#8 Related #9562 Related #10402 Related #9081 * Build: remove deprecated dependencies * Build: install latest core requirements when using Conda * Docs: update default versions documentation to match the changes * Feedback from review Do not install `sphinx-rtd-theme` and clarify the docstring. * Update documentation
1 parent c5ce36e commit cd498b5

File tree

3 files changed

+102
-12
lines changed

3 files changed

+102
-12
lines changed

docs/user/build-default-versions.rst

+20-9
Original file line numberDiff line numberDiff line change
@@ -46,29 +46,36 @@ Mkdocs:
4646
sphinx-rtd-theme:
4747
Projects created before October 20, 2020 (January 21, 2021 for :doc:`/commercial/index`) use ``0.4.3``.
4848
New projects use the latest version.
49+
Projects created after August 7, 2023 won't install this dependency by default.
4950

5051
pip:
5152
Latest version by default.
5253

5354
setuptools:
5455
Projects using ``setup.py install`` as installation method use ``58.2.0`` or older.
5556
All other projects use the latest version.
57+
Projects created after August 7, 2023 will always use the latest version.
5658

5759
mock:
58-
``1.0.1`` (could be removed in the future).
60+
``1.0.1``.
61+
Projects created after August 7, 2023 won't install this dependency by default.
62+
5963

6064
pillow:
61-
``5.4.1`` when using Python 2.7, 3.4, 3.5, 3.6, 3.7. Otherwise, its latest version
62-
(could be removed in the future).
65+
``5.4.1`` when using Python 2.7, 3.4, 3.5, 3.6, 3.7. Otherwise, its latest version.
66+
Projects created after August 7, 2023 won't install this dependency by default.
6367

6468
alabaster:
65-
``0.7.x`` (could be removed in the future).
69+
``0.7.x``.
70+
Projects created after August 7, 2023 won't install this dependency by default.
6671

6772
commonmark:
68-
``0.8.1`` (could be removed in the future).
73+
``0.8.1``.
74+
Projects created after August 7, 2023 won't install this dependency by default.
6975

7076
recommonmark:
71-
``0.5.0`` (could be removed in the future).
77+
``0.5.0``.
78+
Projects created after August 7, 2023 won't install this dependency by default.
7279

7380
Conda
7481
~~~~~
@@ -87,15 +94,19 @@ Sphinx:
8794

8895
sphinx-rtd-theme:
8996
Latest version by default installed via ``conda``.
97+
Projects created after August 7, 2023 won't install this dependency by default.
9098

9199
mock:
92-
Latest version by default installed via ``pip`` (could be removed in the future).
100+
Latest version by default installed via ``pip``.
101+
Projects created after August 7, 2023 won't install this dependency by default.
93102

94103
pillow:
95-
Latest version by default installed via ``pip`` (could be removed in the future).
104+
Latest version by default installed via ``pip``.
105+
Projects created after August 7, 2023 won't install this dependency by default.
96106

97107
recommonmark:
98-
Latest version by default installed via ``conda`` (could be removed in the future).
108+
Latest version by default installed via ``conda``.
109+
Projects created after August 7, 2023 won't install this dependency by default.
99110

100111
Internal dependencies
101112
---------------------

readthedocs/doc_builder/python_environments.py

+75-3
Original file line numberDiff line numberDiff line change
@@ -163,6 +163,60 @@ def install_core_requirements(self):
163163
'--no-cache-dir',
164164
]
165165

166+
if self.project.has_feature(Feature.INSTALL_LATEST_CORE_REQUIREMENTS):
167+
self._install_latest_requirements(pip_install_cmd)
168+
else:
169+
self._install_old_requirements(pip_install_cmd)
170+
171+
def _install_latest_requirements(self, pip_install_cmd):
172+
"""
173+
Install all the latest core requirements.
174+
175+
By enabling the feature flag ``INSTALL_LATEST_CORE_REQUIREMENTS``
176+
projects will automatically get installed all the latest core
177+
requirements: pip, setuptools, sphinx, readthedocs-sphinx-ext and mkdocs.
178+
179+
This is the new behavior and where we are moving towards.
180+
"""
181+
# First, upgrade pip and setuptools to their latest versions
182+
cmd = pip_install_cmd + ["pip", "setuptools"]
183+
self.build_env.run(
184+
*cmd,
185+
bin_path=self.venv_bin(),
186+
cwd=self.checkout_path,
187+
)
188+
189+
# Second, install all the latest core requirements
190+
requirements = []
191+
192+
if self.config.doctype == "mkdocs":
193+
requirements.append("mkdocs")
194+
else:
195+
requirements.extend(
196+
[
197+
"sphinx",
198+
"readthedocs-sphinx-ext",
199+
]
200+
)
201+
202+
cmd = copy.copy(pip_install_cmd)
203+
cmd.extend(requirements)
204+
self.build_env.run(
205+
*cmd,
206+
bin_path=self.venv_bin(),
207+
cwd=self.checkout_path,
208+
)
209+
210+
def _install_old_requirements(self, pip_install_cmd):
211+
"""
212+
Install old core requirements.
213+
214+
There are bunch of feature flags that will be taken in consideration to
215+
decide whether or not upgrade some of the core dependencies to their
216+
latest versions.
217+
218+
This is the old behavior and the one we want to get rid off.
219+
"""
166220
# Install latest pip and setuptools first,
167221
# so it is used when installing the other requirements.
168222
pip_version = self.project.get_feature_value(
@@ -416,7 +470,10 @@ def _append_core_requirements(self):
416470
else:
417471
# Append conda dependencies directly to ``dependencies`` and pip
418472
# dependencies to ``dependencies.pip``
419-
pip_requirements, conda_requirements = self._get_core_requirements()
473+
if self.project.has_feature(Feature.INSTALL_LATEST_CORE_REQUIREMENTS):
474+
pip_requirements, conda_requirements = self._get_new_core_requirements()
475+
else:
476+
pip_requirements, conda_requirements = self._get_old_core_requirements()
420477
dependencies = environment.get('dependencies', [])
421478
pip_dependencies = {'pip': pip_requirements}
422479

@@ -454,7 +511,22 @@ def _append_core_requirements(self):
454511
'environment file.',
455512
)
456513

457-
def _get_core_requirements(self):
514+
def _get_new_core_requirements(self):
515+
# Use conda for requirements it packages
516+
conda_requirements = []
517+
518+
# Install pip-only things.
519+
pip_requirements = []
520+
521+
if self.config.doctype == "mkdocs":
522+
pip_requirements.append("mkdocs")
523+
else:
524+
pip_requirements.append("readthedocs-sphinx-ext")
525+
conda_requirements.extend(["sphinx"])
526+
527+
return pip_requirements, conda_requirements
528+
529+
def _get_old_core_requirements(self):
458530
# Use conda for requirements it packages
459531
conda_requirements = [
460532
'mock',
@@ -483,7 +555,7 @@ def install_core_requirements(self):
483555
# create`` step.
484556
return
485557

486-
pip_requirements, conda_requirements = self._get_core_requirements()
558+
pip_requirements, conda_requirements = self._get_old_core_requirements()
487559
# Install requirements via ``conda install`` command if they were
488560
# not appended to the ``environment.yml`` file.
489561
cmd = [

readthedocs/projects/models.py

+7
Original file line numberDiff line numberDiff line change
@@ -1938,6 +1938,7 @@ def add_features(sender, **kwargs):
19381938
USE_SPHINX_LATEST = 'use_sphinx_latest'
19391939
DEFAULT_TO_MKDOCS_0_17_3 = 'default_to_mkdocs_0_17_3'
19401940
USE_SPHINX_RTD_EXT_LATEST = 'rtd_sphinx_ext_latest'
1941+
INSTALL_LATEST_CORE_REQUIREMENTS = "install_latest_core_requirements"
19411942

19421943
# Search related features
19431944
DISABLE_SERVER_SIDE_SEARCH = 'disable_server_side_search'
@@ -2049,6 +2050,12 @@ def add_features(sender, **kwargs):
20492050
USE_SPHINX_RTD_EXT_LATEST,
20502051
_("Sphinx: Use latest version of the Read the Docs Sphinx extension"),
20512052
),
2053+
(
2054+
INSTALL_LATEST_CORE_REQUIREMENTS,
2055+
_(
2056+
"Build: Install all the latest versions of Read the Docs core requirements"
2057+
),
2058+
),
20522059

20532060
# Search related features.
20542061
(

0 commit comments

Comments
 (0)