Skip to content

Docs: policy for supported tools and dependencies #7859

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 16 commits into from
Closed

Docs: policy for supported tools and dependencies #7859

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
6fdf230
Docs: policy for supported tools and dependencies
stsewd Jan 22, 2021
d454376
Update
stsewd Jan 25, 2021
be3b0b7
More updates
stsewd Jan 25, 2021
664cbb1
Match dates
stsewd Jan 25, 2021
0d94c45
Match dates from .com
stsewd Jan 25, 2021
cb313e6
Apply suggestions from code review
stsewd Jan 26, 2021
88bf264
Updates
stsewd Jan 26, 2021
8b1e92a
Remove some sentences
stsewd Jan 26, 2021
7b6a848
More updates
stsewd Jan 26, 2021
23fe002
Fix linter
stsewd Jan 26, 2021
35d36cc
Merge branch 'master' into supported-versions
stsewd Mar 22, 2021
d111c6d
Small updates
stsewd Mar 22, 2021
e1aac11
Merge branch 'master' into supported-versions
stsewd Apr 6, 2021
94feeb6
Small updates
stsewd Apr 6, 2021
a2bb332
Update docs/supported-tools.rst
stsewd Apr 6, 2021
bf2cc6d
Link to the repo of rtd-ext
stsewd Apr 6, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
#

# You can set these variables from the command line.
SPHINXOPTS = -W
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
Expand Down Expand Up @@ -63,7 +63,7 @@ html:
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."

livehtml:
sphinx-autobuild -p 4444 -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
sphinx-autobuild --port 4444 -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html


dirhtml:
Expand Down
2 changes: 2 additions & 0 deletions docs/index.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,7 @@ of Read the Docs and the larger software documentation ecosystem.
:doc:`Terms of service <terms-of-service>` |
:doc:`DMCA takedown policy <dmca/index>` |
:doc:`Policy for abandoned projects <abandoned-projects>` |
:doc:`/supported-tools` |
:doc:`Release notes & changelog <changelog>`

* **The people and philosophy behind Read the Docs**:
Expand Down Expand Up @@ -236,6 +237,7 @@ of Read the Docs and the larger software documentation ecosystem.
terms-of-service
dmca/index
abandoned-projects
supported-tools
changelog

about
Expand Down
246 changes: 246 additions & 0 deletions docs/supported-tools.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,246 @@
Supported Tools and Dependencies
================================

Read the Docs supports two tools to build your documentation:
`Sphinx <https://www.sphinx-doc.org/>`__ and `MkDocs <https://www.mkdocs.org/>`__.
In order to provide :doc:`several features </features>`,
Read the Docs needs to inject or modify some content while building your docs.

When an incompatible change happens in one of these tools or dependencies,
we need to change our code as well to keep our features working,
this is done with backwards compatibility in mind.
But sometimes is hard to keep compatibility with old and new versions at the same time,
in order to continue moving forward on future development and features we need to drop support for some versions.

.. note::

Your existent documentation will always be kept online and working,
But **when support for a tool or dependency ends, new builds may fail**.

.. contents:: Contents
:local:
:depth: 3

End of support policy
---------------------

Our policy defines how long a given tool or dependency is considered supported.
After it reaches its end of support date,
we don't offer support for builds/docs using these tools or dependencies,
you'll need to upgrade in order to receive support.
For :doc:`/commercial/index` we provide an extended support of six months after the official end of support date.

For tools that define an EOL date, we try to follow that date for our policy.
For tools that release their versions incrementally without an EOL date,
we choose a date based on: the release of mayor versions (versions with breaking changes),
the date since it was last updated, and its usage on our platform.

.. note::

Some recent versions may be supported, but aren't listed on these tables
(specially minor updates from supported versions),
contact us if you have doubts.

Default dependencies
--------------------

We install some dependencies in order to build your project without specifying their dependencies.
In the past we used to set these dependencies to a specific version and update them after some time,
but by doing so would break some builds and new projects wouldn't be able to use a new version by default.
For this reason we are now installing their latest version (or latest supported version) by default.

.. note::

In order to keep your builds reproducible,
it's highly recommended declaring its dependencies and versions explicitly.

.. TODO: link to this guide once it's written https://github.com/readthedocs/readthedocs.org/issues/7852.

External dependencies (Python)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

These are the dependencies that are installed by default when using a Python environment:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has any of this changed?


Sphinx:
Projects created before Oct 20, 2020 use ``1.8.x``.
New projects use the latest version.

Mkdocs:
Projects created before April 3, 2019 (April 23, 2019 for :doc:`/commercial/index`) use ``0.17.3``.
New projects use the latest version.

sphinx-rtd-theme:
Projects created before October 20, 2020 (January 21, 2021 for :doc:`/commercial/index`) use ``0.4.3``.
New projects use the latest version.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about not installing the theme in the future as well. Like, it's an external dependency, but it's a "core" product as well.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, not sure what is best here. I think we can probably just not install it and let users handle it?

Copy link

@Gallaecio Gallaecio Apr 7, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not installing it means fixing the “what happened?” issue when you find out that the rendering of your documentation locally does not match what Read The Docs shows.

However, it’s a great theme, and if it’s not installed by default, it will be somewhat harder for people to discover it, or at least it will become less popular due to people tending to leave whatever is default.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Gallaecio we are trying to do less "magic" for users in favor of being more explicit and having reproducible builds locally. In this case users would need to add the theme to their requirements, we would update or add guides to help users with this.


pip:
Latest version by default.

setuptools:
Latest version by default.

mock:
``1.0.1`` (could be removed in the future).

pillow:
``5.4.1`` (could be removed in the future).

alabaster:
``0.7.x`` (could be removed in the future).

commonmark:
``0.8.1`` (could be removed in the future).

recommonmark:
``0.5.0`` (could be removed in the future).
Comment on lines +87 to +100
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should put all these under a feature flag to stop installing them on new projects.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should remove all these optional ones at the same date for new projects.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we replace all occurrences of "could be removed in the future" with a set date? That will make this policy easier to publish.

Following this, there are definitely some action items to pick. For instance, adding warnings in build logs and contacting users directly. But I think it's fine to start with a nice coherent policy.


External dependencies (Conda)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

These are the dependencies that are installed by default when using a Conda environment:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has any of this changed?


Conda:
Miniconda2 ``4.6.14``
(could be updated to use the latest version by default).
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can start sampling more projects to use update_conda_at_startup, we have noticed that this solves some OOM errors, and it's faster :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. However, we have found also that sometimes a conda upgrade broke the build. So, there is some tradeoff here.

I think the best solution here is to continue the path towards mamba.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we should push to mamba as a replacement for conda, it should be an option. We already update pip on each build, if conda breaks is probably temporary and not so usual.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's true we update pip. However, pip is way more stable than conda in our experience 😄

Copy link
Member

@humitos humitos Apr 7, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another difference between upgrading pip and conda is that conda could take a lot of time to update.


Mkdocs:
Latest version by default installed via ``conda``.

Sphinx:
Latest version by default installed via ``conda``.

sphinx-rtd-theme:
Latest version by default installed via ``conda``.
Comment on lines +111 to +118
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may mention these come from conda-forge channel.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't set any explicit channel when adding these deps

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, we probably should then. The default channel usually has old versions. We will end up with different versions installed if you use pip than if you use conda.

For example, https://pypi.org/project/sphinx-rtd-theme/ latest version is 0.5.1 and we are currently installing 0.4.3 (https://readthedocs.org/projects/time-test/builds/12859408/)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure if we control on publishing to conda (either the default or forge), I think the safest way is to install via pip the theme at least.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have control on default but we do on conda-forge.

I'm not sure how safe it is to mix conda and pip in the same environment or if it's recommended or not. We may need to do some research here before installing a package by default using pip in a conda environment.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@astrojuanlu is it 100% safe to mix conda and pip in the same environment?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I understand, no it's not 100 % safe. However, I don't know if we are at 98 %, 99.9 %, or just 100 % and we can stop worrying about this.

I just did this quick test: I tried mamba install "sphinx<3.5" on a new conda environment and then pip install -U sphinx to see what would happen. Result: pip uninstalled the old Sphinx version (surprising! I think this could not be done in the past, perhaps it changed after conda/conda#7053 was closed), detected that the rest of the dependencies were in place (👍🏽), then pip-installed the newest Sphinx version, and finally it looks like sphinx-quickstart just works.

$ conda list | grep -i sphinx
sphinx                    3.5.3                    pypi_0    pypi
sphinxcontrib-applehelp   1.0.2                      py_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    1.0.3                      py_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.4                      py_0    conda-forge

I will ask in the conda-forge community.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://matrix.to/#/!dUtrSrxSudChYWSwjL:gitter.im/$XfuNmdhoZhl2SR9_5Xx8189DjIjciL2mITUlUvYQh1I?via=gitter.im&via=matrix.org for future reference

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It turns out I asked in the wrong room 🤦🏽 https://matrix.to/#/!SOyumkgPRWoXfQYIFH:matrix.org/$1619164399102681JsZJt:matrix.org?via=gitter.im&via=matrix.org


mock:
Latest version by default installed via ``pip`` (could be removed in the future).

pillow:
Latest version by default installed via ``pip`` (could be removed in the future).

recommonmark:
Latest version by default installed via ``conda`` (could be removed in the future).

Internal dependencies
~~~~~~~~~~~~~~~~~~~~~

Internal dependencies are needed to integrate your docs with Read the Docs.
We guarantee that these dependencies will work with all current supported versions of our tools,
you don't need to specify them in your requirements.

- readthedocs-sphinx-ext
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should link to the repo.


Table of supported versions
---------------------------

Sphinx
~~~~~~

Sphinx releases its versions incrementally.

.. list-table::
:header-rows: 1

* - Version
- Released / Latest Update
- Supported Until

* - ``3.x``
- Apr 5, 2020 / ???
- 5.0 is released or later/early
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does "or later/early" mean? It seems like we don't really know 😄

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a date that isn't decided yet, but are aiming to 5.0, but something can happen, and we should be ready to stop supporting this version or keep supporting it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be clarified in some manner in the policy.


* - ``2.x``
- Mar 28, 2019 / Mar 5, 2020
- 4.0 is released or later/early

* - ``1.8.x``
- Sep 12, 2018 / Mar 10, 2019
- Nov 31, 2022
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just noting for others: this date is dependent on deprecating Python 2.7 first. Sphinx>=2 requires Python>=3.5. A year seems like a good amount of lead time for projects to make the switch.


* - ``1.7.x``
- Feb 12, 2018 / Sep 5, 2018
- Nov 31, 2022
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these the .org or .com EOL? We should probably keep them the same, TBH, and just handle case-by-case users as we need to.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are .org. I'm fine having the same date. I was thinking more about support requests than compatibility support.


* - ``1.6.x``
- May 16, 2017 / Feb 4, 2017
- Nov 31, 2021

* - ``1.5.x``
- Dec 5, 2016 / May 4, 2017
- Nov 31, 2021

* - ``<= 1.4.x``
- Mar 21, 2008 / Nov 23, 2016
- Unsupported

Mkdocs
~~~~~~

MkDocs releases its versions incrementally.

.. list-table::
:header-rows: 1

* - Version
- Released / Latest Update
- Supported Until

* - ``1.1.x``
- Feb 22, 2020 / ???
- 3.0 released or later/early

* - ``1.0.x``
- Aug 3, 2018 / Sep 17, 2018
- 2.0 released or later/early

* - ``0.17.x``
- Oct 19, 2017 / Jul 6, 2018
- Nov 31, 2021

* - ``0.16.x``
- Nov 4, 2017 / Apr 4, 2017
- Nov 31, 2021

* - ``0.15.x``
- Jan 21 2016 / Feb 18, 2016
- Nov 31, 2021

* - ``<= 0.14.x``
- Jan 11, 2014 / Jun 9, 2015
- Unsupported

Python
~~~~~~

Python defines an EOL (End Of Life) date for all its versions.

.. list-table::
:header-rows: 1

* - Version
- EOL Date
- Supported Until

* - ``3.9.x`` (not available yet)
- Oct 05, 2025
- Jan 31, 2026

* - ``3.8.x``
- Oct 14, 2024
- Jan 31, 2025

* - ``3.7.x``
- Jun 27, 2023
- Sep 31, 2023

* - ``3.6.x``
- Dec 23, 2021
- Mar 31, 2022

* - ``3.5.x``
- Sep 13, 2020
- Nov 31, 2021

* - ``2.7.x``
- Jan 01, 2020
- Nov 31, 2021
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is overdue... but we didn't have this policy before... so

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also a huge change, and will likely still lead to a lot of yelling.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did this happen?