Docs: feature documentation for build.jobs

.readthedocs.yml

@@ -14,9 +14,10 @@ build:
   os: ubuntu-20.04
   tools:
     python: "3.8"
-  jobs:
-    pre_build:
-      - sphinx-build -W --keep-going -q -b linkcheck -d _build/doctrees docs/ _build/linkcheck
+  # TODO: uncomment when it's fixed and does not fail anymore
+  # jobs:
+  #   pre_build:
+  #     - sphinx-build -W --keep-going -q -b linkcheck -d _build/doctrees docs/ _build/linkcheck
 
 search:
   ranking:

docs/user/build-customization.rst

@@ -0,0 +1,155 @@
+Build customization
+===================
+
+Read the Docs has a :ref:`well-defined build process <builds>` that may not be enough for all the use cases.
+This page explains how to customize Read the Docs build process to get the most of it and support custom build processes.
+
+In the normal build process the jobs ``checkout``, ``system_dependencies``, ``create_environment``, ``install``, ``build`` and ``upload`` are executed.
+However, Read the Docs exposes in-between jobs to users so they can customize the build process by running shell commands.
+These in-between jobs are:
+
+:post_checkout:
+
+:pre_system_dependencies:
+
+:post_system_dependencies:
+
+:pre_create_environment:
+
+:post_create_environment:
+
+:pre_install:
+
+:post_install:
+
+:pre_build:
+
+:post_build:
+
+
+These jobs can be declared by using a :ref:`config-file/index` with the :ref:`config-file/v2:build.jobs` key on it.
+Let's say the project requires a mandatory command to be executed *before* installing any dependency into the Python environment and *after* the build has finished.
+In that case, a config file similar to this one can be used:
+
+.. code:: yaml
+
+   version: 2
+   build:
+     os: "ubuntu-20.04"
+     tools:
+       python: "3.10"
+     jobs:
+       pre_install:
+         - echo "Executing pre_install step..."
+         - bash ./scripts/pre_install.sh
+       post_build:
+         - echo "Executing post_build step..."
+         - curl -X POST \
+           -F "project=${READTHEDOCS_PROJECT}" \
+           -F "version=${READTHEDOCS_VERSION}" https://my.company.com/webhooks/readthedocs/
+
+.. note::
+
+   Currently, the default jobs (e.g. ``checkout``, ``system_dependencies``, etc) executed by Read the Docs are not possible to overwritte or skip.
+
+
+Examples
+--------
+
+This is a non-exahustive list of common examples where using :ref:`config-file/v2:build.jobs` will be useful.
+Also note that these examples may require some adaption for each projects' use case.
+They are listed here just as a guide.
+
+
+Unshallow clone
+~~~~~~~~~~~~~~~
+
+Read the Docs does not perform a full clone on ``checkout`` step to reduce network data and speed up the build process.
+Because of this reason, if there are extensions in the build process that depend on the full history, they will fail.
+To avoid this, it's possible to unshallow the clone done by Read the Docs:
+
+.. code:: yaml
+
+   version: 2
+   build:
+     os: "ubuntu-20.04"
+     tools:
+       python: "3.10"
+     jobs:
+       post_checkout:
+         - git fetch --unshallow
+
+
+Generate documentation from annotated sources with Doxygen
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It's possible to run Doxygen as part of the build process to generate documentation from annotated sources:
+
+.. code:: yaml
+
+   version: 2
+   build:
+     os: "ubuntu-20.04"
+     tools:
+       python: "3.10"
+     jobs:
+       pre_build:
+         - doxygen
+
+
+Use MkDocs extensions with extra required steps
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+There are some MkDocs extensions that require specific commands to be run to generate extra pages before performing the build.
+For example, `pydoc-markdown <http://niklasrosenstein.github.io/pydoc-markdown/>`_
+
+.. code:: yaml
+
+   version: 2
+   build:
+     os: "ubuntu-20.04"
+     tools:
+       python: "3.10"
+     jobs:
+       pre_build:
+         - pydoc-markdown --build --site-dir "$PWD/_build/html"
+
+
+Avoid having a dirty ``git`` index
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Read the Docs needs to modify some files before performing the build to be able to integrate with some of its features.
+Because of this reason, it could happen the git index gets dirty (e.i. it will detect modified files).
+In case this happens and the project is using any kind of extension that generates a version based on git metadata (like `setuptools_scm <https://github.com/pypa/setuptools_scm/>`_),
+this could case a miss generated version number.
+In that case, the git index can be updated to ignore the files that Read the Docs has modified.
+
+.. code:: yaml
+
+   version: 2
+   build:
+     os: "ubuntu-20.04"
+     tools:
+       python: "3.10"
+     jobs:
+       pre_install:
+         - git update-index --assume-unchanged environment.yml docs/conf.py
+
+
+Perform a check for broken links
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Sphinx comes with a `linkcheck <https://www.sphinx-doc.org/en/master/usage/builders/index.html#sphinx.builders.linkcheck.CheckExternalLinksBuilder>`_ builder that checks for broken external links included in the project's documentation.
+This could be a good addition to the build process to be sure that all the external links are valid over time and readers don't find a dead end when clicking on them.
+
+
+.. code:: yaml
+
+   version: 2
+   build:
+     os: "ubuntu-20.04"
+     tools:
+       python: "3.10"
+     jobs:
+       pre_build:
+         - python -m sphinx -b linkcheck docs/ _build/linkcheck

docs/user/build-default-versions.rst

@@ -0,0 +1,106 @@
+:orphan:
+
+
+.. This page contains good detailed content about the exact versions Read the
+   Docs is install by default, but I don't think it's good content as a
+   user-facing documentation page. However, I'm keeping it around and linking it
+   from the "Build process" page when mentioning there are some dependencies
+   installed by default.
+
+Default versions of dependencies
+================================
+
+Read the Docs supports two tools to build your documentation:
+`Sphinx <https://www.sphinx-doc.org/>`__ and `MkDocs <https://www.mkdocs.org/>`__.
+In order to provide :doc:`several features </features>`,
+Read the Docs injects or modifies some content while building your docs.
+
+In particular, if you don't specify the dependencies of your project,
+we install some of them on your behalf.
+In the past we used to pin these dependencies to a specific version and update them after some time,
+but doing so would break some builds and make it more difficult for new projects to use new versions.
+For this reason, we are now installing their latest version by default.
+
+.. note::
+
+   In order to keep your builds reproducible,
+   it's highly recommended declaring its dependencies and versions explicitly.
+   See :doc:`/guides/reproducible-builds`.
+
+External dependencies
+---------------------
+
+Python
+~~~~~~
+
+These are the dependencies that are installed by default when using a Python environment:
+
+Sphinx:
+  Projects created before Oct 20, 2020 use ``1.8.x``.
+  New projects use the latest version.
+
+Mkdocs:
+  Projects created before April 3, 2019 (April 23, 2019 for :doc:`/commercial/index`) use ``0.17.3``.
+  Projects created before March 9, 2021 use ``1.0.4``.
+  New projects use the latest version.
+
+sphinx-rtd-theme:
+  Projects created before October 20, 2020 (January 21, 2021 for :doc:`/commercial/index`) use ``0.4.3``.
+  New projects use the latest version.
+
+pip:
+  Latest version by default.
+
+setuptools:
+  ``58.2.0`` or older.
+
+mock:
+  ``1.0.1`` (could be removed in the future).
+
+pillow:
+  ``5.4.1`` (could be removed in the future).
+
+alabaster:
+  ``0.7.x`` (could be removed in the future).
+
+commonmark:
+  ``0.8.1`` (could be removed in the future).
+
+recommonmark:
+  ``0.5.0`` (could be removed in the future).
+
+Conda
+~~~~~
+
+These are the dependencies that are installed by default when using a Conda environment:
+
+Conda:
+   Miniconda2 ``4.6.14``
+   (could be updated in the future to use the latest version by default).
+
+Mkdocs:
+  Latest version by default installed via ``conda``.
+
+Sphinx:
+  Latest version by default installed via ``conda``.
+
+sphinx-rtd-theme:
+  Latest version by default installed via ``conda``.
+
+mock:
+  Latest version by default installed via ``pip`` (could be removed in the future).
+
+pillow:
+  Latest version by default installed via ``pip`` (could be removed in the future).
+
+recommonmark:
+  Latest version by default installed via ``conda`` (could be removed in the future).
+
+Internal dependencies
+---------------------
+
+Internal dependencies are needed to integrate your docs with Read the Docs.
+We guarantee that these dependencies will work with all current supported versions of our tools,
+you don't need to specify them in your requirements.
+
+- `readthedocs-sphinx-ext <https://github.com/readthedocs/readthedocs-sphinx-ext>`__