build-customization.rst

Build customization

Read the Docs has a :doc:`well-defined build process <builds>` that works for many projects, but we offer additional customization to support more uses of our platform. This page explains how to extend the build process using :term:`user-defined build jobs` to execute custom commands, and also how to override the build process completely:

Extend the build process

If you are using Sphinx or Mkdocs and need to execute additional commands.

Override the build process

If you want full control over your build. This option supports any tool that generates HTML as part of the build.

Table of contents

• Extend the build process
  • build.jobs examples
    • Unshallow clone
    • Cancel build based on a condition
    • Generate documentation from annotated sources with Doxygen
    • Use MkDocs extensions with extra required steps
    • Avoid having a dirty Git index
    • Perform a check for broken links
    • Support Git LFS (Large File Storage)
    • Install Node.js dependencies
    • Install dependencies with Poetry
• Override the build process
  • build.commands examples
    • Pelican
    • Docsify
  • Search support

Extend the build process

In the normal build process, the pre-defined jobs checkout, system_dependencies, create_environment, install, build and upload are executed. However, Read the Docs exposes extra jobs to users so they can customize the build process by running shell commands. These extra jobs are:

Step	Customizable jobs
Checkout	post_checkout
System dependencies	pre_system_dependencies, post_system_dependencies
Create environment	pre_create_environment, post_create_environment
Install	pre_install, post_install
Build	pre_build, post_build
Upload	There are no customizable jobs currently

Note

Currently, the pre-defined jobs (checkout, system_dependencies, etc) executed by Read the Docs cannot be overridden or skipped.

These jobs can be declared by using a :doc:`/config-file/index` with the :ref:`config-file/v2:build.jobs` key on it. Let's say the project requires commands to be executed before installing any dependency into the Python environment and after the build has finished. In that case, a config file similar to this one can be used:

version: 2
build:
  os: "ubuntu-20.04"
  tools:
    python: "3.10"
  jobs:
    pre_install:
      - bash ./scripts/pre_install.sh
    post_build:
      - curl -X POST \
        -F "project=${READTHEDOCS_PROJECT}" \
        -F "version=${READTHEDOCS_VERSION}" https://example.com/webhooks/readthedocs/

There are some caveats to knowing when using user-defined jobs:

• The current working directory is at the root of your project's cloned repository
• Environment variables are expanded in the commands (see :doc:`environment-variables`)
• Each command is executed in a new shell process, so modifications done to the shell environment do not persist between commands
• Any command returning non-zero exit code will cause the build to fail immediately (note there is a special exit code to cancel the build)
• build.os and build.tools are required when using build.jobs

build.jobs examples

We've included some common examples where using :ref:`config-file/v2:build.jobs` will be useful. These examples may require some adaptation for each projects' use case, we recommend you use them as a starting point.

Unshallow clone

Read the Docs does not perform a full clone on checkout job to reduce network data and speed up the build process. Because of this, extensions that depend on the full Git history will fail. To avoid this, it's possible to unshallow the clone done by Read the Docs:

version: 2
build:
  os: "ubuntu-20.04"
  tools:
    python: "3.10"
  jobs:
    post_checkout:
      - git fetch --unshallow

Cancel build based on a condition

When a command exits with code 183, Read the Docs will cancel the build immediately. You can use this approach to cancel builds that you don't want to complete based on some conditional logic.

Note

Why 183 was chosen for the exit code?

It's the word "skip" encoded in ASCII. Then it's taken the 256 modulo of it because the Unix implementation does this automatically for exit codes greater than 255.

>>> sum(list('skip'.encode('ascii')))
439
>>> 439 % 256
183

Here is an example that cancels builds from pull requests when there are no changes to the docs/ folder compared to the origin/main branch:

version: 2
build:
  os: "ubuntu-22.04"
  tools:
    python: "3.11"
  jobs:
    post_checkout:
      # Cancel building pull requests when there aren't changed in the docs directory.
      # `--quiet` exits with a 1 when there **are** changes,
      # so we invert the logic with a !
      #
      # If there are no changes (exit 0) we force the command to return with 183.
      # This is a special exit code on Read the Docs that will cancel the build immediately.
      - |
        if [ $READTHEDOCS_VERSION_TYPE = "external" ];
        then
          ! git diff --quiet origin/main -- docs/ && exit 183;
        fi

This other example shows how to cancel a build if the commit message contains skip ci on it:

version: 2
build:
  os: "ubuntu-22.04"
  tools:
    python: "3.11"
  jobs:
    post_checkout:
      # Use `git log` to check if the latest commit contains "skip ci",
      # in that case exit the command with 183 to cancel the build
      - (git --no-pager log --pretty="tformat:%s -- %b" -1 | grep -viq "skip ci") || exit 183

Generate documentation from annotated sources with Doxygen

It's possible to run Doxygen as part of the build process to generate documentation from annotated sources:

version: 2
build:
  os: "ubuntu-20.04"
  tools:
    python: "3.10"
  jobs:
    pre_build:
    # Note that this HTML won't be automatically uploaded,
    # unless your documentation build includes it somehow.
      - doxygen

Use MkDocs extensions with extra required steps

There are some MkDocs extensions that require specific commands to be run to generate extra pages before performing the build. For example, pydoc-markdown

version: 2
build:
  os: "ubuntu-20.04"
  tools:
    python: "3.10"
  jobs:
    pre_build:
      - pydoc-markdown --build --site-dir "$PWD/_build/html"

Avoid having a dirty Git index

Read the Docs needs to modify some files before performing the build to be able to integrate with some of its features. Because of this reason, it could happen the Git index gets dirty (it will detect modified files). In case this happens and the project is using any kind of extension that generates a version based on Git metadata (like setuptools_scm), this could cause an invalid version number to be generated. In that case, the Git index can be updated to ignore the files that Read the Docs has modified.

version: 2
build:
  os: "ubuntu-20.04"
  tools:
    python: "3.10"
  jobs:
    pre_install:
      - git update-index --assume-unchanged environment.yml docs/conf.py

Perform a check for broken links

Sphinx comes with a linkcheck builder that checks for broken external links included in the project's documentation. This helps ensure that all external links are still valid and readers aren't linked to non-existent pages.

version: 2
build:
  os: "ubuntu-20.04"
  tools:
    python: "3.10"
  jobs:
    pre_build:
      - python -m sphinx -b linkcheck docs/ _build/linkcheck

Support Git LFS (Large File Storage)

In case the repository contains large files that are tracked with Git LFS, there are some extra steps required to be able to download their content. It's possible to use post_checkout user-defined job for this.

version: 2
build:
  os: "ubuntu-20.04"
  tools:
    python: "3.10"
  jobs:
    post_checkout:
      # Download and uncompress the binary
      # https://git-lfs.github.com/
      - wget https://github.com/git-lfs/git-lfs/releases/download/v3.1.4/git-lfs-linux-amd64-v3.1.4.tar.gz
      - tar xvfz git-lfs-linux-amd64-v3.1.4.tar.gz
      # Modify LFS config paths to point where git-lfs binary was downloaded
      - git config filter.lfs.process "`pwd`/git-lfs filter-process"
      - git config filter.lfs.smudge  "`pwd`/git-lfs smudge -- %f"
      - git config filter.lfs.clean "`pwd`/git-lfs clean -- %f"
      # Make LFS available in current repository
      - ./git-lfs install
      # Download content from remote
      - ./git-lfs fetch
      # Make local files to have the real content on them
      - ./git-lfs checkout

Install Node.js dependencies

It's possible to install Node.js together with the required dependencies by using :term:`user-defined build jobs`. To setup it, you need to define the version of Node.js to use and install the dependencies by using build.jobs.post_install:

version: 2
build:
  os: "ubuntu-22.04"
  tools:
    python: "3.9"
    nodejs: "16"
  jobs:
    post_install:
      # Install dependencies defined in your ``package.json``
      - npm ci
      # Install any other extra dependencies to build the docs
      - npm install -g jsdoc

Install dependencies with Poetry

Projects managed with Poetry, can use the post_create_environment user-defined job to use Poetry for installing Python dependencies. Take a look at the following example:

version: 2

build:
  os: "ubuntu-22.04"
  tools:
    python: "3.10"
  jobs:
    post_create_environment:
      # Install poetry
      # https://python-poetry.org/docs/#osx--linux--bashonwindows-install-instructions
      - curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
      # Tell poetry to not use a virtual environment
      - $HOME/.poetry/bin/poetry config virtualenvs.create false
      # Install project's dependencies
      - $HOME/.poetry/bin/poetry install

sphinx:
  configuration: docs/conf.py

Override the build process

Warning

This feature is in a beta phase and could suffer incompatible changes or even removed completely in the near feature. It does not yet support some of the Read the Docs' features like the :term:`flyout menu`, and ads. However, we do plan to support these features in the future. Use this feature at your own risk.

If your project requires full control of the build process, and :ref:`extending the build process <build-customization:extend the build process>` is not enough, all the commands executed during builds can be overridden using the :ref:`config-file/v2:build.commands` configuration file key.

As Read the Docs does not have control over the build process, you are responsible for running all the commands required to install requirements and build your project properly. Once the build process finishes, the contents of the _readthedocs/html/ directory will be hosted.

build.commands examples

This section contains some examples that showcase what is possible with :ref:`config-file/v2:build.commands`. Note that you may need to modify and adapt these examples depending on your needs.

Pelican

Pelican is a well-known static site generator that's commonly used for blogs and landing pages. If you are building your project with Pelican you could use a configuration file similar to the following:

version: 2
build:
  os: "ubuntu-22.04"
  tools:
    python: "3.10"
  commands:
    - pip install pelican[markdown]
    - pelican --settings docs/pelicanconf.py --output _readthedocs/html/ docs/

Docsify

Docsify generates documentation websites on the fly, without the need to build static HTML. These projects can be built using a configuration file like this:

version: 2
build:
  os: "ubuntu-22.04"
  tools:
    nodejs: "16"
  commands:
    - mkdir --parents _readthedocs/html/
    - cp --recursive docs/* _readthedocs/html/

Search support

Read the Docs will automatically index the content of all your HTML files, respecting the :ref:`search <config-file/v2:search>` options from your config file.

You can access the search results from the :guilabel:`Search` tab of your project, or by using the :doc:`/server-side-search/api`.

Note

In order for Read the Docs to index your HTML files correctly, they should follow some of the conventions described at :doc:`rtd-dev:search-integration`.