diff --git a/docs/user/build-customization.rst b/docs/user/build-customization.rst index e31abe48d48..fbd2812df25 100644 --- a/docs/user/build-customization.rst +++ b/docs/user/build-customization.rst @@ -1,30 +1,30 @@ -Build customization -=================== +Build process customization +=========================== -Read the Docs has a :doc:`well-defined build process ` that works for many projects, -but we offer additional customization to support more uses of our platform. -This page explains how to extend the build process using :term:`user-defined build jobs` to execute custom commands, -and also how to override the build process completely: +Read the Docs has a :doc:`well-defined build process ` that works for many projects. +We also allow customization of builds in primary ways: `Extend the build process`_ - If you are using Sphinx or Mkdocs and need to execute additional commands. + Keep using the default build process, + adding your own commands. `Override the build process`_ - If you want full control over your build. This option supports any tool that generates HTML as part of the build. - -.. contents:: Table of contents - :local: + This option gives you *full control* over your build. + Read the Docs supports any tool that generates HTML. Extend the build process ------------------------ In the normal build process, the pre-defined jobs ``checkout``, ``system_dependencies``, ``create_environment``, ``install``, ``build`` and ``upload`` are executed. -However, Read the Docs exposes extra jobs to users so they can customize the build process by running shell commands. -These extra jobs are: +Read the Docs also exposes these jobs, +which allows you to customize the build process by adding shell commands. + +The jobs where users can customize our default build process are: .. list-table:: :header-rows: 1 + :widths: 25 75 * - Step - Customizable jobs @@ -39,23 +39,22 @@ These extra jobs are: * - Build - ``pre_build``, ``post_build`` * - Upload - - There are no customizable jobs currently + - No customizable jobs currently .. note:: - Currently, the pre-defined jobs (``checkout``, ``system_dependencies``, etc) executed by Read the Docs cannot be overridden or skipped. - + The pre-defined jobs (``checkout``, ``system_dependencies``, etc) cannot be overridden or skipped. + You can fully customize things in :ref:`build-customization:override the build process`. -These jobs can be declared by using a :doc:`/config-file/index` with the :ref:`config-file/v2:build.jobs` key on it. -Let's say the project requires commands to be executed *before* installing any dependency into the Python environment and *after* the build has finished. -In that case, a config file similar to this one can be used: +These jobs are defined using the :doc:`/config-file/v2` with the :ref:`config-file/v2:build.jobs` key. +This example configuration defines commands to be executed *before* installing and *after* the build has finished: .. code-block:: yaml :caption: .readthedocs.yaml version: 2 build: - os: "ubuntu-20.04" + os: "ubuntu-22.04" tools: python: "3.10" jobs: @@ -67,7 +66,8 @@ In that case, a config file similar to this one can be used: -F "version=${READTHEDOCS_VERSION}" https://example.com/webhooks/readthedocs/ -There are some caveats to knowing when using user-defined jobs: +User-defined job limitations +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * The current working directory is at the root of your project's cloned repository * Environment variables are expanded for each individual command (see :doc:`/reference/environment-variables`) @@ -78,19 +78,19 @@ There are some caveats to knowing when using user-defined jobs: ``build.jobs`` examples -+++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~ We've included some common examples where using :ref:`config-file/v2:build.jobs` will be useful. These examples may require some adaptation for each projects' use case, we recommend you use them as a starting point. -Unshallow clone -~~~~~~~~~~~~~~~ +Unshallow git clone +^^^^^^^^^^^^^^^^^^^ Read the Docs does not perform a full clone on ``checkout`` job to reduce network data and speed up the build process. Because of this, extensions that depend on the full Git history will fail. -To avoid this, it's possible to unshallow the clone done by Read the Docs: +To avoid this, it's possible to unshallow the :program:`git clone`: .. code-block:: yaml :caption: .readthedocs.yaml @@ -106,7 +106,7 @@ To avoid this, it's possible to unshallow the clone done by Read the Docs: Cancel build based on a condition -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When a command exits with code ``183``, Read the Docs will cancel the build immediately. @@ -170,7 +170,7 @@ This other example shows how to cancel a build if the commit message contains `` Generate documentation from annotated sources with Doxygen -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ It's possible to run Doxygen as part of the build process to generate documentation from annotated sources: @@ -190,7 +190,7 @@ It's possible to run Doxygen as part of the build process to generate documentat Use MkDocs extensions with extra required steps -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ There are some MkDocs extensions that require specific commands to be run to generate extra pages before performing the build. For example, `pydoc-markdown `_ @@ -209,7 +209,7 @@ For example, `pydoc-markdown Avoid having a dirty Git index -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Read the Docs needs to modify some files before performing the build to be able to integrate with some of its features. Because of this reason, it could happen the Git index gets dirty (it will detect modified files). @@ -231,7 +231,7 @@ In that case, the Git index can be updated to ignore the files that Read the Doc Perform a check for broken links -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Sphinx comes with a `linkcheck `_ builder that checks for broken external links included in the project's documentation. This helps ensure that all external links are still valid and readers aren't linked to non-existent pages. @@ -251,7 +251,7 @@ This helps ensure that all external links are still valid and readers aren't lin Support Git LFS (Large File Storage) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In case the repository contains large files that are tracked with Git LFS, there are some extra steps required to be able to download their content. @@ -284,7 +284,7 @@ It's possible to use ``post_checkout`` user-defined job for this. Install Node.js dependencies -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ It's possible to install Node.js together with the required dependencies by using :term:`user-defined build jobs`. To setup it, you need to define the version of Node.js to use and install the dependencies by using ``build.jobs.post_install``: @@ -307,7 +307,7 @@ To setup it, you need to define the version of Node.js to use and install the de Install dependencies with Poetry -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Projects managed with `Poetry `__, can use the ``post_create_environment`` user-defined job to use Poetry for installing Python dependencies. @@ -344,30 +344,53 @@ Override the build process .. warning:: - This feature is in a *beta phase* and could suffer incompatible changes or even removed completely in the near feature. - It does not yet support some of the Read the Docs' features like the :term:`flyout menu`, and ads. - However, we do plan to support these features in the future. - Use this feature at your own risk. - + This feature is in *beta* and could change without warning. + It does not yet support some of the Read the Docs' features like the :term:`flyout menu`. + We do our best to not break existing configurations, + but use this feature at your own risk. If your project requires full control of the build process, and :ref:`extending the build process ` is not enough, -all the commands executed during builds can be overridden using the :ref:`config-file/v2:build.commands` configuration file key. +all the commands executed during builds can be overridden using the :ref:`config-file/v2:build.commands`. As Read the Docs does not have control over the build process, -you are responsible for running all the commands required to install requirements and build your project properly. -Once the build process finishes, the contents of the ``_readthedocs/html/`` directory will be hosted. +you are responsible for running all the commands required to install requirements and build your project. + +Where to put files +~~~~~~~~~~~~~~~~~~ +It is your responsibility to generate HTML and other formats of your documentation using :ref:`config-file/v2:build.commands`. +The contents of the ``_readthedocs//`` directory will be hosted as part of your documentation. + +Supported :ref:`formats ` are published if they exist in the following directories: + +* ``_readthedocs/html/`` (required) +* ``_readthedocs/htmlzip/`` +* ``_readthedocs/pdf/`` +* ``_readthedocs/epub/`` + +Search support +~~~~~~~~~~~~~~ + +Read the Docs will automatically index the content of all your HTML files, +respecting the :ref:`search ` option. + +You can access the search from the Read the Docs :term:`dashboard`, +or by using the :doc:`/server-side-search/api`. + +.. note:: + + In order for Read the Docs to index your HTML files correctly, + they should follow the conventions described at :doc:`rtd-dev:search-integration`. ``build.commands`` examples -+++++++++++++++++++++++++++ +~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This section contains some examples that showcase what is possible with :ref:`config-file/v2:build.commands`. +This section contains examples that showcase what is possible with :ref:`config-file/v2:build.commands`. Note that you may need to modify and adapt these examples depending on your needs. - Pelican -~~~~~~~ +^^^^^^^ `Pelican `__ is a well-known static site generator that's commonly used for blogs and landing pages. If you are building your project with Pelican you could use a configuration file similar to the following: @@ -386,7 +409,7 @@ If you are building your project with Pelican you could use a configuration file Docsify -~~~~~~~ +^^^^^^^ `Docsify `__ generates documentation websites on the fly, without the need to build static HTML. These projects can be built using a configuration file like this: @@ -402,19 +425,3 @@ These projects can be built using a configuration file like this: commands: - mkdir --parents _readthedocs/html/ - cp --recursive docs/* _readthedocs/html/ - - -Search support -++++++++++++++ - -Read the Docs will automatically index the content of all your HTML files, -respecting the :ref:`search ` options from your config file. - -You can access the search results from the :guilabel:`Search` tab of your project, -or by using the :doc:`/server-side-search/api`. - -.. note:: - - In order for Read the Docs to index your HTML files correctly, - they should follow some of the conventions described - at :doc:`rtd-dev:search-integration`. diff --git a/docs/user/builds.rst b/docs/user/builds.rst index 72344111c63..746c39df68d 100644 --- a/docs/user/builds.rst +++ b/docs/user/builds.rst @@ -1,47 +1,49 @@ -Build process -============= +Build process overview +====================== Once a project has been imported and a build is triggered, -Read the Docs executes specific :term:`pre-defined jobs ` to build the project's documentation and update the hosted content. +Read the Docs executes a set of :term:`pre-defined jobs ` to build and upload documentation. This page explains in detail what happens behind the scenes, -and an overview of how you can change this process. +and includes an overview of how you can change this process. +Understanding the build process +------------------------------- -Understanding what's going on ------------------------------ - -Understanding how your content is built helps with debugging the problems that may appear in the process. -It also allows you customize the steps of the build process. +Understanding how your content is built helps with debugging any problems you might hit. +It also gives you the knowledge to customize the build process. .. note:: - All the steps are run inside a Docker container with the image the project defines in :ref:`config-file/v2:build.os`, - and all the :doc:`pre-defined environment variables ` and :doc:`custom environment variables ` defined are exposed to them. + All the steps are run inside a Docker container, using the image defined in :ref:`config-file/v2:build.os`. + The build has access to all :doc:`pre-defined environment variables ` and :doc:`custom environment variables `. -The following are the pre-defined jobs executed by Read the Docs: +The build process includes the following jobs: :checkout: - Checks out project's code from the URL's repository defined for this project. - It will use `git clone`, `hg clone`, etc depending on the version control system you choose. + Checks out a project's code from the repository URL. + On |com_brand|, + this environment includes the SSH deploy key that gives access to the repository. :system_dependencies: - Installs operating system & system-level dependencies. - This includes specific version of languages (e.g. Python, Node.js, Go, Rust) and also ``apt`` packages. + Installs operating system and runtime dependencies. + This includes specific versions of a language (e.g. Python, Node.js, Go, Rust) and also ``apt`` packages. - At this point, :ref:`config-file/v2:build.tools` can be used to define a language version, + :ref:`config-file/v2:build.tools` can be used to define a language version, and :ref:`config-file/v2:build.apt_packages` to define ``apt`` packages. :create_environment: Creates a Python environment to install all the dependencies in an isolated and reproducible way. - Depending on what's defined by the project a virtualenv or a conda environment (:ref:`config-file/v2:conda`) will be used. + Depending on what's defined by the project, + a virtualenv or a conda environment (:ref:`config-file/v2:conda`) will be used. :install: - Install :doc:`default common dependencies `. + Install :doc:`default and project dependencies `. + This includes any requirements you have configured in :ref:`config-file/v2:requirements file`. If the project has extra Python requirements, :ref:`config-file/v2:python.install` can be used to specify them. @@ -58,28 +60,25 @@ The following are the pre-defined jobs executed by Read the Docs: :upload: Once the build process finishes successfully, - the resulting artifacts are uploaded to our servers, and the CDN is purged so the newer version of the documentation is served. + the resulting artifacts are uploaded to our servers. + Our :doc:`CDN ` is then purged so your docs are *always up to date*. .. seealso:: - If there are extra steps required to build the documentation, - or you need to execute additional commands to integrate with other tools, - it's possible to run user-defined commands and :doc:`customize the build process `. - + If you require additional build steps or customization, + it's possible to run user-defined commands and :doc:`customize the build process `. -When to cancel builds ---------------------- +Cancelling Builds +----------------- -There may be situations where you want to cancel a particular running build. -Cancelling running builds will allow your team to speed up review times and also help us reduce server costs and ultimately, -our environmental footprint. +There may be situations where you want to cancel a running build. +Cancelling builds allows your team to speed up review times and also help us reduce server costs and our environmental footprint. -Consider the following scenarios: +A couple common reasons you might want to cancel builds are: * the build has an external dependency that hasn't been updated * there were no changes on the documentation files -* many other use cases that can be solved with custom logic For these scenarios, Read the Docs supports three different mechanisms to cancel a running build: @@ -88,20 +87,21 @@ Read the Docs supports three different mechanisms to cancel a running build: Once a build was triggered, project administrators can go to the build detail page - and click the button "Cancel build". + and click :guilabel:`Cancel build`. :Automatically: - When Read the Docs detects a push to a branch that it's currently building the documentation, - it cancels the running build and start a new build using the latest commit from the new push. + When Read the Docs detects a push to a version that is already building, + it cancels the running build and starts a new build using the latest commit. :Programatically: You can use user-defined commands on ``build.jobs`` or ``build.commands`` (see :doc:`build-customization`) - to check for a condition and exit it with the code ``183`` if you want to cancel the running build or ``0``, otherwise. + to check for your own cancellation condition and then return exit code ``183`` to cancel a build. + You can exit with the code ``0`` to continue running the build. - In this case, Read the Docs will communicate to your Git platform (GitHub/GitLab) that the build succeeded (green tick ✅) - so the pull request is in a mergeable state. + When this happens, Read the Docs will notify your Git platform (GitHub/GitLab) that the build succeeded (✅), + so the pull request doesn't have any failing checks. .. tip:: @@ -111,11 +111,23 @@ Read the Docs supports three different mechanisms to cancel a running build: Build resources --------------- -Every build has limited resources to avoid misuse of the platform. -Currently, these build limits are: +Every build has limited resources assigned to it. +Generally, |com_brand| users get double the build resources, +with the option to increase that. + +Our build limits are: .. tabs:: + .. tab:: |com_brand| + + * 30 minutes build time + * 7GB of memory + * Concurrent builds vary based on your pricing plan + + If you are having trouble with your documentation builds, + you can reach our support at support@readthedocs.com. + .. tab:: |org_brand| * 15 minutes build time @@ -128,12 +140,3 @@ Currently, these build limits are: If your business is hitting build limits hosting documentation on Read the Docs, please consider :doc:`Read the Docs for Business ` which has much higher build resources. - - .. tab:: |com_brand| - - * 30 minutes build time - * 7GB of memory - * Concurrent builds vary based on your pricing plan - - If you are having trouble with your documentation builds, - you can reach our support at support@readthedocs.com. diff --git a/docs/user/glossary.rst b/docs/user/glossary.rst index 044fdd40a2a..6491bdf4a12 100644 --- a/docs/user/glossary.rst +++ b/docs/user/glossary.rst @@ -7,8 +7,11 @@ so that you have a reference for how we're using them. .. glossary:: dashboard - `Main page `_ where you can see all your projects with their build status - and import a new project. + The "admin" site where Read the Docs projects are managed and configured. + This varies for our two properties: + + * |com_brand|: https://readthedocs.com/dashboard/. + * |org_brand|: https://readthedocs.org/dashboard/. default version Projects have a *default version*, usually the latest stable version of a project. diff --git a/docs/user/integrations.rst b/docs/user/integrations.rst index c3b4dd3dba2..d27e3d39b86 100644 --- a/docs/user/integrations.rst +++ b/docs/user/integrations.rst @@ -6,28 +6,32 @@ (Perhaps reuse this: https://about.readthedocs.com/images/homepage.png) + Continuous Documentation Deployment =================================== Read the Docs is a *Continuous Documentation Deployment* platform for your software project. Every time you change something in your documentation, Read the Docs will detect your change and build your documentation. -This happens using *webhooks*. The Continuous Integration and Continuous Deployment (CI/CD) features are configured with your repository provider, -such as GitHub, Bitbucket or GitLab, -and with each change committed to your repository, Read the Docs is notified by the webhook. +such as GitHub, Bitbucket or GitLab. +With each change committed to your repository, we are notified by the configured *webhook*. + +When a receive a *webhook* notification, we match it to a project's *Integration*. +When a webhook is received, the matching project will then: -When we receive an integration notification, we determine if the change is related to an active version for your project. -If it is, a build is triggered for that version. +* :doc:`Build ` the latest commit. +* Synchronize your versions based on the latest tag and branch data in Git. +* Run your :doc:`automation rules`. +* Auto-cancel any currently running builds of the same version. +* Add a log entry to the integration's :guilabel:`Recent Activity`. Continuous Documentation for software projects ---------------------------------------------- -Documentation fits into any CI/CD pipeline through the idea and objectives of *Documentation as Code*. -A straight-forward method to achieve this is by maintaining documentation alongside the source code, -meaning that the documentation's life cycle is contained within the same Git repository as the source code. - -When changes happen to the source code, changes should also happen to the documentation. +Documentation fits into any CI/CD pipeline by following a process known as *Documentation as Code (Docs as code)*. +The primary method of doing this is by maintaining documentation alongside the source code, +meaning that the documentation's life cycle is the same as your software project. By managing these changes in the same life cycle, you can benefit from **documentation and source code being part of the same code review process**. @@ -36,21 +40,15 @@ keep it updated with minimal effort. This allows more iteration on documentation, and increases overall value from the documentation you write. -Continuous Documentation for all projects ------------------------------------------ - -All categories of documentation benefit from being *continuously* built and published through Read the Docs. - -If you are managing your documentation in Git, -you will be able to preview changes and see accepted changes published immediately using :doc:`/pull-requests`. +As part of this quick feedback loop, +You can preview documentation changes immediately using :doc:`pull request previews `. Automated versioning -------------------- With Read the Docs' automated CI/CD pipeline, you will be able to fully align your project's **release cycle** with your documentation. -For instance, a new version of a software project can build and publish a new version of its documentation. +For instance, a new version of a software project can build and publish a :doc:`new documentation version `. -Read the Docs is notified of all changes to your Git branches and tags. When you release a new version for your project, you are likely also adding a version tag to your Git repository. These Git events can be configured to build and publish your documentation automatically with :doc:`/automation-rules`.