Build: skip build based on commands' exit codes (#9649)

* Build: skip build based on commands' exit codes

Define a particular exit code (439) to skip a build.

If any of the commands returns this exit code, the build will be cancelled
automatically and won't run any of the following commands. When this happens,
the build will be marked as `cancelled` and no email/webhook notifications will
be sent.

Why 439 was chosen for the exit code?

It's the word "skip" encoded in ASCII. Then it's taken the 256 modulo of it because
the Unix implementation does this automatically for exit codes greater than 255

```
>>> sum(list('skip'.encode('ascii')))
439
>>> 439 % 256
183
```

* Update readthedocs/doc_builder/exceptions.py

Co-authored-by: Anthony <[email protected]>

* Build: send SUCCESS external build status on skipped builds

When the command exists with the magic exit code, we send SUCCESS to
GitHub/GitLab as status so the PR check passes and the PR can be merged.

* Docs: skip build based on a condition

* Lint: small darker change

* Docs: reference fixed

* Docs: improve comment on YAML

* Build: make usage of `status` variable

Thanks to Benjamin for pointing it to me ;)

* Apply suggestions from code review

Co-authored-by: Benjamin Balder Bach <[email protected]>

* Docs: update code example for skip builds

Use Bash's `if` to only run this code on pull requests.

* Docs: example showing how to skip the build based on commit message

* Docs: refactor "skipping a build" section (#9717)

* Docs: refactor "skipping a build" section

- Move the explanation about the "Cancelling builds" feature to the "Builds"
page
- Keep the examples (How-To) for "Cancel a build" into the "Build customization"
page

* Docs: use 183 instead of 439 exit code

> This error code isn't a valid exit
> code (tldp.org/LDP/abs/html/exitcodes.html), we shouldn't document a code above
> 255 otherwise users will get confused.

From Eric's review.

* Apply suggestions from code review

Co-authored-by: Eric Holscher <[email protected]>

* Minor fix underline

* Use multi-line bash examples

They are easier to read

* Update docs/user/builds.rst

Co-authored-by: Eric Holscher <[email protected]>

Co-authored-by: Eric Holscher <[email protected]>

* Docs: use `grep` instead of `case`

* Docs: small note about "Why 183?"

Co-authored-by: Anthony <[email protected]>
Co-authored-by: Benjamin Balder Bach <[email protected]>
Co-authored-by: Eric Holscher <[email protected]>

Author: 4 people

docs/user/build-customization.rst

@@ -73,6 +73,7 @@ There are some caveats to knowing when using user-defined jobs:
 * Environment variables are expanded in the commands (see :doc:`environment-variables`)
 * Each command is executed in a new shell process, so modifications done to the shell environment do not persist between commands
 * Any command returning non-zero exit code will cause the build to fail immediately
+  (note there is a special exit code to `cancel the build <cancel-build-based-on-a-condition>`_)
 * ``build.os`` and ``build.tools`` are required when using ``build.jobs``
 
 
@@ -104,6 +105,70 @@ To avoid this, it's possible to unshallow the clone done by Read the Docs:
          - git fetch --unshallow
 
 
+Cancel build based on a condition
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When a command exits with code ``183``,
+Read the Docs will cancel the build immediately.
+You can use this approach to cancel builds that you don't want to complete based on some conditional logic.
+
+.. note:: Why 183 was chosen for the exit code?
+
+   It's the word "skip" encoded in ASCII.
+   Then it's taken the 256 modulo of it because
+   `the Unix implementation does this automatically <https://tldp.org/LDP/abs/html/exitcodes.html>`_
+   for exit codes greater than 255.
+
+   .. code-block:: python
+
+      >>> sum(list('skip'.encode('ascii')))
+      439
+      >>> 439 % 256
+      183
+
+
+Here is an example that cancels builds from pull requests when there are no changes to the ``docs/`` folder compared to the ``origin/main`` branch:
+
+.. code-block:: yaml
+   :caption: .readthedocs.yaml
+
+   version: 2
+   build:
+     os: "ubuntu-22.04"
+     tools:
+       python: "3.11"
+     jobs:
+       post_checkout:
+         # Cancel building pull requests when there aren't changed in the docs directory.
+         # `--quiet` exits with a 1 when there **are** changes,
+         # so we invert the logic with a !
+         #
+         # If there are no changes (exit 0) we force the command to return with 183.
+         # This is a special exit code on Read the Docs that will cancel the build immediately.
+         - |
+           if [ $READTHEDOCS_VERSION_TYPE = "external" ];
+           then
+             ! git diff --quiet origin/main -- docs/ && exit 183;
+           fi
+
+
+This other example shows how to cancel a build if the commit message contains ``skip ci`` on it:
+
+.. code-block:: yaml
+   :caption: .readthedocs.yaml
+
+   version: 2
+   build:
+     os: "ubuntu-22.04"
+     tools:
+       python: "3.11"
+     jobs:
+       post_checkout:
+         # Use `git log` to check if the latest commit contains "skip ci",
+         # in that case exit the command with 183 to cancel the build
+         - (git --no-pager log --pretty="tformat:%s -- %b" -1 | grep -viq "skip ci") || exit 183
+
+
 Generate documentation from annotated sources with Doxygen
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

docs/user/builds.rst

@@ -68,6 +68,46 @@ The following are the pre-defined jobs executed by Read the Docs:
     it's possible to run user-defined commands and :doc:`customize the build process <build-customization>`.
 
 
+When to cancel builds
+---------------------
+
+There may be situations where you want to cancel a particular running build.
+Cancelling running builds will allow your team to speed up review times and also help us reduce server costs and ultimately,
+our environmental footprint.
+
+Consider the following scenarios:
+
+* the build has an external dependency that hasn't been updated
+* there were no changes on the documentation files
+* many other use cases that can be solved with custom logic
+
+For these scenarios,
+Read the Docs supports three different mechanisms to cancel a running build:
+
+:Manually:
+
+   Once a build was triggered,
+   project administrators can go to the build detail page
+   and click the button "Cancel build".
+
+:Automatically:
+
+   When Read the Docs detects a push to a branch that it's currently building the documentation,
+   it cancels the running build and start a new build using the latest commit from the new push.
+
+:Programatically:
+
+   You can use user-defined commands on ``build.jobs`` or ``build.commands`` (see :doc:`build-customization`)
+   to check for a condition and exit it with the code ``183`` if you want to cancel the running build or ``0``, otherwise.
+
+   In this case, Read the Docs will communicate to your Git platform (GitHub/GitLab) that the build succeeded (green tick ✅)
+   so the pull request is in a mergeable state.
+
+   .. tip::
+
+      Take a look at :ref:`build-customization:cancel build based on a condition` section for some examples.
+
+
 Build resources
 ---------------
 

readthedocs/doc_builder/constants.py

@@ -1,13 +1,11 @@
-# -*- coding: utf-8 -*-
 
 """Doc build constants."""
 
-import structlog
 import re
 
+import structlog
 from django.conf import settings
 
-
 log = structlog.get_logger(__name__)
 
 PDF_RE = re.compile('Output written on (.*?)')
@@ -30,3 +28,11 @@
 DOCKER_OOM_EXIT_CODE = 137
 
 DOCKER_HOSTNAME_MAX_LEN = 64
+
+# Why 183 exit code?
+#
+# >>> sum(list('skip'.encode('ascii')))
+# 439
+# >>> 439 % 256
+# 183
+RTD_SKIP_BUILD_EXIT_CODE = 183

readthedocs/doc_builder/environments.py

@@ -30,8 +30,9 @@
     DOCKER_SOCKET,
     DOCKER_TIMEOUT_EXIT_CODE,
     DOCKER_VERSION,
+    RTD_SKIP_BUILD_EXIT_CODE,
 )
-from .exceptions import BuildAppError, BuildUserError
+from .exceptions import BuildAppError, BuildUserError, BuildUserSkip
 
 log = structlog.get_logger(__name__)
 
@@ -468,6 +469,8 @@ def run_command_class(
                     project_slug=self.project.slug if self.project else '',
                     version_slug=self.version.slug if self.version else '',
                 )
+            elif build_cmd.exit_code == RTD_SKIP_BUILD_EXIT_CODE:
+                raise BuildUserSkip()
             else:
                 # TODO: for now, this still outputs a generic error message
                 # that is the same across all commands. We could improve this

readthedocs/doc_builder/exceptions.py

@@ -42,6 +42,11 @@ class BuildUserError(BuildBaseException):
     )
 
 
+class BuildUserSkip(BuildUserError):
+    message = gettext_noop("This build was manually skipped using a command exit code.")
+    state = BUILD_STATE_CANCELLED
+
+
 class ProjectBuildsSkippedError(BuildUserError):
     message = gettext_noop('Builds for this project are temporarily disabled')
 

readthedocs/projects/tasks/builds.py

@@ -44,6 +44,7 @@
     BuildCancelled,
     BuildMaxConcurrencyError,
     BuildUserError,
+    BuildUserSkip,
     MkDocsYAMLParseError,
     ProjectBuildsSkippedError,
     YAMLParseError,
@@ -280,6 +281,7 @@ class UpdateDocsTask(SyncRepositoryMixin, Task):
         YAMLParseError,
         BuildCancelled,
         BuildUserError,
+        BuildUserSkip,
         RepositoryError,
         MkDocsYAMLParseError,
         ProjectConfigurationError,
@@ -289,6 +291,7 @@ class UpdateDocsTask(SyncRepositoryMixin, Task):
     exceptions_without_notifications = (
         BuildCancelled,
         BuildMaxConcurrencyError,
+        BuildUserSkip,
         ProjectBuildsSkippedError,
     )
 
@@ -450,8 +453,8 @@ def on_failure(self, exc, task_id, args, kwargs, einfo):
             )
         # Known errors in the user's project (e.g. invalid config file, invalid
         # repository, command failed, etc). Report the error back to the user
-        # using the `message` attribute from the exception itself. Otherwise,
-        # use a generic message.
+        # using the `message` and `state` attributes from the exception itself.
+        # Otherwise, use a generic message and default state.
         elif isinstance(exc, BuildUserError):
             if hasattr(exc, 'message') and exc.message is not None:
                 self.data.build['error'] = exc.message
@@ -490,11 +493,21 @@ def on_failure(self, exc, task_id, args, kwargs, einfo):
             version_type = None
             if self.data.version:
                 version_type = self.data.version.type
+
+            # NOTE: autoflake gets confused here. We need the NOQA for now.
+            status = BUILD_STATUS_FAILURE
+            if isinstance(exc, BuildUserSkip):
+                # The build was skipped by returning the magic exit code,
+                # marked as CANCELLED, but communicated to GitHub as successful.
+                # This is because the PR has to be available for merging when the build
+                # was skipped on purpose.
+                status = BUILD_STATUS_SUCCESS
+
             send_external_build_status(
                 version_type=version_type,
                 build_pk=self.data.build['id'],
                 commit=self.data.build_commit,
-                status=BUILD_STATUS_FAILURE,
+                status=status,
             )
 
         # Update build object