Merge tag '9.12.0' into rel

Author: ericholscher

CHANGELOG.rst

@@ -1,3 +1,31 @@
+Version 9.12.0
+--------------
+
+:Date: May 02, 2023
+
+* `@benjaoming <https://github.com/benjaoming>`__: Allow build.commands without build.tools (`#10281 <https://github.com/readthedocs/readthedocs.org/pull/10281>`__)
+* `@benjaoming <https://github.com/benjaoming>`__: Remove raise_for_exception=False tests (`#10280 <https://github.com/readthedocs/readthedocs.org/pull/10280>`__)
+* `@github-actions[bot] <https://github.com/github-actions[bot]>`__: Dependencies: all packages updated via pip-tools (`#10278 <https://github.com/readthedocs/readthedocs.org/pull/10278>`__)
+* `@stsewd <https://github.com/stsewd>`__: Remove dead code (`#10275 <https://github.com/readthedocs/readthedocs.org/pull/10275>`__)
+* `@benjaoming <https://github.com/benjaoming>`__: Dev: Disable cacheops in proxito docker environment (`#10274 <https://github.com/readthedocs/readthedocs.org/pull/10274>`__)
+* `@stsewd <https://github.com/stsewd>`__: Tests: be explicit about the privacy level (`#10273 <https://github.com/readthedocs/readthedocs.org/pull/10273>`__)
+* `@stsewd <https://github.com/stsewd>`__: Fix typo in tests (`#10271 <https://github.com/readthedocs/readthedocs.org/pull/10271>`__)
+* `@stsewd <https://github.com/stsewd>`__: Update docs about setuptools dependency (`#10270 <https://github.com/readthedocs/readthedocs.org/pull/10270>`__)
+* `@stsewd <https://github.com/stsewd>`__: Build: Pin setuptools only when required (`#10268 <https://github.com/readthedocs/readthedocs.org/pull/10268>`__)
+* `@github-actions[bot] <https://github.com/github-actions[bot]>`__: Dependencies: all packages updated via pip-tools (`#10267 <https://github.com/readthedocs/readthedocs.org/pull/10267>`__)
+* `@benjaoming <https://github.com/benjaoming>`__: Backend: Make Features ordered in a nice way (`#10262 <https://github.com/readthedocs/readthedocs.org/pull/10262>`__)
+* `@stsewd <https://github.com/stsewd>`__: Proxito: allow overlapping public and external version domains (`#10260 <https://github.com/readthedocs/readthedocs.org/pull/10260>`__)
+* `@ericholscher <https://github.com/ericholscher>`__: Revert "Proxito: inject hosting integration header for `build.commands` (#10219)" (`#10259 <https://github.com/readthedocs/readthedocs.org/pull/10259>`__)
+* `@stsewd <https://github.com/stsewd>`__: Test explicitly without organizations (`#10258 <https://github.com/readthedocs/readthedocs.org/pull/10258>`__)
+* `@ericholscher <https://github.com/ericholscher>`__: Release 9.11.0 (`#10255 <https://github.com/readthedocs/readthedocs.org/pull/10255>`__)
+* `@stsewd <https://github.com/stsewd>`__: Tests: set production domain explicitly (`#10253 <https://github.com/readthedocs/readthedocs.org/pull/10253>`__)
+* `@benjaoming <https://github.com/benjaoming>`__: Docs: Style guide stash (`#10250 <https://github.com/readthedocs/readthedocs.org/pull/10250>`__)
+* `@benjaoming <https://github.com/benjaoming>`__: Docs: New entries to glossary (`#10249 <https://github.com/readthedocs/readthedocs.org/pull/10249>`__)
+* `@stsewd <https://github.com/stsewd>`__: Proxito: handle http to https redirects for all requests (`#10199 <https://github.com/readthedocs/readthedocs.org/pull/10199>`__)
+* `@ericholscher <https://github.com/ericholscher>`__: Fix checking of PR status (`#10085 <https://github.com/readthedocs/readthedocs.org/pull/10085>`__)
+* `@ewdurbin <https://github.com/ewdurbin>`__: implement multiple .readthedocs.yml files per repo (`#10001 <https://github.com/readthedocs/readthedocs.org/pull/10001>`__)
+* `@benjaoming <https://github.com/benjaoming>`__: Contextualize 404 page (`#9657 <https://github.com/readthedocs/readthedocs.org/pull/9657>`__)
+
 Version 9.11.0
 --------------
 

dockerfiles/settings/proxito.py

@@ -6,7 +6,11 @@
 class ProxitoDevSettings(CommunityProxitoSettingsMixin, DockerBaseSettings):
     DONT_HIT_DB = False
 
-    CACHEOPS_ENABLED = True
+    # Disabled because of issues like
+    # ResponseError: Error running script (call to f_1880dea5c524f6a37a650f715fa630416a2fe1fd):
+    # @user_script:50: @user_script: 50: Wrong number of args calling Redis command From Lua script
+    # (this issue goes away after a few reloads)
+    CACHEOPS_ENABLED = False
 
     # El Proxito does not have django-debug-toolbar installed
     @property

docs/conf.py

@@ -74,7 +74,7 @@
 
 master_doc = "index"
 copyright = "Read the Docs, Inc & contributors"
-version = "9.11.0"
+version = "9.12.0"
 release = version
 exclude_patterns = ["_build", "shared", "_includes"]
 default_role = "obj"

docs/dev/style-guide.rst

@@ -54,11 +54,25 @@ Word list
 
 We have a specific way that we write common words:
 
+* ``build command`` is the name of each step in the file.
+  We try to avoid confusion with pipelines, jobs and steps from other CIs,
+  as we do not have a multi-dimentional build sequence.
+* ``build job`` is the name of the general and pre-defined steps that can be overridden.
+  They are similar to "steps" in pipelines,
+  but on Read the Docs they are pre-defined.
+  So it's important to have a unique name.
+* ``Git`` should be upper case. Except when referring to the :program:`git` command, then it should be written as `:program:\`git\``.
 * ``Git repository`` for the place that stores Git repos. We used to use ``VCS``, but this is deprecated.
 * ``Git provider`` for generic references to GitHub/Bitbucket/GitLab/Gitea etc.
   We avoid "host" and "platform" because they are slightly more ambiguous.
+* ``how to`` do the thing is explained in a ``how-to guide`` (notice hyphen and spaces).
+* ``lifecycle`` is spelled without hyphen nor space.
 * ``open source`` should be lower case, unless you are definitely referring to `OSI's Open Source Definition <https://opensource.org/osd>`.
-* ``Git`` should be upper case. Except when referring to the :program:`git` command, then it should be written as `:program:\`git\``.
+* ``.readthedocs.yaml`` is the general name of the build configuration file.
+  Even though we allow custom paths to the config file,
+  we only validate ``.readthedocs.yaml`` as the file name.
+  Older variations of the name are considered legacy.
+  We do not refer to it in general terms.
 
 Substitutions
 -------------

docs/user/build-customization.rst

@@ -360,14 +360,25 @@ Where to put files
 ~~~~~~~~~~~~~~~~~~
 
 It is your responsibility to generate HTML and other formats of your documentation using :ref:`config-file/v2:build.commands`.
-The contents of the ``_readthedocs/<format>/`` directory will be hosted as part of your documentation.
+The contents of the ``$READTHEDOCS_OUTPUT/<format>/`` directory will be hosted as part of your documentation.
+
+We store the the base folder name ``_readthedocs/`` in the environment variable ``$READTHEDOCS_OUTPUT`` and encourage that you use this to generate paths.
 
 Supported :ref:`formats <downloadable-documentation:accessing offline formats>` are published if they exist in the following directories:
 
-* ``_readthedocs/html/`` (required)
-* ``_readthedocs/htmlzip/``
-* ``_readthedocs/pdf/``
-* ``_readthedocs/epub/``
+* ``$READTHEDOCS_OUTPUT/html/`` (required)
+* ``$READTHEDOCS_OUTPUT/htmlzip/``
+* ``$READTHEDOCS_OUTPUT/pdf/``
+* ``$READTHEDOCS_OUTPUT/epub/``
+
+.. note::
+
+   Remember to create the folders before adding content to them.
+   You can ensure that the output folder exists by adding the following command:
+
+   .. code-block:: console
+
+       mkdir -p $READTHEDOCS_OUTPUT/html/
 
 Search support
 ~~~~~~~~~~~~~~

docs/user/build-default-versions.rst

@@ -52,7 +52,8 @@ pip:
   Latest version by default.
 
 setuptools:
-  ``58.2.0`` or older.
+  Projects using ``setup.py install`` as installation method use ``58.2.0`` or older.
+  All other projects use the latest version.
 
 mock:
   ``1.0.1`` (could be removed in the future).

docs/user/build-notifications.rst

@@ -9,7 +9,7 @@ Email notifications:
   This makes sure you know when your builds have failed.
 
 Build Status Webhooks:
-  Build notifications can happen via webhooks.
+  Build notifications can happen via :term:`webhooks <webhook>`.
   This means that we are able to support a wide variety of services that receive notifications.
 
   Slack and Discord are supported through ready-made templates.

docs/user/connected-accounts.rst

@@ -88,7 +88,7 @@ which allow us to build your documentation on every change to your repository.
           We ask for this so you can create a Read the Docs account and login with your GitHub credentials.
 
       Administering webhooks (``admin:repo_hook``)
-          We ask for this so we can create webhooks on your repositories when you import them into Read the Docs.
+          We ask for this so we can create :term:`webhooks <webhook>` on your repositories when you import them into Read the Docs.
           This allows us to build the docs when you push new commits.
 
       Read access to your organizations (``read:org``)
@@ -115,7 +115,7 @@ which allow us to build your documentation on every change to your repository.
       We request permissions for:
 
       Administering your repositories (``repository:admin``)
-        We ask for this so we can create webhooks on your repositories when you import them into Read the Docs.
+        We ask for this so we can create :term:`webhooks <webhook>` on your repositories when you import them into Read the Docs.
         This allows us to build the docs when you push new commits.
         NB! This permission scope does **not** include any write access to code.
 

docs/user/faq.rst

@@ -354,7 +354,7 @@ and as a result, it tends to look a bit better with the default theme.
 I need to install a package in a environment with pinned versions
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-To ensure proper installation of a Python package, the ``pip`` :ref:`install method <config-file/v2:python.install>` will automatically upgrade every dependency to its most recent version in case they aren't pinned by the package definition.
+To ensure proper installation of a Python package, the ``pip`` :ref:`install method <config-file/v2:python.install>` will automatically upgrade every dependency to its most recent version in case they aren't:term:`pinned <pinning>` by the package definition.
 If instead you'd like to pin your dependencies outside the package, you can add this line to your requirements or environment file (if you are using Conda).
 
 In your ``requirements.txt`` file::

docs/user/glossary.rst

@@ -6,6 +6,11 @@ so that you have a reference for how we're using them.
 
 .. glossary::
 
+   CI/CD
+      CI/CD is a common way to write *Continuous Integration and Continuous Deployment*.
+      In some scenarios, they exist as two separate platforms.
+      Read the Docs is a combined CI/CD platform made for documentation.
+
    dashboard
       The "admin" site where Read the Docs projects are managed and configured.
       This varies for our two properties:
@@ -39,36 +44,97 @@ so that you have a reference for how we're using them.
       and rules for publication of documentation can be :doc:`automated </automation-rules>`.
       Similar to :term:`Docs as Code`.
 
+   pinning
+      To *pin* a requirement means to explicitly specify which version should be used.
+      *Pinning* software requirements is the most important technique to make a project :term:`reproducible`.
+
+      When documentation builds,
+      software dependencies are installed in their latest versions permitted by the pinning specification.
+      Since software packages are frequently released,
+      we are usually trying to avoid incompatibilities in a new release from suddenly breaking a documentation build.
+
+      Examples of Python dependencies::
+
+          # Exact pinning: Only allow Sphinx 5.3.0
+          sphinx==5.3.0
+
+          # Loose pinning: Lower and upper bounds result in the latest 5.3.x release
+          sphinx>=5.3,<5.4
+
+          # Very loose pinning: Lower and upper bounds result in the latest 5.x release
+          sphinx>=5,<6
+
+      Read the Docs recommends using **exact pinning**.
+
+      See: :doc:`/guides/reproducible-builds`.
+
    pre-defined build jobs
       Commands executed by Read the Docs when performing the build process.
       They cannot be overwritten by the user.
 
-   profile page
-      Page where you can see the projects of a certain user.
-
    project home
       Page where you can access all the features of Read the Docs,
       from having an overview to browsing the latest builds or administering your project.
 
    project page
       Another name for :term:`project home`.
 
+   reproducible
+      A documentation project is said to be *reproducible* when its sources build correctly on Read the Docs over a periode of many years.
+      You can also think of being *reproducible* as being *robust* or *resillient*.
+
+      Being "reproducible" is an important positive quality goal of documentation.
+
+      When builds are not reproducible and break due to external factors,
+      they need frequent troubleshooting and manual fixing.
+
+      The most common external factor is that new versions of software dependencies are released.
+
+      See: :doc:`/guides/reproducible-builds`.
+
+   root URL
+      Home URL of your documentation without the ``/<lang>`` and ``/<version>`` segments.
+      For projects without custom domains, the one ending in ``.readthedocs.io/``
+      (for example, ``https://docs.readthedocs.io`` as opposed to ``https://docs.readthedocs.io/en/latest``).
+
    slug
       A unique identifier for a project or version. This value comes from the
       project or version name, which is reduced to lowercase letters, numbers,
       and hyphens. You can retrieve your project or version slugs from
       :doc:`our API <api/v3>`.
 
+   static website
+      A static site or static website is a collection of HTML files, images, CSS and JavaScript that are served statically,
+      as opposed to dynamic websites that generate a unique response for each request, using databases and user sessions.
+
+      Static websites are highly portable, as they do not depend on the webserver.
+      They can also be viewed offline.
+
+      Documentation projects served on Read the Docs are *static websites*.
+
+      Tools to manage and generate static websites are commonly known as *static site generators* and there is a big overlap with documentation tools.
+      Some static site generators are also documentation tools,
+      and some documentation tools are also used to generate normal websites.
+
+      For instance, :doc:`Sphinx <sphinx:index>` is made for documentation but also used for blogging.
+
    subproject
       Project A can be configured such that when requesting a URL ``/projects/<subproject-slug>``,
       the root of project B is returned.
       In this case, *project B* is the subproject.
       Read more in :doc:`/subprojects`.
 
-   root URL
-      Home URL of your documentation without the ``/<lang>`` and ``/<version>`` segments.
-      For projects without custom domains, the one ending in ``.readthedocs.io/``
-      (for example, ``https://docs.readthedocs.io`` as opposed to ``https://docs.readthedocs.io/en/latest``).
-
    user-defined build jobs
       Commands defined by the user that Read the Docs will execute when performing the build process.
+
+   webhook
+      A webhook is a special URL that can be called from another service,
+      usually with a secret token.
+      It is commonly used to start a build or a deployment or to send a status update.
+
+      There are two important types of webhooks for Read the Docs:
+
+      * Git providers have webhooks which are special URLs that Read the Docs can call in order to notify about documentation builds.
+      * Read the Docs has a unique webhook for each project that the Git provider calls when changes happen in Git.
+
+      See also: :doc:`/guides/git-integrations` and :doc:`/build-notifications`

docs/user/guides/build-notifications.rst

@@ -6,7 +6,7 @@ Build notifications can alert you when your builds fail so you can take immediat
 
 .. note::
 
-   Currently we don't send notifications or trigger webhooks
+   Currently we don't send notifications or trigger :term:`webhooks <webhook>`
    on :doc:`builds from pull requests </pull-requests>`.
 
 

docs/user/guides/git-integrations.rst

@@ -114,7 +114,7 @@ Provider-specific instructions
 Additional integration
 ----------------------
 
-You can configure multiple webhooks.
+You can configure multiple incoming :term:`webhooks <webhook>`.
 
 To manually set up an integration, go to :guilabel:`Admin` > :guilabel:`Integrations` >  :guilabel:`Add integration`
 dashboard page and select the integration type you'd like to add.

docs/user/guides/reproducible-builds.rst

@@ -2,7 +2,7 @@ How to create reproducible builds
 =================================
 
 Your documentation depends on a number of dependencies to be built.
-If your docs don't have reproducible builds,
+If your docs don't have :term:`reproducible` builds,
 an update in a dependency can break your builds when least expected,
 or make your docs look different from your local version.
 This guide will help you to keep your builds working over time,
@@ -15,7 +15,6 @@ so that you can focus on content.
 Use a ``.readthedocs.yaml`` configuration file
 ----------------------------------------------
 
-
 We recommend using a :doc:`configuration file </config-file/v2>` to manage your documentation.
 Our config file *provides you per version settings*,
 and *those settings live in your Git repository*.
@@ -26,7 +25,7 @@ and ensures that all your versions can be rebuilt from a reproducible configurat
 Use a requirements file for Python dependencies
 -----------------------------------------------
 
-We recommend using a Pip :ref:`requirements file <pip:requirements-file-format>` or Conda :ref:`environment file <config-file/v2:conda.environment>` to pin Python dependencies.
+We recommend using a Pip :ref:`requirements file <pip:requirements-file-format>` or Conda :ref:`environment file <config-file/v2:conda.environment>` to :term:`pin <pinning>` Python dependencies.
 This ensures that top-level dependencies and extensions don't change.
 
 A configuration file with explicit dependencies looks like this:
@@ -66,7 +65,7 @@ A configuration file with explicit dependencies looks like this:
 Pin your transitive dependencies
 --------------------------------
 
-Once you have pinned your own dependencies,
+Once you have :term:`pinned <pinning>` your own dependencies,
 the next things to worry about are the dependencies of your dependencies.
 These are called *transitive dependencies*,
 and they can upgrade without warning if you do not pin these packages as well.
@@ -144,7 +143,7 @@ Check list ✅
 -------------
 
 If you followed this guide,
-you have pinned:
+you have:term:`pinned <pinning>`:
 
 * tool versions (Python, Node)
 * top-level dependencies (Sphinx, Sphinx extensions)

docs/user/tutorial/index.rst

@@ -79,7 +79,7 @@ On the authorization page, click the green :guilabel:`Authorize readthedocs` but
 
    Read the Docs needs elevated permissions to perform certain operations
    that ensure that the workflow is as smooth as possible,
-   like installing webhooks.
+   like installing :term:`webhooks <webhook>`.
    If you want to learn more,
    check out :ref:`connected-accounts:permissions for connected accounts`.
 

media/css/core.css

@@ -1331,6 +1331,9 @@ div.module.project-subprojects li.subproject a.subproject-edit:before {
     content: "\f044";
 }
 
+#content ul.normal_list {list-style: disc; margin-left: 20px;}
+#content code {background: #eee; border: 1px solid #ccc; padding: 3px; display: inline-block;}
+
 /* Pygments */
 div.highlight pre .hll { background-color: #ffffcc }
 div.highlight pre .c { color: #60a0b0; font-style: italic } /* Comment */

package.json

@@ -1,6 +1,6 @@
 {
   "name": "readthedocs",
-  "version": "9.11.0",
+  "version": "9.12.0",
   "description": "Read the Docs build dependencies",
   "author": "Read the Docs, Inc <[email protected]>",
   "scripts": {

readthedocs/__init__.py

@@ -1,4 +1,4 @@
 """Read the Docs."""
 
 
-__version__ = "9.11.0"
+__version__ = "9.12.0"

readthedocs/api/mixins.py

@@ -7,6 +7,7 @@
 from readthedocs.core.unresolver import UnresolverError, unresolve
 from readthedocs.core.utils import get_cache_tag
 from readthedocs.projects.models import Project
+from readthedocs.proxito.cache import add_cache_tags
 
 log = structlog.get_logger(__name__)
 
@@ -31,7 +32,7 @@ def dispatch(self, request, *args, **kwargs):
         response = super().dispatch(request, *args, **kwargs)
         cache_tags = self._get_cache_tags()
         if cache_tags:
-            response["Cache-Tag"] = ",".join(cache_tags)
+            add_cache_tags(response, cache_tags)
         return response
 
     def _get_cache_tags(self):