Merge tag '9.11.0' into rel

Author: ericholscher

CHANGELOG.rst

@@ -1,3 +1,22 @@
+Version 9.11.0
+--------------
+
+:Date: April 18, 2023
+
+* `@benjaoming <https://github.com/benjaoming>`__: Fix a little test failure (`#10248 <https://github.com/readthedocs/readthedocs.org/pull/10248>`__)
+* `@benjaoming <https://github.com/benjaoming>`__: Scripts: Add export statements and instruction to fetch awscli (compile_version_upload_s3.sh) (`#10245 <https://github.com/readthedocs/readthedocs.org/pull/10245>`__)
+* `@github-actions[bot] <https://github.com/github-actions[bot]>`__: Dependencies: all packages updated via pip-tools (`#10244 <https://github.com/readthedocs/readthedocs.org/pull/10244>`__)
+* `@stsewd <https://github.com/stsewd>`__: API V3: make tests explicit (`#10236 <https://github.com/readthedocs/readthedocs.org/pull/10236>`__)
+* `@ericholscher <https://github.com/ericholscher>`__: Release 9.10.1 (`#10235 <https://github.com/readthedocs/readthedocs.org/pull/10235>`__)
+* `@dependabot[bot] <https://github.com/dependabot[bot]>`__: Bump peter-evans/create-pull-request from 4 to 5 (`#10233 <https://github.com/readthedocs/readthedocs.org/pull/10233>`__)
+* `@github-actions[bot] <https://github.com/github-actions[bot]>`__: Dependencies: all packages updated via pip-tools (`#10232 <https://github.com/readthedocs/readthedocs.org/pull/10232>`__)
+* `@agjohnson <https://github.com/agjohnson>`__: Add notes on private repo support in our install docs (`#10230 <https://github.com/readthedocs/readthedocs.org/pull/10230>`__)
+* `@stsewd <https://github.com/stsewd>`__: Analytics API: check if absolute_uri isn't present (`#10227 <https://github.com/readthedocs/readthedocs.org/pull/10227>`__)
+* `@humitos <https://github.com/humitos>`__: Proxito: inject hosting integration header for `build.commands` (`#10219 <https://github.com/readthedocs/readthedocs.org/pull/10219>`__)
+* `@humitos <https://github.com/humitos>`__: API: hosting integrations endpoint versioning/structure (`#10216 <https://github.com/readthedocs/readthedocs.org/pull/10216>`__)
+* `@benjaoming <https://github.com/benjaoming>`__: Search: index <dl>s as sections and remove Sphinx domain logic (`#10128 <https://github.com/readthedocs/readthedocs.org/pull/10128>`__)
+* `@ewdurbin <https://github.com/ewdurbin>`__: implement multiple .readthedocs.yml files per repo (`#10001 <https://github.com/readthedocs/readthedocs.org/pull/10001>`__)
+
 Version 9.10.1
 --------------
 

docs/_static/js/readthedocs-doc-diff.js

@@ -74,7 +74,7 @@
 
 master_doc = "index"
 copyright = "Read the Docs, Inc & contributors"
-version = "9.10.1"
+version = "9.11.0"
 release = version
 exclude_patterns = ["_build", "shared", "_includes"]
 default_role = "obj"

docs/conf.py

@@ -117,6 +117,13 @@ Tags to be ignored:
 
 - ``nav``
 
+Special rules that are derived from specific documentation tools applied in the generic parser:
+
+.. https://squidfunk.github.io/mkdocs-material/reference/code-blocks/#adding-line-numbers
+
+- ``.linenos``, ``.lineno`` (line numbers in code-blocks, comes from both MkDocs and Sphinx)
+- ``.headerlink`` (added by Sphinx to links in headers)
+
 Example:
 
 .. code-block:: html
@@ -133,12 +140,27 @@ Example:
 Sections
 ~~~~~~~~
 
-Sections are composed of a title, and a content.
-A section title can be a ``h`` tag, or a ``header`` tag containing a ``h`` tag,
-the ``h`` tag or its parent can contain an ``id`` attribute, which will be used to link to the section.
+Sections are stored in a dictionary composed of an ``id``, ``title`` and ``content`` key.
 
-All content below the title, until a new section is found, will be indexed as part of the section content.
-Example:
+Sections are defined as:
+
+* ``h1-h7``, all content between one heading level and the next header on the same level is used as content for that section.
+* ``dt`` elements with an ``id`` attribute, we map the ``title`` to the ``dt`` element and the content to the ``dd`` element.
+
+All sections have to be identified by a DOM container's ``id`` attribute,
+which will be used to link to the section.
+How the id is detected varies with the type of element:
+
+* ``h1-h7`` elements use the ``id`` attribute of the header itself if present, or
+  its ``section`` parent (if exists).
+* ``dt`` elements use the ``id`` attribute of the ``dt`` element.
+
+To avoid duplication and ambiguous section references,
+all indexed ``dl`` elements are removed from the DOM before indexing of other sections happen.
+
+Here is an example of how all content below the title,
+until a new section is found,
+will be indexed as part of the section content:
 
 .. code-block:: html
    :emphasize-lines: 2-10, 12-17, 21-26

docs/dev/search-integration.rst

@@ -24,6 +24,10 @@ The following how-to guides help you solve common tasks and challenges in the se
     Need several projects under the same umbrella?
     Start using subprojects, which is a way to host multiple projects under a "main project".
 
+⏩️ :doc:`Using a .readthedocs.yaml file in a sub-folder </guides/setup/monorepo>`
+    This guide shows how to configure a Read the Docs project to use a custom path for the ``.readthedocs.yaml`` build configuration.
+    *Monorepos* that have multiple documentation projects in the same Git repository can benefit from this feature.
+
 ⏩️ :doc:`Hiding a version </guides/hiding-a-version>`
     Is your version (flyout) menu overwhelmed and hard to navigate?
     Here's how to make it shorter.
@@ -44,4 +48,5 @@ The following how-to guides help you solve common tasks and challenges in the se
    Managing custom domains </guides/custom-domains>
    Managing subprojects </guides/subprojects>
    Hiding a version </guides/hiding-a-version>
+   Using a .readthedocs.yaml file in a sub-folder </guides/setup/monorepo>
    Using custom URL redirects in documentation projects </guides/redirects>

docs/user/guides/setup/index.rst

@@ -0,0 +1,97 @@
+.. Next steps: Show an example pattern for a monorepo layout or link to an example project
+
+How to use a .readthedocs.yaml file in a sub-folder
+===================================================
+
+This guide shows how to configure a Read the Docs project to use a custom path for the ``.readthedocs.yaml`` build configuration.
+`Monorepos <https://en.wikipedia.org/wiki/Monorepo>`__ that have multiple documentation projects in the same Git repository can benefit from this feature.
+
+By default,
+Read the Docs will use the ``.readthedocs.yaml`` at the top level of your Git repository.
+But if a Git repository contains multiple documentation projects that need different build configurations,
+you will need to have a ``.readthedocs.yaml`` file in multiple sub-folders.
+
+.. seealso::
+
+   `sphinx-multiproject <https://sphinx-multiproject.readthedocs.io/en/latest/>`__
+       If you are only using Sphinx projects and want to share the same build configuration,
+       you can also use the ``sphinx-multiproject`` extension.
+
+   :doc:`/guides/environment-variables`
+       You might also be able to reuse the same configuration file across multiple projects,
+       using only environment variables.
+       This is possible if the configuration pattern is very similar and the documentation tool is the same.
+
+Implementation considerations
+-----------------------------
+
+This feature is currently *project-wide*.
+A custom build configuration file path is applied to all versions of your documentation.
+
+.. warning::
+
+   Changing the configuration path will apply to all versions.
+   Different versions of the project may not be able to build again if this path is changed.
+
+Adding an additional project from the same repository
+-----------------------------------------------------
+
+Once you have added the first project from the :ref:`Import Wizard <intro/import-guide:Automatically import your docs>`,
+it will show as if it has already been imported and cannot be imported again.
+In order to add another project with the same repository,
+you will need to use the :ref:`Manual Import <intro/import-guide:Manually import your docs>`.
+
+Setting the custom build configuration file
+-------------------------------------------
+
+Once you have added a Git repository to a project that needs a custom configuration file path,
+navigate to :menuselection:`Admin --> Advanced Settings` and add the path to the :guilabel:`Build configuration file` field.
+
+.. image:: /img/screenshot-howto-build-configuration-file.png
+   :alt: Screenshot of where to find the :guilabel:`Build configuration file` setting.
+
+After pressing :guilabel:`Save`,
+you need to ensure that relevant versions of your documentation are built again.
+
+.. tip::
+
+   Having multiple different build configuration files can be complex.
+   We recommend setting up 1-2 projects in your Monorepo and getting them to build and publish successfully before adding additional projects to the equation.
+
+Next steps
+----------
+
+Once you have your monorepo pattern implemented and tested and it's ready to roll out to all your projects,
+you should also consider the Read the Docs project setup for these individual projects.
+
+Having individual projects gives you the full flexibility of the Read the Docs platform to make individual setups for each project.
+
+For each project, it's now possible to configure:
+
+* Sets of maintainers (or :doc:`organizations </commercial/organizations>` on |com_brand|)
+* :doc:`Custom redirect rules </guides/custom-domains>`
+* :doc:`Custom domains </guides/custom-domains>`
+* :doc:`Automation rules </automation-rules>`
+* :doc:`Traffic and search analytics </reference/analytics>`
+* Additional documentation tools with individual :doc:`build processes </build-customization>`:
+  One project might use :doc:`Sphinx <sphinx:index>`,
+  while another project setup might use `Asciidoctor <https://asciidoctor.org/>`__.
+
+...and much more. *All* settings for a Read the Docs project is available for each individual project.
+
+.. seealso::
+
+   :doc:`/guides/subprojects`
+      More information on nesting one project inside another project.
+      In this setup, it is still possible to use the same monorepo for each subproject.
+
+Other tips
+----------
+
+For a monorepo,
+it's not desirable to have changes in unrelated sub-folders trigger new builds.
+
+Therefore,
+you should consider setting up :ref:`conditional build cancellation rules <build-customization:Cancel build based on a condition>`.
+The configuration is added in each ``.readthedocs.yaml``,
+making it possible to write one conditional build rules per documentation project in the Monorepo 💯️

docs/user/guides/setup/monorepo.rst

@@ -1,6 +1,6 @@
 {
   "name": "readthedocs",
-  "version": "9.10.1",
+  "version": "9.11.0",
   "description": "Read the Docs build dependencies",
   "author": "Read the Docs, Inc <[email protected]>",
   "scripts": {

docs/user/img/screenshot-howto-build-configuration-file.png

@@ -1,4 +1,4 @@
 """Read the Docs."""
 
 
-__version__ = "9.10.1"
+__version__ = "9.11.0"

package.json

@@ -73,26 +73,27 @@ def get_skip(self, obj):
 
     class Meta(ProjectSerializer.Meta):
         fields = ProjectSerializer.Meta.fields + (
-            'enable_epub_build',
-            'enable_pdf_build',
-            'conf_py_file',
-            'analytics_code',
-            'analytics_disabled',
-            'cdn_enabled',
-            'container_image',
-            'container_mem_limit',
-            'container_time_limit',
-            'install_project',
-            'use_system_packages',
-            'skip',
-            'requirements_file',
-            'python_interpreter',
-            'features',
-            'has_valid_clone',
-            'has_valid_webhook',
-            'show_advertising',
-            'environment_variables',
-            'max_concurrent_builds',
+            "enable_epub_build",
+            "enable_pdf_build",
+            "conf_py_file",
+            "analytics_code",
+            "analytics_disabled",
+            "cdn_enabled",
+            "container_image",
+            "container_mem_limit",
+            "container_time_limit",
+            "install_project",
+            "use_system_packages",
+            "skip",
+            "requirements_file",
+            "python_interpreter",
+            "features",
+            "has_valid_clone",
+            "has_valid_webhook",
+            "show_advertising",
+            "environment_variables",
+            "max_concurrent_builds",
+            "readthedocs_yaml_path",
         )
 
 

readthedocs/__init__.py

@@ -13,6 +13,7 @@
 
 from readthedocs.builds.constants import TAG
 from readthedocs.builds.models import Build, Version
+from readthedocs.projects.constants import PUBLIC
 from readthedocs.projects.models import Project
 from readthedocs.redirects.models import Redirect
 
@@ -55,7 +56,9 @@ def setUp(self):
             main_language_project=None,
             users=[self.me],
             versions=[],
-            external_builds_enabled=False
+            external_builds_enabled=False,
+            external_builds_privacy_level=PUBLIC,
+            privacy_level=PUBLIC,
         )
         for tag in ('tag', 'project', 'test'):
             self.project.tags.add(tag)
@@ -83,6 +86,7 @@ def setUp(self):
             has_pdf=True,
             has_epub=True,
             has_htmlzip=True,
+            privacy_level=PUBLIC,
         )
 
         self.build = fixture.get(
@@ -110,6 +114,8 @@ def setUp(self):
             main_language_project=None,
             users=[self.other],
             versions=[],
+            external_builds_privacy_level=PUBLIC,
+            privacy_level=PUBLIC,
         )
 
         # Make all non-html true so responses are complete
@@ -140,6 +146,8 @@ def _create_new_project(self):
             main_language_project=None,
             users=[self.me],
             versions=[],
+            external_builds_privacy_level=PUBLIC,
+            privacy_level=PUBLIC,
         )
 
     def _create_subproject(self):
@@ -157,6 +165,8 @@ def _create_subproject(self):
             main_language_project=None,
             users=[self.me],
             versions=[],
+            external_builds_privacy_level=PUBLIC,
+            privacy_level=PUBLIC,
         )
         self.project_relationship = self.project.add_subproject(self.subproject)
 

readthedocs/api/v2/serializers.py

@@ -33,6 +33,7 @@ class BuildAdmin(admin.ModelAdmin):
         "date",
         "builder",
         "length",
+        "readthedocs_yaml_path",
         "pretty_config",
     )
     readonly_fields = (

readthedocs/api/v3/tests/mixins.py

@@ -0,0 +1,27 @@
+# Generated by Django 3.2.18 on 2023-04-04 13:03
+
+from django.db import migrations, models
+
+import readthedocs.projects.validators
+
+
+class Migration(migrations.Migration):
+
+    dependencies = [
+        ("builds", "0049_automation_rule_copy"),
+    ]
+
+    operations = [
+        migrations.AddField(
+            model_name="build",
+            name="readthedocs_yaml_path",
+            field=models.CharField(
+                blank=True,
+                default=None,
+                max_length=1024,
+                null=True,
+                validators=[readthedocs.projects.validators.validate_build_config_file],
+                verbose_name="Custom build configuration file path used in this build",
+            ),
+        ),
+    ]

readthedocs/builds/admin.py

@@ -81,6 +81,7 @@
     SPHINX_SINGLEHTML,
 )
 from readthedocs.projects.models import APIProject, Project
+from readthedocs.projects.validators import validate_build_config_file
 from readthedocs.projects.version_handling import determine_stable_version
 
 log = structlog.get_logger(__name__)
@@ -707,6 +708,14 @@ class Build(models.Model):
         null=True,
         blank=True,
     )
+    readthedocs_yaml_path = models.CharField(
+        _("Custom build configuration file path used in this build"),
+        max_length=1024,
+        default=None,
+        blank=True,
+        null=True,
+        validators=[validate_build_config_file],
+    )
 
     length = models.IntegerField(_('Build Length'), null=True, blank=True)