Docs: Fix undeclared labels after refactor + fix root causes

docs/conf.py

@@ -87,9 +87,16 @@
     "rtd-dev": ("https://dev.readthedocs.io/en/latest/", None),
     "jupyter": ("https://docs.jupyter.org/en/latest/", None),
 }
-# Redundant in Sphinx 5.0
+
+# Intersphinx: Do not try to resolve unresolved labels that aren't explicitly prefixed.
+# The default setting for intersphinx_disabled_reftypes can cause some pretty bad
+# breakage because we have rtd and rtd-dev stable versions in our mappings.
+# Hence, if we refactor labels, we won't see broken references, since the
+# currently active stable mapping keeps resolving.
+# Recommending doing this on all projects with Intersphinx.
 # https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html#confval-intersphinx_disabled_reftypes
-intersphinx_disabled_reftypes = ["std:doc"]
+intersphinx_disabled_reftypes = ["*"]
+
 myst_enable_extensions = [
     "deflist",
 ]

docs/user/analytics.rst

@@ -19,7 +19,7 @@ and then click on :guilabel:`Traffic Analytics`.
 
    Traffic analytics demo
 
-You can also access to analytics data from :ref:`search results <server-side-search/index:Search Analytics>`.
+You can also access analytics data from :doc:`search results </guides/search-analytics>`.
 
 .. note::
 

docs/user/automatic-redirects.rst

@@ -89,4 +89,4 @@ Questionable practice 🟡
      Users will be aware that the documentation **project** still exists but has changed.
 
      The default Read the Docs 404 page is designed to be helpful,
-     but you can also design your own, see :ref:`hosting:Custom Not Found (404) Pages`.
+     but you can also design your own, see :doc:`/reference/404-not-found`.

docs/user/automation-rules.rst

@@ -82,7 +82,7 @@ Make version private
 Set version as default
   Sets the version as the :term:`default version`.
   It also activates and builds the version.
-  See :ref:`automatic-redirects:Root URL`.
+  See :ref:`root_url_redirect`.
 
 Delete version
   When a branch or tag is deleted from your repository,

docs/user/builds.rst

@@ -50,7 +50,7 @@ The build process includes the following jobs:
 
    .. tip::
 
-    We strongly recommend :ref:`pinning all the versions <guides/reproducible-builds:pinning dependencies>` required to build the documentation to avoid unexpected build errors.
+    We strongly recommend :doc:`pinning all the versions </guides/reproducible-builds>` required to build the documentation to avoid unexpected build errors.
 
 :build:
 

docs/user/canonical-urls.rst

@@ -34,7 +34,7 @@ The canonical URL takes the following into account:
 
 * The default version of your project (usually "latest" or "stable").
 * The canonical :doc:`custom domain </custom-domains>` if you have one,
-  otherwise the default :ref:`subdomain <hosting:subdomain support>` will be used.
+  otherwise the default :ref:`subdomain <default_subdomain>` will be used.
 
 For example, if you have a project named ``example-docs``
 with a custom domain ``https://docs.example.com``,

docs/user/commercial/organizations.rst

@@ -30,7 +30,7 @@ The best way to think about this relationship is:
 .. warning::
 
    Owners, Members and Teams behave differently if you are using
-   :ref:`SSO with VCS provider (GitHub, Bitbucket or GitLab) <commercial/single-sign-on:SSO with VCS provider (GitHub, Bitbucket or GitLab)>`
+   :ref:`sso_git_provider`.
 
 Team Types
 ~~~~~~~~~~

docs/user/commercial/single-sign-on.rst

@@ -15,6 +15,7 @@ Currently, we support two different types of single sign-on:
 
 Users can log out by using the :ref:`Log Out <versions:Logging out>` link in the RTD flyout menu.
 
+.. _sso_git_provider:
 
 Single Sign-on with GitHub, Bitbucket, or GitLab
 ------------------------------------------------

docs/user/config-file/v2.rst

@@ -429,7 +429,7 @@ We don't currently support PPA's or other custom repositories.
 .. note::
 
    When possible avoid installing Python packages using apt (``python3-numpy`` for example),
-   :ref:`use pip or Conda instead <guides/reproducible-builds:pinning dependencies>`.
+   :doc:`use pip or conda instead </guides/reproducible-builds>`.
 
 
 build.jobs

docs/user/custom-domains.rst

@@ -5,7 +5,11 @@ You can serve your documentation project from your own domain,
 for instance ``docs.example.com``.
 This is great for maintaining a consistent brand for your product and its documentation.
 
-*By default*, your documentation is served from a Read the Docs :ref:`subdomain <hosting:subdomain support>` using the project's :term:`slug`:
+.. _default_subdomain:
+
+.. rubric:: Default subdomains
+
+*By default*, your documentation is served from a Read the Docs *subdomain* using the project's :term:`slug`:
 
 * ``<slug>.readthedocs.io`` for |org_brand|
 * ``<slug>.readthedocs-hosted.com`` for |com_brand|.
@@ -73,7 +77,7 @@ In most cases, the ``CNAME`` record is used.
 This is all that's needed for a web browser to resolve your domain name to Read the Docs' servers and for our servers to match the right documentation project.
 You can find step-by-step instructions for this in :doc:`/guides/custom-domains`.
 
-Read the Docs uses a :ref:`hosting:Content Delivery Network (CDN)` to host and serve your documentation pages.
+Read the Docs uses a :doc:`/reference/cdn` to host and serve your documentation pages.
 This final step isn't changed by a custom domain
 and therefore the response times are unaffected as the delivery of resources happens through the same CDN setup.
 

docs/user/faq.rst

@@ -38,8 +38,8 @@ you can use an important word or message from the error to search for a solution
       Common errors and solutions for build failures.
 
    Other FAQ entries
-      * :ref:`faq:My documentation requires additional dependencies`
-      * :ref:`faq:I get import errors on libraries that depend on C modules`
+      * :ref:`faq:How do I add additional software dependencies for my documentation?`
+      * :ref:`faq:why do i get import errors from libraries depending on c modules?`
 
 
 .. Old reference
@@ -61,9 +61,6 @@ Make sure you have one of the following files at the top level of your documenta
    you can navigate to a specific page that you know is part of the documentation build,
    for example `/en/latest/README.html`.
 
-.. Old reference
-.. _I get import errors on libraries that depend on C modules:
-
 Why do I get import errors from libraries depending on C modules?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -101,7 +98,7 @@ and start suggesting the latest (or newer) one.
 
 To accomplish this, you can add a ``robots.txt`` file to your documentation's root so it ends up served at the root URL of your project
 (for example, https://yourproject.readthedocs.io/robots.txt).
-We have documented how to set this up in our :ref:`hosting:Custom robots.txt Pages` docs.
+We have documented how to set this up in :doc:`/reference/robots`.
 
 
 How do I change the version slug of my project?
@@ -126,9 +123,6 @@ We also keep an up-to-date :doc:`changelog </changelog>`.
 Additional features and configuration
 -------------------------------------
 
-.. Old reference
-.. _My documentation requires additional dependencies:
-
 How do I add additional software dependencies for my documentation?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -155,18 +149,15 @@ You can also set your project documentation to install your Python project itsel
      Reference for adding Debian packages with apt for the Ubuntu-based builders
 
    Other FAQ entries
-      * :ref:`faq:My documentation requires additional dependencies`
-      * :ref:`faq:I get import errors on libraries that depend on C modules`
-
+      * :ref:`faq:How do I add additional software dependencies for my documentation?`
+      * :ref:`faq:Why do I get import errors from libraries depending on C modules?`
 
-.. Old reference
-.. _My project requires some additional settings:
 
 Can I have access to additional features or settings?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 If this is just a dependency issue,
-see :ref:`faq:My documentation requires additional dependencies`.
+see :ref:`faq:How do I add additional software dependencies for my documentation?`.
 
 Read the Docs offers some settings (feature flags) which can be used for a variety of purposes.
 To enable these settings,
@@ -357,7 +348,7 @@ and as a result, it tends to look a bit better with the default theme.
 .. note::
 
    To use these extensions you need to specify the dependencies on your project
-   by following this :ref:`guide <guides/reproducible-builds:using a configuration file>`.
+   by following this :doc:`guide </guides/reproducible-builds>`.
 
 
 I need to install a package in a environment with pinned versions

docs/user/guides/automation-rules.rst

@@ -36,7 +36,7 @@ In the :guilabel:`Automation Rule` form, you need to fill in 4 fields:
 
    * :ref:`Any version <automation-rules:Predefined matches>` matches all values.
    * :ref:`SemVer versions <automation-rules:Predefined matches>` matches only values that have the `SemVer`_ format.
-   * :ref:`Custom match <automation-rules:User defined matches>` matches your own pattern (entered below).
+   * :ref:`Custom match <automation-rules:Custom matches>` matches your own pattern (entered below).
      If you choose this option,
      a field :guilabel:`Custom match` will automatically appear below the drop-down where you can add a regular expression in `Python regex format`_.
 

docs/user/guides/deprecating-content.rst

@@ -29,7 +29,7 @@ For example, if you have the following versions and want to deprecate v1.
 
 For cases like this you can *hide* a version.
 Hidden versions won't be listed in the versions menu of your docs,
-and they will be listed in a :ref:`robots.txt file <hosting:custom robots.txt pages>`
+and they will be listed in a :doc:`robots.txt file </reference/robots>`
 to stop search engines of showing results for that version.
 
 Users can still see all versions in the dashboard of your project.
@@ -57,7 +57,7 @@ For example, if you have documentation about two APIs and you want to deprecate
 A simple way is just adding a warning at the top of the page,
 this will warn users visiting that page,
 but it won't stop users from being redirected to that page from search results.
-You can add an entry of that page in a :ref:`custom robots.txt <hosting:custom robots.txt pages>` file
+You can add an entry of that page in a :doc:`custom robots.txt </reference/robots>` file
 to avoid search engines of showing those results. For example::
 
    # robots.txt

docs/user/guides/importing-private-repositories.rst

@@ -122,4 +122,4 @@ To build your documentation on every commit,
 you'll need to manually add a webhook, see :doc:`/integrations`.
 If you are using an unsupported integration,
 you may need to setup a custom integration
-using our :ref:`generic webhook <integrations:using the generic api integration>`.
+using our :ref:`generic webhook <webhook-integration-generic>`.

docs/user/guides/intersphinx.rst

@@ -51,6 +51,14 @@ And use the ``intersphinx_mapping`` configuration to indicate the name and link
        "sphinx": ("https://www.sphinx-doc.org/en/master/", None),
    }
 
+   # We recommend adding the following config value.
+   # Sphinx defaults to automatically resolve *unresolved* labels using all your Intersphinx mappings.
+   # This behavior has unintended side-effects, namely that documentations local references can
+   # suddenly resolve to an external location.
+   # See also:
+   # https://www.sphinx-doc.org/en/master/usage/extensions/intersphinx.html#confval-intersphinx_disabled_reftypes
+   intersphinx_disabled_reftypes = ["*"]
+
 .. note::
 
    If you are using Read the Docs' subprojects, you also need to enable the Intersphinx extension on each of the subprojects.
@@ -95,6 +103,7 @@ Result:
 
       python -m sphinx.ext.intersphinx https://www.sphinx-doc.org/en/master/objects.inv
 
+
 Intersphinx in Read the Docs
 ----------------------------
 

docs/user/guides/pull-requests.rst

@@ -74,7 +74,7 @@ Build status is not being reported to your VCS provider
    your account with your VCS provider.
 
 .. seealso::
-   - :ref:`integrations:Debugging webhooks`
+   - :ref:`guides/git-integrations:Debugging webhooks`
    - :ref:`github-permission-troubleshooting`
 
 .. _OAuth App: https://github.com/settings/applications

docs/user/guides/technical-docs-seo-guide.rst

@@ -79,8 +79,8 @@ Best practices for documentation SEO
 Once a crawler or spider finds your site, it will follow links and redirects
 in an attempt to find any and all pages on your site.
 While there are a few ways to guide the search engine in its crawl
-for example by using a :ref:`sitemap <guides/technical-docs-seo-guide:Use a sitemap.xml file>`
-or a :ref:`robots.txt file <guides/technical-docs-seo-guide:Use a robots.txt file>`
+for example by using a :ref:`sitemap <seo_sitemap.xml>`
+or a :ref:`robots.txt file <seo_robots.txt>`
 which we'll discuss shortly,
 the most important thing is making sure the spider can follow links on your site
 and get to all your pages.
@@ -182,6 +182,8 @@ When using images, make sure to set the image alt text or set a caption on figur
 .. _md-in-html: https://python-markdown.github.io/extensions/md_in_html/
 
 
+.. _seo_redirects:
+
 Redirects ✅️
 ~~~~~~~~~~~~~
 
@@ -196,6 +198,10 @@ Read the Docs supports a few different kinds of :doc:`user defined redirects </u
 that should cover all the different cases such as redirecting a certain page for all project versions,
 or redirecting one version to another.
 
+.. seealso:
+
+   :doc:`/automatic-redirects`
+      Following best practices to avoid broken links is great for search engine ranking.
 
 Canonical URLs ✅️
 ~~~~~~~~~~~~~~~~~~
@@ -212,6 +218,8 @@ under :guilabel:`Admin` > :guilabel:`Domains`
 in the Read the Docs dashboard.
 
 
+.. _seo_robots.txt:
+
 Use a robots.txt file ✅️
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -230,6 +238,7 @@ that is written to your documentation root on your default branch/version.
 See the `Google's documentation on robots.txt <https://support.google.com/webmasters/answer/6062608>`_
 for additional details.
 
+.. _seo_sitemap.xml:
 
 Use a sitemap.xml file ✅️
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -242,7 +251,7 @@ or any alternate language versions of a page.
 
 Read the Docs generates a sitemap for you that contains the last time
 your documentation was updated as well as links to active versions, subprojects, and translations your project has.
-We have a small separate guide on :ref:`sitemaps <hosting:Sitemaps>`.
+We have a small separate guide on :doc:`sitemaps </reference/sitemaps>`.
 
 See the `Google docs on building a sitemap <https://support.google.com/webmasters/answer/183668>`_.
 
@@ -301,7 +310,7 @@ Some of the most valuable feedback these provide include:
 * Google and Bing will show pages that were previously indexed that now give a 404
   (or more rarely a 500 or other status code).
   These will remain in the index for some time but will eventually be removed.
-  This is a good opportunity to create a :ref:`redirect <guides/technical-docs-seo-guide:Redirects>`.
+  This is a good opportunity to create a :ref:`redirect <seo_redirects>`.
 * These tools will show any crawl issues with your documentation.
 * Search Console and Webmaster Tools will highlight security issues found
   or if Google or Bing took action against your site because they believe it is spammy.

docs/user/intro/getting-started-with-mkdocs.rst

@@ -65,7 +65,7 @@ you can start using Read the Docs by :doc:`importing your docs </intro/import-gu
 
 .. warning::
 
-   We strongly recommend to :ref:`pin the MkDocs version <guides/reproducible-builds:pinning dependencies>`
+   We strongly recommend to :doc:`pin the MkDocs version </guides/reproducible-builds>`
    used for your project to build the docs to avoid potential future incompatibilities.
 
 Get inspired!

docs/user/intro/getting-started-with-sphinx.rst

@@ -72,7 +72,7 @@ by :doc:`importing your docs </intro/import-guide>`.
 
 .. warning::
 
-   We strongly recommend to :ref:`pin the Sphinx version <guides/reproducible-builds:pinning dependencies>`
+   We strongly recommend to :doc:`pin the Sphinx version </guides/reproducible-builds>`
    used for your project to build the docs to avoid potential future incompatibilities.
 
 .. _this template: https://www.writethedocs.org/guide/writing/beginners-guide-to-docs/#id1

docs/user/intro/import-guide.rst

@@ -51,13 +51,12 @@ Add an optional homepage URL and some tags, and then click **Next**.
 Once your project is created, you'll need to manually configure the repository
 webhook if you would like to have new changes trigger builds for your
 project on Read the Docs. Go to your project's :guilabel:`Admin` > :guilabel:`Integrations` page to
-configure a new webhook, or see :ref:`our steps for webhook creation <integrations:Integration Creation>`
-for more information on this process.
+configure a new webhook.
 
 .. seealso::
 
    :doc:`/guides/git-integrations`
-      Once you have imported your git project, use this guide to manually set up basic and additional integration.
+      Once you have imported your git project, use this guide to manually set up basic and additional *webhook* integration.
 
 .. note::
     The ``Admin`` page can be found at ``https://readthedocs.org/dashboard/<project-slug>/edit/``.
@@ -82,7 +81,7 @@ You can configure these settings in a ``.readthedocs.yaml`` file.
 See our :doc:`/config-file/index` docs for more details.
 
 It is also important to note that the default version of Sphinx is ``v1.8.5``.
-We recommend to set the version your project uses :ref:`explicitily <guides/reproducible-builds:don't rely on implicit dependencies>`.
+We recommend to set the version your project uses :doc:`explicitily with pinned dependencies </guides/reproducible-builds>`.
 
 Read the Docs will host multiple versions of your code. You can read more about
 how to use this well on our :doc:`/versions` page.

docs/user/reference/404-not-found.rst

@@ -45,7 +45,7 @@ We have examples for some of the most popular tools below.
 
       MkDocs automatically generates a ``404.html`` which Read the Docs will use.
       However, assets will not be loaded correctly unless you define the `site_url`_ configuration value as your site's
-      :ref:`canonical base URL <canonical-urls:MkDocs>`.
+      :ref:`canonical base URL <guides/canonical-urls:MkDocs>`.
 
 .. _sphinx-notfound-page: https://pypi.org/project/sphinx-notfound-page
 .. _html_extra_path: https://www.sphinx-doc.org/en/master/usage/configuration.html#confval-html_extra_path

docs/user/reference/analytics.rst

@@ -27,7 +27,7 @@ Learn more in :doc:`/analytics`.
 Search Analytics
 ----------------
 
-When someone visits your documentation and uses the built-in :ref:`server-side search <server-side-search:Server Side Search>` feature,
+When someone visits your documentation and uses the built-in :doc:`server side search </server-side-search/index>` feature,
 Read the Docs will collect analytics on their search term.
 
 Those are aggregated into a simple view of the

docs/user/reference/environment-variables.rst

@@ -39,7 +39,7 @@ All :doc:`build processes </builds>` have the following environment variables au
 
 .. envvar:: READTHEDOCS_VIRTUALENV_PATH
 
-    Path for the :ref:`virtualenv that was created for this build <builds:Understanding what's going on>`.
+    Path for the :doc:`virtualenv that was created for this build </builds>`.
     Only exists for builds using Virtualenv and not Conda.
 
     :Example: ``/home/docs/checkouts/readthedocs.org/user_builds/project/envs/version``

docs/user/tutorial/index.rst

@@ -625,7 +625,7 @@ Browsing Search Analytics
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Apart from traffic analytics, Read the Docs also offers the possibility
-to inspect :ref:`what search terms your readers use <server-side-search/index:Search Analytics>`
+to inspect :doc:`what search terms your readers use </guides/search-analytics>`
 on your documentation.
 This can inform decisions on what areas to reinforce,
 or what parts of your project are less understood or more difficult to find.

docs/user/user-defined-redirects.rst

@@ -63,6 +63,7 @@ Another way to handle this is the ``latest`` version.
 You can set your ``latest`` version to a specific version and just always link to ``latest``.
 You can reach this page by going to https://docs.readthedocs.io/en/latest/automatic-redirects.html.
 
+.. _root_url_redirect:
 
 Root URL redirect at ``/``
 ~~~~~~~~~~~~~~~~~~~~~~~~~~