Add pre-commit check for docs cross-references (#5486)

* 🔧 Add check for docs cross-references #5317

* ✨ check jupyter notebooks #5317

* 🔧 add all links in the intersphinx mapping

* 📝 add cross-references

* 📝 remove cross-references

* 📝 add cross-references

* 🔧 exclude files

* Update .pre-commit-config.yaml

* 📝 add ignore file

Co-authored-by: Oriol Abril-Pla <[email protected]>

* ✏️ remove duplicated

* Fix ref in developer_guide.rst

Co-authored-by: Oriol Abril-Pla <[email protected]>

* Fix ref in developer_guide.rst

Co-authored-by: Oriol Abril-Pla <[email protected]>

* Fix reference in developer_guide.rst

Co-authored-by: Oriol Abril-Pla <[email protected]>

* Fix ref in examples.md

Co-authored-by: Oriol Abril-Pla <[email protected]>

* Fix ref in  learn/examples.md

Co-authored-by: Oriol Abril-Pla <[email protected]>

* Fix ref in learn/examples.md

Co-authored-by: Oriol Abril-Pla <[email protected]>

* Fix ref in learn/usage_overview.rst

Co-authored-by: Oriol Abril-Pla <[email protected]>

* 🙈 ignore index.md

* fix arviz cross references in notebooks

Co-authored-by: Oriol Abril-Pla <[email protected]>

Author: symeneses

.pre-commit-config.yaml

@@ -60,3 +60,35 @@ repos:
       entry: from \.[\.\w]* import
       types: [python]
       language: pygrep
+    - id: no-internal-links
+      name: Check no internal links are in the docs
+      description: >-
+        'A quick check for the links in the intersphinx '
+        'mapping inside `docs/source/`'
+        'See docs/source/conf.py for more information.'
+      files: ^docs/source/
+      # Files that should be ignored:
+      # - index.md
+      # - 404.md
+      # - contributing/release_checklist.md
+      # The other files are under development and should be
+      # removed from the list once they are revised.
+      exclude: >
+          (?x)(index.md|
+               404.md|
+               contributing/release_checklist.md|
+               contributing/developer_guide.rst|
+               learn/examples.md)
+      entry: >
+          (?x)(arviz-devs.github.io|
+               aesara.readthedocs.io|
+               aeppl.readthedocs.io|
+               docs.pymc.io|
+               numpy.org/doc|
+               pymc-examples.readthedocs.io|
+               myst-parser.readthedocs.io|
+               myst-nb.readthedocs.io|
+               docs.python.org|
+               xarray.pydata.org)
+      language: pygrep
+      types_or: [markdown, rst, jupyter]

docs/source/Advanced_usage_of_Aesara_in_PyMC.rst

@@ -102,7 +102,7 @@ Good reasons for defining a custom `Op` might be the following:
   and sufficiently parallelizable to make the cost worth it.
   We would definitely like to hear about any such examples.
 
-Aesara has extensive `documentation, <https://aesara.readthedocs.io/en/latest/extending/index.html>`_
+Aesara has extensive :doc:`documentation <aesara:index>`
 about how to write new Ops.
 
 

docs/source/PyMC_and_Aesara.rst

@@ -17,7 +17,7 @@ their posterior probability density up to a constant factor. We then use
 symbolic manipulations of this function to also get access to its gradient.
 
 For a thorough introduction to Aesara see the
-`aesara docs <https://aesara.readthedocs.io/en/latest/>`_,
+:doc:`aesara docs <aesara:index>`,
 but for the most part you don't need detailed knowledge about it as long
 as you are not trying to define new distributions or other extensions
 of PyMC. But let's look at a simple example to get a rough
@@ -81,7 +81,7 @@ like NumPy arrays. Most NumPy functions are available in `aesara.tensor`
 can be found in `at.nlinalg` and `at.slinalg` (the NumPy and SciPy
 operations respectively). Some support for sparse matrices is available
 in `aesara.sparse`. For a detailed overview of available operations,
-see `the aesara api docs <https://aesara.readthedocs.io/en/latest/library/tensor/index.html>`_.
+see :mod:`the aesara api docs <aesara.tensor>`.
 
 A notable exception where Aesara variables do *not* behave like
 NumPy arrays are operations involving conditional execution.

docs/source/api/distributions.rst

@@ -1,3 +1,4 @@
+.. _api_distributions:
 *************
 Distributions
 *************

docs/source/contributing/developer_guide.rst

@@ -7,25 +7,23 @@
 PyMC Developer Guide
 ====================
 
-`PyMC <https://docs.pymc.io/>`__ is a Python package for Bayesian
+:doc:`PyMC <index>` is a Python package for Bayesian
 statistical modeling built on top of
-`Aesara <https://aesara.readthedocs.io/en/latest/index.html>`__. This
+:doc:`Aesara <aesara:index>`. This
 document aims to explain the design and implementation of probabilistic
 programming in PyMC, with comparisons to other PPL like TensorFlow Probability (TFP)
 and Pyro in mind. A user-facing API
-introduction can be found in the `API
-quickstart <https://docs.pymc.io/notebooks/api_quickstart.html>`__. A more accessible, user facing deep introduction can be found in
+introduction can be found in the :ref:`API quickstart <pymc_overview>`. A more accessible, user facing deep introduction can be found in
 `Peadar Coyle's probabilistic programming primer <https://github.com/springcoil/probabilisticprogrammingprimer>`__
 
 Distribution
 ------------
 
 A high-level introduction of ``Distribution`` in PyMC can be found in
-the `documentation <https://docs.pymc.io/Probability_Distributions.html>`__. The source
+the :ref:`documentation <api_distributions>`. The source
 code of the probability distributions is nested under
-`pymc/distributions <https://github.com/pymc-devs/pymc/blob/main/pymc/distributions/>`__,
-with the ``Distribution`` class defined in `distribution.py
-<https://github.com/pymc-devs/pymc/blob/main/pymc/distributions/distribution.py#L23-L44>`__.
+:ref:`pymc/distributions <api_distributions>`,
+with the :class:`~pymc.Distribution` class defined in ``distribution.py``.
 A few important points to highlight in the Distribution Class:
 
 .. code:: python
@@ -66,8 +64,7 @@ functions. Instead, to access a stateless distribution, you need to call
 https://docs.pymc.io/Probability\_Distributions.html#using-pymc-distributions-without-a-model).
 
 With this distinction in mind, we can take a closer look at the
-stateless distribution part of pymc (see distribution api in `doc
-<https://docs.pymc.io/api/distributions.html>`__), which divided into:
+stateless distribution part of pymc (see distribution api in :ref:`doc <api_distributions>`), which divided into:
 
 - Continuous
 
@@ -319,14 +316,13 @@ cannot be transformed.
    correction for inverse mapping). The use cases we considered are limited
    to the set of distributions that are bounded, and the transformation
    maps the bounded set to the real line - see
-   `doc
-   <https://docs.pymc.io/notebooks/api_quickstart.html#Automatic-transforms-of-bounded-RVs>`__.
+   :ref:`API quickstart <pymc_overview#Automatic-transforms-of-bounded-RVs>`.
    However, other transformations are possible.
    In general, PyMC does not provide explicit functionality to transform
    one distribution to another. Instead, a dedicated distribution is
    usually created in order to optimise performance. But getting a
    ``TransformedDistribution`` is also possible (see also in
-   `doc <https://docs.pymc.io/notebooks/api_quickstart.html#Transformed-distributions-and-changes-of-variables>`__):
+   :ref:`doc <pymc_overview##Transformed-distributions-and-changes-of-variables>`):
 
    .. code:: python
 

docs/source/contributing/developer_guide_implementing_distribution.md

@@ -23,8 +23,8 @@ This guide does not attempt to explain the rationale behind the `Distributions`
 {class}`~aesara.tensor.random.op.RandomVariable` are responsible for implementing the random sampling methods, which in version 3 of PyMC used to be one of the standard `Distribution` methods, alongside `logp` and `logcdf`.
 The `RandomVariable` is also responsible for parameter broadcasting and shape inference.
 
-Before creating a new `RandomVariable` make sure that it is not offered in the [Numpy library](https://numpy.org/doc/stable/reference/random/generator.html#distributions).
-If it is, it should be added to the [Aesara library](https://github.com/aesara-devs/aesara) first and then imported into the PyMC library.
+Before creating a new `RandomVariable` make sure that it is not offered in the {mod}`Numpy library <numpy.random>`.
+If it is, it should be added to the {doc}`Aesara library <aesara>` first and then imported into the PyMC library.
 
 In addition, it might not always be necessary to implement a new `RandomVariable`.
 For example if the new `Distribution` is just a special parametrization of an existing `Distribution`.

docs/source/glossary.md

@@ -11,7 +11,7 @@ Functional Programming
   This contrasts with functions or methods that depend on variables that are not explicitly passed as an input (such as accessing `self.variable` inside a method) or that alter the inputs or other state variables in-place, instead of returning new distinct variables as outputs.
 
 Dispatching
-  Choosing which function or method implementation to use based on the type of the input variables (usually just the first variable). For some examples, see Python's documentation for the [singledispatch](https://docs.python.org/3/library/functools.html#functools.singledispatch) decorator.
+  Choosing which function or method implementation to use based on the type of the input variables (usually just the first variable). For some examples, see Python's documentation for the {func}`singledispatch <~functools.singledispatch>` decorator.
 
 [Dispersion](https://en.wikipedia.org/wiki/Statistical_dispersion)
   In statistics, dispersion (also called variability, scatter, or spread) is the extent to which a distribution is stretched or squeezed

docs/source/learn/examples.md

@@ -3,14 +3,11 @@
 
 TODO add "core" notebooks:
 
-- https://docs.pymc.io/en/stable/pymc-examples/examples/generalized_linear_models/GLM-linear.html
-  - https://github.com/pymc-devs/pymc-examples/blob/main/examples/generalized_linear_models/GLM-linear.ipynb
+- {ref}`GLM-linear`
 
-- https://docs.pymc.io/en/stable/pymc-examples/examples/case_studies/multilevel_modeling.html
-  - https://github.com/pymc-devs/pymc-examples/blob/main/examples/case_studies/multilevel_modeling.ipynb
+- {ref}`nb:multilevel_modeling`
 
-- https://docs.pymc.io/en/stable/pymc-examples/examples/diagnostics_and_criticism/model_comparison.html
-  - https://github.com/pymc-devs/pymc-examples/blob/main/examples/diagnostics_and_criticism/model_comparison.ipynb
+- {ref}`model_comparison`
   TODO: Add reference to ArviZ docs since it was taken from there
 
 - https://docs.pymc.io/en/stable/Probability_Distributions.html
@@ -20,11 +17,9 @@ TODO add "core" notebooks:
   update needed: from "pymc and theano" to "pymc and aesara" tutorial (not just auto-renaming from theano to aesara but showing correct use). Very related to migration guide as well, though the migration guide already covers it.
   - https://github.com/pymc-devs/pymc/blob/37ba9a3e3a19b738f48cb30007f4d70c33bdd0f6/docs/source/PyMC_and_Aesara.rst
 
-- https://docs.pymc.io/en/stable/pymc-examples/examples/diagnostics_and_criticism/posterior_predictive.html
-  - https://github.com/pymc-devs/pymc-examples/blob/2002ebd815a199be89b011039906b197bca42361/examples/diagnostics_and_criticism/posterior_predictive.ipynb
+- {ref}`posterior_predictive`
 
-- https://docs.pymc.io/en/stable/Gaussian_Processes.html
-  - https://github.com/pymc-devs/pymc/blob/37ba9a3e3a19b738f48cb30007f4d70c33bdd0f6/docs/source/Gaussian_Processes.rst
+- {mod}`pymc.gp`
 
 TODO link to the pymc-examples
 TODO add categories to all notebooks

docs/source/learn/examples/posterior_predictive.ipynb

@@ -6,7 +6,7 @@ Usage Overview
 
 For a detailed overview of building models in PyMC, please read the appropriate sections in the rest of the documentation. For a flavor of what PyMC models look like, here is a quick example.
 
-First, let's import PyMC and `ArviZ <https://arviz-devs.github.io/arviz/>`__ (which handles plotting and diagnostics):
+First, let's import PyMC and :doc:`ArviZ <arviz:index>` (which handles plotting and diagnostics):
 
 ::