Merge pull request #1049 from python-jsonschema/referencing

Replace `$ref` & `$dynamicRef` support / `RefResolver` with the new referencing library

Author: Julian

.github/workflows/ci.yml

@@ -38,18 +38,6 @@ jobs:
             toxenv: pypy3-formatnongpl-build
           - name: pypy-3.9
             toxenv: pypy3-formatnongpl-tests
-          - name: 3.7
-            toxenv: py37-noextra-build
-          - name: 3.7
-            toxenv: py37-noextra-tests
-          - name: 3.7
-            toxenv: py37-format-build
-          - name: 3.7
-            toxenv: py37-format-tests
-          - name: 3.7
-            toxenv: py37-formatnongpl-build
-          - name: 3.7
-            toxenv: py37-formatnongpl-tests
           - name: 3.8
             toxenv: py38-noextra-build
           - name: 3.8

CHANGELOG.rst

@@ -1,3 +1,10 @@
+v4.18.0
+=======
+
+* ``jsonschema.RefResolver`` is now deprecated in favor of the new `referencing library <https://github.com/python-jsonschema/referencing/>`_.
+  ``referencing`` will begin in beta, but already is more compliant than the existing ``$ref`` support.
+  Please file issues on the ``referencing`` tracker if there is functionality missing from it.
+
 v4.17.3
 =======
 

README.rst

@@ -59,8 +59,7 @@ It can also be used from the command line by installing `check-jsonschema <https
 Features
 --------
 
-* Partial support for `Draft 2020-12 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft202012Validator>`_ and `Draft 2019-09 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft201909Validator>`_, except for ``dynamicRef`` / ``recursiveRef`` and ``$vocabulary`` (in-progress).
-  Full support for `Draft 7 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft7Validator>`_, `Draft 6 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft6Validator>`_, `Draft 4 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft4Validator>`_ and `Draft 3 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft3Validator>`_
+* Full support for `Draft 2020-12 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft202012Validator>`_, `Draft 2019-09 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft201909Validator>`_, `Draft 7 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft7Validator>`_, `Draft 6 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft6Validator>`_, `Draft 4 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft4Validator>`_ and `Draft 3 <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/validators/#jsonschema.validators.Draft3Validator>`_
 
 * `Lazy validation <https://python-jsonschema.readthedocs.io/en/latest/api/jsonschema/protocols/#jsonschema.protocols.Validator.iter_errors>`_ that can iteratively report *all* validation errors.
 

docs/api/jsonschema/validators/index.rst

@@ -4,3 +4,4 @@
 .. automodule:: jsonschema.validators
    :members:
    :undoc-members:
+   :private-members: _RefResolver

docs/conf.py

@@ -38,29 +38,36 @@
 
 # See sphinx-doc/sphinx#10785
 _TYPE_ALIASES = {
-    "jsonschema._format._F",  # format checkers
+    "jsonschema._format._F": ("data", "_F"),
 }
 
 
-def _resolve_type_aliases(app, env, node, contnode):
-    if (
-        node["refdomain"] == "py"
-        and node["reftype"] == "class"
-        and node["reftarget"] in _TYPE_ALIASES
-    ):
+def _resolve_broken_refs(app, env, node, contnode):
+    if node["refdomain"] != "py":
+        return
+
+    if node["reftarget"].startswith("referencing."):  # :( :( :( :( :(
+        node["reftype"] = "data"
+        from sphinx.ext import intersphinx
+        return intersphinx.resolve_reference_in_inventory(
+            env, "referencing", node, contnode,
+        )
+
+    kind, target = _TYPE_ALIASES.get(node["reftarget"], (None, None))
+    if kind is not None:
         return app.env.get_domain("py").resolve_xref(
             env,
             node["refdoc"],
             app.builder,
-            "data",
-            node["reftarget"],
+            kind,
+            target,
             node,
             contnode,
         )
 
 
 def setup(app):
-    app.connect("missing-reference", _resolve_type_aliases)
+    app.connect("missing-reference", _resolve_broken_refs)
 
 
 # = Builders =
@@ -116,6 +123,7 @@ def entire_domain(host):
 
 intersphinx_mapping = {
     "python": ("https://docs.python.org/3", None),
+    "referencing": ("https://referencing.readthedocs.io/en/stable/", None),
     "ujs": ("https://json-schema.org/understanding-json-schema/", None),
 }
 

docs/errors.rst

@@ -273,6 +273,7 @@ error objects.
 
 .. testcode::
 
+    from jsonschema.exceptions import ErrorTree
     tree = ErrorTree(v.iter_errors(instance))
 
 As you can see, `jsonschema.exceptions.ErrorTree` takes an

docs/faq.rst

@@ -85,9 +85,10 @@ The JSON object ``{}`` is simply the Python `dict` ``{}``, and a JSON Schema lik
 
    The :kw:`$ref` keyword is a single notable exception.
 
-   Specifically, in the case where `jsonschema` is asked to `resolve a remote reference <jsonschema.validators.RefResolver>`, it has no choice but to assume that the remote reference is serialized as JSON, and to deserialize it using the `json` module.
+   Specifically, in the case where `jsonschema` is asked to resolve a remote reference, it has no choice but to assume that the remote reference is serialized as JSON, and to deserialize it using the `json` module.
 
    One cannot today therefore reference some remote piece of YAML and have it deserialized into Python objects by this library without doing some additional work.
+   See `Resolving References to Schemas Written in YAML <referencing:Resolving References to Schemas Written in YAML>` for details.
 
 In practice what this means for JSON-like formats like YAML and TOML is that indeed one can generally schematize and then validate them exactly as if they were JSON by simply first deserializing them using libraries like ``PyYAML`` or the like, and passing the resulting Python objects into functions within this library.
 
@@ -99,60 +100,6 @@ In such cases one is recommended to first pre-process the data such that the res
 In the previous example, if the desired behavior is to transparently coerce numeric properties to strings, as Javascript might, then do the conversion explicitly before passing data to this library.
 
 
-How do I configure a base URI for $ref resolution using local files?
---------------------------------------------------------------------
-
-`jsonschema` supports loading schemas from the filesystem.
-
-The most common mistake when configuring a `jsonschema.validators.RefResolver`
-to retrieve schemas from the local filesystem is to give it a base URI
-which points to a directory, but forget to add a trailing slash.
-
-For example, given a directory ``/tmp/foo/`` with ``bar/schema.json``
-within it, you should use something like:
-
-.. code-block:: python
-
-    from pathlib import Path
-
-    import jsonschema.validators
-
-    path = Path("/tmp/foo")
-    resolver = jsonschema.validators.RefResolver(
-        base_uri=f"{path.as_uri()}/",
-        referrer=True,
-    )
-    jsonschema.validate(
-        instance={},
-        schema={"$ref": "bar/schema.json"},
-        resolver=resolver,
-    )
-
-where note:
-
-    * the base URI has a trailing slash, even though
-      `pathlib.PurePath.as_uri` does not add it!
-    * any relative refs are now given relative to the provided directory
-
-If you forget the trailing slash, you'll find references are resolved a
-directory too high.
-
-You're likely familiar with this behavior from your browser. If you
-visit a page at ``https://example.com/foo``, then links on it like
-``<a href="./bar">`` take you to ``https://example.com/bar``, not
-``https://example.com/foo/bar``. For this reason many sites will
-redirect ``https://example.com/foo`` to ``https://example.com/foo/``,
-i.e. add the trailing slash, so that relative links on the page will keep the
-last path component.
-
-There are, in summary, 2 ways to do this properly:
-
-* Remember to include a trailing slash, so your base URI is
-  ``file:///foo/bar/`` rather than ``file:///foo/bar``, as shown above
-* Use a file within the directory as your base URI rather than the
-  directory itself, i.e. ``file://foo/bar/baz.json``, which will of course
-  cause ``baz.json`` to be removed while resolving relative URIs
-
 Why doesn't my schema's default property set the default on my instance?
 ------------------------------------------------------------------------
 

docs/index.rst

@@ -12,6 +12,7 @@ Contents
 
     validate
     errors
+    referencing
     creating
     faq
     api/index