Merge pull request #1555 from json-schema-org/gregsdennis/dynamic-ref

clean up `$dynamic*`

Author: gregsdennis

adr/2024-11-2-assertion-format.md

@@ -1,78 +1,101 @@
-# [short title of solved problem and solution]
+# `format` as an assertion
 
-* Status: proposed
+- Status: proposed
   <!-- will update below to only those who participated in the vote -->
-* Deciders: @gregsdennis @jdesrosiers @julian @jviotti @mwadams @karenetheridge @relequestual
-* Date: 2024-11-02
-* Technical Story: https://github.com/json-schema-org/json-schema-spec/issues/1520
-* Voting issue: https://github.com/json-schema-org/TSC/issues/19
-    For - @gregsdennis @jdesrosiers @jviotti @mwadams @karenetheridge
-    Neutral - @relequestual
-    Against - @julian
+- Deciders: @gregsdennis @jdesrosiers @julian @jviotti @mwadams @karenetheridge
+  @relequestual
+- Date: 2024-11-02
+- Technical Story: <https://github.com/json-schema-org/json-schema-spec/issues/1520>
+- Voting issue: <https://github.com/json-schema-org/TSC/issues/19>
+  - For - @gregsdennis @jdesrosiers @jviotti @mwadams @karenetheridge
+  - Neutral - @relequestual
+  - Against - @julian
 
 ## Context and Problem Statement
 
 There's a long and sticky history around format.
 
 1. Going back all the way to Draft 01, format has never required validation.
-2. Whether to support format validation has always been the decision of the implementation.
-3. The extent to which formats are validated has also been the decision of the implementation.
-
-The result of all of this is that implementation support for validation has been spotty at best. Despite the JSON Schema specs referencing very concretely defined formats (by referencing other specs), implementations that do support validation don't all support each format equally. This has been the primary driving force behind keeping format as an opt-in validation.
-
-With 2019-09, we decided that it was time to give the option of format validation to the schema author. They could enable validation by using a meta-schema which listed the Format Vocabulary with a true value, which meant, "format validation is required to process this schema."
-
-In 2020-12, we further refined this by offering two separate vocabularies, one that treats the keyword as an annotation and one that treats it as an assertion. The argument was that the behavior of a keyword shouldn't change based on whether the vocabulary was required or not.
-
-However, the fact remains that our users consistently report (via questions in Slack, GitHub, and StackOverflow) that they expect format to validate. (The most recent case I can think of was only last week, in .Net's effort to build a short-term solution for schema generation from types.)
+1. Whether to support format validation has always been the decision of the
+   implementation.
+1. The extent to which formats are validated has also been the decision of the
+   implementation.
+
+The result of all of this is that implementation support for validation has been
+spotty at best. Despite the JSON Schema specs referencing very concretely
+defined formats (by referencing other specs), implementations that do support
+validation don't all support each format equally. This has been the primary
+driving force behind keeping format as an opt-in validation.
+
+With 2019-09, we decided that it was time to give the option of format
+validation to the schema author. They could enable validation by using a
+meta-schema which listed the Format Vocabulary with a true value, which meant,
+"format validation is required to process this schema."
+
+In 2020-12, we further refined this by offering two separate vocabularies, one
+that treats the keyword as an annotation and one that treats it as an assertion.
+The argument was that the behavior of a keyword shouldn't change based on
+whether the vocabulary was required or not.
+
+However, the fact remains that our users consistently report (via questions in
+Slack, GitHub, and StackOverflow) that they expect format to validate. (The most
+recent case I can think of was only last week, in .Net's effort to build a
+short-term solution for schema generation from types.)
 
 Due to this consistency in user expectations, we have decided to:
 
 1. make format an assertion keyword, and
-2. strictly enforce it by moving the appropriate tests into the required section of the Test Suite and building them more completely.
+1. strictly enforce it by moving the appropriate tests into the required section
+   of the Test Suite and building them more completely.
 
 ## Decision Drivers
 
-* User expectation
-* Current behavior
-* Historical context
-* Disparity of current implementation support vs the proposed requirements
+- User expectation
+- Current behavior
+- Historical context
+- Disparity of current implementation support vs the proposed requirements
 
 ## Considered Options
 
 ### `format` remains an annotation keyword by default
 
-This is the current state. The primary benefit is that we don't need to make a breaking change.
-
-The primary downside is that the current system of (1) configuring the tool or (2) incluing the `format-assertion` vocab[^1] is confusing for many and doesn't align with user expectations.
+This is the current state. The primary benefit is that we don't need to make a
+breaking change.
 
-[^1] The `format-assertion` vocabulary will no longer be an option since we have demoted vocabularies to a proposal for the stable release.  This leaves tool configuration as the only option to enable `format` validation.
+The primary downside is that the current system of (1) configuring the tool or
+(2) incluing the `format-assertion` vocab[^1] is confusing for many and doesn't
+align with user expectations.
 
 ### `format` becomes an assertion keyword by default
 
-We change the spec to require `format` validation.  Furthermore:
+We change the spec to require `format` validation. Furthermore:
 
-* Implementations SHOULD support `format` with the defined values
-* Implementations MAY support others, but only by explicit config
-* Implementations MUST refuse to process a schema that contains an unsupported format
+- Implementations SHOULD support `format` with the defined values
+- Implementations MAY support others, but only by explicit config
+- Implementations MUST refuse to process a schema that contains an unsupported
+  format
 
 ## Decision Outcome
 
-The TSC has decided via vote (see voting issue above) that we should change `format` to act as an assertion by default, in line with option (2).
+The TSC has decided via vote (see voting issue above) that we should change
+`format` to act as an assertion by default, in line with option (2).
 
 ### Positive Consequences <!-- optional -->
 
-* Aligns with user expectations.
-* Users are still able to have purely annotative behavior through use of something like `x-format`.
-* Increased consistency for `format` validation across implementations.
+- Aligns with user expectations.
+- Users are still able to have purely annotative behavior through use of
+  something like `x-format`.
+- Increased consistency for `format` validation across implementations.
 
 ### Negative Consequences <!-- optional -->
 
-* This is a breaking change, which means that we will likely have to re-educate the users who correctly treat it as an annotation.
-* Older schemas which do not specify a version (`$schema`) may change their validation outcome.
-* The burden on implementations will be greater since format validation was previously optional.
-
-## Links <!-- optional -->
+- This is a breaking change, which means that we will likely have to re-educate
+  the users who correctly treat it as an annotation.
+- Older schemas which do not specify a version (`$schema`) may change their
+  validation outcome.
+- The burden on implementations will be greater since format validation was
+  previously optional.
 
-* [Link type] [Link to ADR] <!-- example: Refined by [ADR-0005](0005-example.md) -->
-* … <!-- numbers of links can vary -->
+[^1]: The `format-assertion` vocabulary will no longer be an option since we
+have demoted vocabularies to a proposal for the stable release. This leaves tool
+configuration as the only option to enable `format` validation.

specs/jsonschema-core.md

@@ -546,8 +546,8 @@ determined at runtime.
 
 While custom identifier keywords are possible, extension designers should take
 care not to disrupt the functioning of core keywords. For example, the
-`$dynamicAnchor` keyword in this specification limits its IRI resolution effects
-to the matching `$dynamicRef` keyword, leaving the behavior of `$ref`
+`$dynamicAnchor` keyword in this specification limits its resolution behavior
+to matching `$dynamicRef` keywords, leaving the behavior of `$ref`
 undisturbed.
 
 ### Applicators {#applicators}
@@ -947,17 +947,18 @@ The `$anchor` and `$dynamicAnchor` keywords are used to specify such fragments.
 They are identifier keywords that can only be used to create plain name
 fragments, rather than absolute IRIs as seen with `$id`.
 
-The base IRI to which the resulting fragment is appended is the canonical IRI of
-the schema resource containing the `$anchor` or `$dynamicAnchor` in question.
-As discussed in the previous section, this is either the nearest `$id` in the
-same or parent schema object, or the base IRI for the document as determined
-according to [RFC 3987](#rfc3987) and [RFC 3986](#rfc3986).
+`$anchor` defines a reference target for `$ref`. The fragment defined by this
+keyword is appended to the IRI of the schema resource containing it. As
+discussed in {{id-keyword}}, this is either the nearest `$id` in the same or an
+ancestor schema object, or the base IRI for the document as determined according
+to [RFC 3987](#rfc3987) and [RFC 3986](#rfc3986).
 
-Separately from the usual usage of IRIs, `$dynamicAnchor` indicates that the
-fragment is an extension point when used with the `$dynamicRef` keyword. This
-low-level, advanced feature makes it easier to extend recursive schemas such as
-the meta-schemas, without imposing any particular semantics on that extension.
-See the section on [`$dynamicRef`](#dynamic-ref) for details.
+In contrast, `$dynamicAnchor` operates independently of resource IRIs and is
+instead dependent on the dynamic scope of the evaluation. `$dynamicAnchor`
+defines a reference target for the `$dynamicRef` keyword. This advanced feature
+makes it easier to extend recursive schemas such as the meta-schemas, without
+imposing any particular semantics on that extension. See {{dynamic-ref}} for
+details.
 
 In most cases, the normal fragment behavior both suffices and is more intuitive.
 Therefore it is RECOMMENDED that `$anchor` be used to create plain name
@@ -980,18 +981,9 @@ result is undefined, and even if documented will not be interoperable.
 
 #### Schema References {#references}
 
-Several keywords can be used to reference a schema which is to be applied to the
-current instance location. `$ref` and `$dynamicRef` are applicator keywords,
-applying the referenced schema to the instance.
-
-As the values of `$ref` and `$dynamicRef` are IRI References, this allows the
-possibility to externalise or divide a schema across multiple files, and
-provides the ability to validate recursive structures through self-reference.
-
-The resolved IRI produced by these keywords is not necessarily a network
-locator, only an identifier. A schema need not be downloadable from the address
-if it is a network-addressable URL. Implementations which can access the network
-SHOULD default to operating offline.
+`$ref` and `$dynamicRef` can be used to reference a schema which is to be
+applied to the current instance location. As such, they are considered
+applicators, applying the referenced schema to the instance.
 
 ##### Direct References with `$ref` {#ref}
 
@@ -1006,31 +998,38 @@ Resolved against the current IRI base, it produces the IRI of the schema to
 apply. This resolution is safe to perform on schema load, as the process of
 evaluating an instance cannot change how the reference resolves.
 
+The resolved IRI produced by `$ref` is not necessarily a network
+locator, only an identifier. A schema need not be downloadable from the address
+if it is a network-addressable URL. Implementations which can access the network
+SHOULD default to operating offline.
+
 ##### Dynamic References with `$dynamicRef` {#dynamic-ref}
 
 The `$dynamicRef` keyword is an applicator that allows for deferring the full
 resolution until runtime, at which point it is resolved each time it is
 encountered while evaluating an instance.
 
 Together with `$dynamicAnchor`, `$dynamicRef` implements a cooperative extension
-mechanism that is primarily useful with recursive schemas (schemas that
-reference themselves). The extension point is defined with `$dynamicAnchor` and
-only exhibits runtime dynamic behavior when referenced with `$dynamicRef`.
+mechanism that is primarily useful to to create open schemas, where
+`$dynamicRef` defines the extension point and `$dynamicAnchor` defines the
+target.
+
+The value of the `$dynamicRef` property MUST be formatted as a valid
+[IRI plain name fragment](#fragments).[^3]
 
-The value of the `$dynamicRef` property MUST be a string which is a
-IRI reference that contains a valid [plain name fragment](#anchors). Resolved
-against the current IRI base, it indicates the schema resource used as the
-starting point for runtime resolution. This initial resolution is safe to
-perform on schema load.
+[^3]: `$dynamicAnchor` defines the anchor with plain text, e.g. `foo`. Although
+the value of `$dynamicRef` is not an IRI fragment, for historical reasons, the
+value still uses an IRI fragment syntax, e.g. `#foo`.
 
-The schema to apply is the outermost schema resource in the [dynamic
-scope](#scopes) that defines a `$dynamicAnchor` that matches the plain name
-fragment in the initially resolved IRI.
+Resolution of `$dynamicRef` begins by identifying the outermost schema
+resource in the [dynamic scope](#scopes) which defines a matching
+`$dynamicAnchor`. The schema to apply is the subschema of this resource which
+contains the matching `$dynamicAnchor`.
 
-For a full example using these keyword, see {{recursive-example}}.[^6]
+For a full example using these keywords, see {{recursive-example}}.[^6]
 
-[^6]: The difference between the hyper-schema meta-schema in pre-2019 drafts and
-an this draft dramatically demonstrates the utility of these keywords.
+[^6]: The differences in the hyper-schema meta-schemas from draft-07 and draft
+2019-09 dramatically demonstrates the utility of these keywords.
 
 #### Schema Re-Use With `$defs` {#defs}
 

specs/jsonschema-use-cases.xml

@@ -23,7 +23,7 @@
             </address>
         </author>
 
-        <date year="2024"/>
+        <date year="2025"/>
         <workgroup>JSON Schema</workgroup>
         <keyword>JSON</keyword>
         <keyword>Schema</keyword>

specs/relative-json-pointer.xml

@@ -40,7 +40,7 @@
             </address>
         </author>
 
-        <date year="2024"/>
+        <date year="2025"/>
         <workgroup>Internet Engineering Task Force</workgroup>
         <keyword>JSON</keyword>
         <keyword>JavaScript</keyword>