Standardize implementation-defined behavior

[awwright]

The specification currently specifies that schemas without any "$schema" keyword are "implementation defined" (emphasis added):

The "$schema" keyword SHOULD be used in the document root schema object, and MAY be used in the root schema objects of embedded schema resources. It MUST NOT appear in non-resource root schema objects. If absent from the document root schema, the resulting behavior is implementation-defined.

This under-constrains the validation behavior and permits behavior that I would never expect to be possible (it wouldn't be wrong to say that {"type":"string"} permits null). And it provides no guarantees about reverse-compatibility; updates to the meta-schema that are reverse-incompatible will have broad effects. (That is to say, dialects don't actually achieve the intended goal of forward compatibility, it just pushes this responsibility onto implementations, probably in platform-specific ways.)

Further, there's at least a few implementations that do not read "$schema" and even if they should, requiring this keyword would be impractical. It's perfectly clear what is meant when someone writes {"type":"string"} without any context.

It's understood that implementations aren't always up-to-date on a spec; that some implementations only follow older versions of a spec normally goes without saying. When we say that documents without "$schema" is implementation-defined behavior, we're either being redundant, or we're saying something more than that.

The "$schema" keyword is, of course, useful. It can declare a subset of the JSON Schema vocabulary (e.g. some document databases might not be able to implement the full vocabulary); or user or implementation-specific keywords (vocabularies). And it can be used as a heuristic to decide if older behavior for a keyword should be used.

But it can't solve all versioning problems. A custom meta-schema might not define compatibility with any draft/release of JSON Schema. We still need to define the behavior for these situations.

By comparison, text/html and application/xhtml+xml have a single document that defines how to interpret all documents, even ones marked with an older version number. Some HTML versions allow you to specify a DTD, but these restrict the elements you're allowed to use, they aren't required, they don't actually change the semantics of the elements, and their omission has a standardized behavior.

---

A simple test for potential solutions is: null (and other values) cannot be valid against {"type":"string"}. Currently the behavior is under-constrained (under-specified) and there's nothing to say that this would be wrong.

Potential solutions:

• Error on a missing $schema keyword: This would break a very large number of existing implementations and documents. null wouldn't be valid, but it wouldn't necessarily be invalid either. (And even if we were authoring from scratch, requiring version identifiers in documents is strongly discouraged in Internet media types, and very unpopular among document authors.)

• Implementations assume the latest known, standard $schema: This would give stronger guarantees about cross-platform compatibility (presumably, "type" would always mean the same thing). However, this would defeat the intention of "$schema" as a dialect identifier. If an implementation updates its default "$schema", this would break reverse compatibility if the new dialect is not reverse-compatible. (This is also true wherever the behavior is undefined or implementation-defined.)

• Single media type specification: Newer drafts replace older drafts in their entirety, though implementations may choose to implement older behavior for reverse compatibility reasons. Because the URI of the meta-schema changes with every draft, "$schema" can be used as a heuristic to determine if a schema is expecting an older, superseded behavior. All releases would have to be reverse compatible, or at least, changes would have to be carefully weighed, especially if there's orthogonal implementations. This was the behavior through draft-07.

[gregsdennis]

I think the intent in that specific paragraph is to say that an implementation can choose which draft it will use to process the schema. I expect most will choose the latest that they support.

[Relequestual]

This makes sense, it might not make sense to proscribe a fatal error for every situation.

My preference would be to change what's defined here.
What I'd really like is for hard error if $schema is not provided, however I understand (with frustration) that it might be impractical to require this.

However, I think we COULD say that, the implementation MUST select a dialect it knows about as default, which SHOULD be a JSON Schema org defined dialect.

Additionally I'd like to add that if the dialect is chosen by the implementation (And not via $schema or some other user provided means), then they SHOULD throw a warning (as appropriate for the language) if deprecated keywords are used.

This would be more impacting tools used to write schemas with immediate feedback. I'm not fully aware of the effects this could have for general purpose applications.

[awwright]

I added an abstract in the OP, but I'm not sure if that actually clarified what I'm looking for. I'll try to revise it as I better understand the problem space we're dealing with. (Edit: I ended up rewriting most of the OP.)

My preference would be to change what's defined here.
What I'd really like is for hard error if $schema is not provided, however I understand (with frustration) that it might be impractical to require this.

Yes, the behavior should be better specified, and not left undefined. However I'm not sure an error is reasonable. There's no situation where {"type":"string"} should be an error, it's perfectly obvious to me what's being asked for there.

the implementation MUST select a dialect it knows about as default, which SHOULD be a JSON Schema org defined dialect.

This is more reasonable. Implementations shouldn't be allowed to pick any arbitrary behavior on their whim, the selected behavior should be unsurprising.

However I don't think "dialects" are a good way to reason about this. Media types are a way to version.

We're currently having this debate over the application/problem+json media type, how do we add new standard keywords in a way that's backward-compatible; the official spec says this would require a new media type. But due to a concern over media type proliferation, we seem to have agreed that we can define a prefix that specifies new global keywords going forward (maybe * which seems to be unused in the wild).

[Relequestual]

However I don't think "dialects" are a good way to reason about this. Media types are a way to version.

Be that as it may, JSON Schema has MANY uses outside of HTTP requests.
And, regardless, JSON Schema defines "what set of stuffs" to use via the dialect identifier.

[awwright]

Be that as it may, JSON Schema has MANY uses outside of HTTP requests.

Sure, though this doesn't necessarily preclude using HTTP and Internet features. I wasn't seriously suggesting this as a solution, but it's interesting to think about.

And, regardless, JSON Schema defines "what set of stuffs" to use via the dialect identifier.

I think I understand your position now, but I still have questions and problems that need addressing:

• This issue—the dialect identifier is not required, and cannot be assumed in a cross-platform, backward-compatible way.
• Each dialect/meta-schema seems to be co-equal with all the others, but there's nothing stating this. Older drafts remain somewhat hidden. If true, this would contradict the usual rule that newer drafts replace older ones. (In my understanding, validators can implement older behavior, but for the purpose of preventing breakage, not because older drafts have the same endorsement as newer ones.)
• It is not clear that $schema can be used to change the behavior of $ref: they're both core keywords; they're not defined as part of the dialect. If maintaining strict backward-compatibility in a cross-platform way is important, we would have to write the older behavior into the specification, as well as the mechanism for detecting when to use it.

[jdesrosiers]

The example that { "type": "string" } should always mean the same thing no matter which dialect is used is not correct. The vocabulary system allows you to define your own keywords even ones who's names and semantics conflict with official vocabularies. I can create a dialect that replaces the official validation vocabulary (that includes the type keyword) with my own validation vocabulary and that defines a type keyword that means something very different. A very realistic example might be a dialect who wants to define more specific types such as int32 and int64 instead of int.

So, an implementation needs to know or assume a dialect somehow to correctly evaluate even the simplest schema. The vocabulary system makes just about anything possible.

In my implementation, I require that a dialect is declared somehow. It doesn't have to be with $schema. There are multiple ways you can declare it, but if you don't it's an error. There's no way to safely assume what dialect was intended.

[awwright]

@jdesrosiers But I didn't say anything to the effect of "no matter which dialect is used"; the example { "type": "string" } has no dialect (or at least no explicit one). In this situation, it cannot be true that null could be valid. This is a perfectly valid schema, and we can't break support for it.

[jdesrosiers]

@awwright I'm not sure what distinction you're trying to make. Every schema is written in some dialect whether that dialect is declared or not. If the dialect is not declared, an implementation may choose one to use to interpret the schema. That dialect does not need to be an official JSON Schema dialect. It could be a dialect where type is just an annotation. In that case, an implementation could evaluate { "type": "string" } against null and not be wrong.

This is a perfectly valid schema, and we can't break support for it.

This is a false assumption. We've made backwards incompatible changes in almost every release. We've never made any guarantees that any keyword will always work a certain way in every dialect past, current, or future. The vocabulary system allows users to create dialects that do almost anything and they are not required to be backwards compatible with official JSON Schema releases.

[awwright]

Every schema is written in some dialect whether that dialect is declared or not

The idea of "dialect" was comparatively recently introduced. E.g. { "type": "string" } is one of the most common schemas in existence. Regardless of how they are technically handled today, they were not authored with the understanding of a dialect (they could not have been). These are the schemas we cannot break.

That dialect does not need to be an official JSON Schema dialect. It could be a dialect where type is just an annotation.

Ok, but { "type": "string" } is obviously not one of these cases. The author clearly intended the standardized meaning.

We've made backwards incompatible changes in almost every release.

I've addressed this; we make very few changes that actually force implementations to change behavior in a breaking way; and only in very well-researched situations (like $ref).

If you mean how we occasionally remove behavior: HTTP, email, etc, also remove behavior in every new release, that's not the same thing as "backwards incompatible" (removing something from a tech spec usually doesn't force implementations to change their behavior).

The vocabulary system allows users to create dialects that do almost anything and they are not required to be backwards compatible with official JSON Schema releases.

Ok, but I'm not talking about custom dialects/vocabularies/meta-schemas. The question is: What happens in the default case?

[jdesrosiers]

The idea of "dialect" was comparatively recently introduced.

The name "dialect" is relatively new (if you consider three years new). The concept is at least as old as the $schema keyword and predates the vocabulary system. The earliest significant example is probably when Swagger chose to use a heavily modified version of JSON Schema. As it evolved and became OpenAPI, at least two more dialects have been defined. The vocabulary system allows you to define your own dialects, but it just formalizes what people have been doing all along.

{ "type": "string" } is one of the most common schemas in existence. Regardless of how they are technically handled today, they were not authored with the understanding of a dialect (they could not have been).

I don't understand what you're trying to say. I can create a dialect that does something weird with type and write that schema with the intention of it using my custom dialect. You seem to talking about a schema written before the vocabulary system existed and where third-party dialects are ignored. In that case, you'd be right. But the vocabulary system does exist and third-party dialects do exist. You can't ignore that.

These are the schemas we cannot break.

You say this as if we're considering a change to the specification that might break that schema. This problem already exists. Custom dialects that can do almost anything are already a reality. You can't put that genie back in the bottle. I'm not defending it. I'm just saying we have to accept what already exists in the wild.

Ok, but I'm not talking about custom dialects/vocabularies/meta-schemas. The question is: What happens in the default case?

We don't have one source of truth for what the default behavior is. Right now, every dialect declares it's own rules and historically, those rules have been very permissive. Even if in a future release we constrain what can be done in the default case, implementations that were written for previous drafts would not be affected. They could still choose whatever dialect they want (or error, or whatever) in the default case. If the default behavior is different between dialects, there's no way for an implementation to know which default to follow.

[awwright]

The concept is at least as old as the $schema keyword and predates the vocabulary system

The ability to use a keyword to change the dialect so that any keyword can mean anything does not go back that far.

The concept of "$schema" was introduced in draft-03, as a to provide a hint to validators. Its use by both authors and validators was completely optional:

5.29. $schema

This attribute defines a URI of a JSON Schema that is the schema of
the current schema. When this attribute is defined, a validator
SHOULD use the schema referenced by the value's URI (if known and
available) when resolving Hyper Schema (Section 6) links
(Section 6.1).

A validator MAY use this attribute's value to determine which version
of JSON Schema the current schema is written in, and provide the
appropriate validation features and behavior. Therefore, it is
RECOMMENDED that all schema authors include this attribute in their
schemas to prevent conflicts with future JSON Schema specification
changes.

This is consistent with "$schema" being a meta-schema reference, that could optionally be used as a versioning heuristic.

At the very earliest, the idea that you could use "$schema" to switch behaviors was draft-04, when Hyper-Schema was published as a separate specification. Even at this point, I don't believe $schema was required in order to switch behavior. If I used a JSON Schema validator that was hypermedia enabled, I would expect the hypermedia functions to work even in the absence of the "$schema" keyword.

In draft-05 a.k.a. draft-wright-json-schema-00, I removed specific references to older values of "$schema", replacing it with a generic paragraph about how it's OK to implement values found in other publications. This is where the idea that "$schema" can switch behaviors properly comes from; and it was very carefully worded to maximize forward compatibility.

---

I can create a dialect that does something weird with type and write that schema with the intention of it using my custom dialect.

You seem to be caught up on the idea that I can define a custom dialect and give "type" whatever semantics I want. I am specifically saying that I am not using this functionality. When someone publishes a post on Stack Overflow about why "0" is valid against {"minimum": 1}, nobody is asking them what dialect they are using.

We don't need to ask and it doesn't matter. We know we're talking about the validation keywords defined in the latest draft. But how does a validator know this? Where is this written? It appears that it's not.

(Now if you want to adjust the settings, or write an implementation that does something special ($data), or create a new dialect that does something unexpected, go for it; that's not what I'm objecting to here.)

[gregsdennis]

@awwright @jdesrosiers

I get that you two are trying to come to a common understanding, but it seems to me like you're both splitting hairs.

This conversation has veered away from its original purpose, which was to unify implementations' behaviors when $schema is missing. I think we can agree that letting implementations do what they want goes against interoperability.

This isn't an error state like the other cases where we say "implementation-defined" or "undefined" behavior. In this case, we have a schema that can be processed. Implementations should do it the same way.

I don't really have stake in which way this goes, except that I need to know what to implement.