Should we define how Inclusion (transclusion) should be implemented given `$ref` is delegation?

[Relequestual]

Assuming #523 gets resolved (It has! $ref is now delegation), we are implying that transclusion is not a standard process.

As such, should we specify how transclusion should happen if it's required (edit: by the user of a schema)? There are valid use cases where it might be acceptable.

If we should specify how transclusion should happen, should this be in the spec or on the website only?

Previously, I had some ideas about how to address this. Note: The section is incorrectly titled as it should be "Resolving referneces by Transclusion", as dereferencing is the act of resolving the location, not resolving the reference action required.

We should mention that it may also be known to people as "inclusion", but it is techically "transclusion".

See attached image (source)

This issue is created by request of #585

[handrews]

@Relequestual thanks for splitting this out.

As such, should we specify how transclusion should happen if it's required? There are valid use cases where it might be acceptable.

You mention both "required" and "acceptable" here. I'm pretty sure I understand "acceptable", but under what circumstances would it be required?

If this is a process that is required to be supported, then I can see it as transclusion. Otherwise to me it's just processing that happens outside of the spec. Collapsing branches of an allOf down to a single schema object is another type of processing that can be done based on how the keyword is specified (and it's how Doca works), but is not a required part of the specification at all. I see this in the same way, personally.

[awwright]

I'm not sure where to start, but first I have to admit I've never really understood what problem we're trying to solve. I sort of thought if I read long enough it'd become clear.

I don't think we're approaching this with the right vocabulary.

I don't think asking "is this transclusion or inclusion?" is productive since those aren't well defined terms in JSON Schema. Normally when I see those terms getting used, this problem is already well defined:

How would you process a document (schema) so that you can replace a single reference to a document with content that eliminates that reference?

(In the case of recursive schemas you'll never be able to eliminate all references, but you can expand them indefinitely as necessary to process any given, finite document.)

The idea behind "$ref" is that it was pure transclusion, i.e. without knowing any semantics of the document, you could substitute the reference with the target document. This was impractical, because it eliminates the ability to have schemas describe documents with a "$ref" key, and it breaks how URI dereferencing is supposed to work.

My current understanding is this: When you encounter an object that contains "$ref", then substitute that object with the target document; and if necessary, change/set the "$id" property so that it's a complete URI (the URI Reference from $ref resolved against the URI base where the $ref was encountered).

It sounds like the proposal is to change it to something like this: When you encounter a property named "$ref": "{target_reference}", then treat it as { "$base": "{target_uri}", "allOf": [ /* contents of target_uri */ ] } (insert this schema into allOf, creating it if it doesn't exist) where {target_reference} is a URI reference, {target_uri} is the resolved URI, "$base" is a (currently hypothetical) keyword that adjusts the URI base without setting any "$id", and /* contents of target_uri */ is the transcluded document.

(Note that I'm also describing two interchangable ways to ensure the correct URI base is used, in the former case you have to parse and the referenced document and modify it, in the latter case you can use the referenced document verbatim, but we have to invent a new keyword.)

Does this make sense? If so I can think up my next question.

[handrews]

@awwright we already decided (in #514) to stop treating $ref as something that gets (logically) replaced with the target document, and instead treat it as a keyword that produces the same result as its target schema.

One thing that this allows is having other schema keywords in the same object as $ref (as noted in #523). If we allow other keywords, then if someone wants to replace the $ref with its target, they cannot just replace the object that has the $ref, because that will also replace the other keywords in the object.

@Relequestual wants to include a process for replacement that works with the new approach. I do not think that we should do that. We just decided to change to the "delegation" approach, and I think the specification should focus on that.

I have pointed out that a tool that wants to do dereferencing that encounters adjacent keywords to $ref could get around that with allOf. And at some point I said that if we want to put this in the spec at all, I would prefer that mechanism over some other phrasing, I forget exactly what.

But I don't think it needs to be in the spec at all. Being able to do that is a simple consequence of the definitions of the keywords. And I don't think the spec should recommend dereferencing anyway, given the various confusing aspects such as handling $id and making sure that you don't change $schema.

[Relequestual]

You mention both "required" and "acceptable" here. I'm pretty sure I understand "acceptable", but under what circumstances would it be required?

When I said "required", I meant by the user of the schema. Sorry I should have been more specific.

I've modified the title to ask if we SHOULD do this or not.
I think based on the last comment, we believe we should NOT do this as part of the spec.

It may be we could include such comments on the site.
In failed PR #585 I came up with some rules that I thought would make sense. Mainly I wanted to make sure that those who wanted to implement transclusion of $ref references, would do so in a consistent way.

I guess it really feels like it should be cookbook material.

(I'd like to keep the PR branch for records.)