Proposal: "jsonpointer" type

[erosb]

Introduction

While the JSON Schema specification itself utilizes JSON Pointers, it has only a very basic support for letting schema authors specify a schema which expects JSON documents to use such JSON Pointers. Currently, when a schema author wants to describe a JSON Pointer, the only thing he/she can do is defining the schema with "type":"string" and probably adding a regex restriction which mandates the string to be a syntactically valid json pointer.

In practical usecases it is useful to be able to describe the schema of the referred JSON value. This proposal targets these usecases.

Purpose

The purpose of this new type is to let schema authors expect that the document being validated contains a JSON Pointer, and this pointer denotes a value which conforms to the restrictions of a schema defined by the schema author.

Syntax

Example schema:

{
    "properties": {
        "ptrToNum": {
            "type": "jsonpointer",
            "referredSchema": {
                "type": "number"
            }
        }
    }
}

Example valid document against the above schema:

{
    "ptrToNum" : "#/settings/width",
    "settings" : {
        "width" : 4
    }
}

Example invalid document against the above schema:

{
    "ptrToNum" : "#/settings/width",
    "settings" : {
        "width" : [ 4 ]
    }
}

Validation

If a schema instance has a "type" : "jsonpointer" property , then it must also have a "referredSchema" key and a then an implementation should perform the following validation steps:

• it should validate if the value is valid against the {"type" : "string"} schema
• it should check if the string value is a valid JSON Pointer as specified by RFC 6901 (question: string representation or URI fragment identifier representation or both?)
• it should evaluate the JSON Pointer. The result of the evaluation is a JSON value. If the evaluation fails then the validation is unsuccessful
• it should validate the evaluated JSON value against the schema given by "referredSchema"

Notes

• I'm not sure about the "referredSchema" keyword , maybe something more expressive can be better
• while writing the above proposal I didn't take the wording of the current spec into account so probably most of it must be re-phrased. I'm also not sure if it should go into the core or the validation spec.
• we should specify what should be done by the implementation if it runs into infinite recursion while resolving JSON pointers
• so please take it as a draft / suggestion in its current form. If I get some positive feedback or some advices then I will create a proper pull request.

[erosb]

@handrews @awwright @Relequestual any thoughts on this?

[handrews]

@erosb can you give a more specific use case for this? I understand how the proposal works, but I cannot think of a situation where I would need this sort of indirect specification.

Also, the value you have given for ptrToNum is not actually a JSON Pointer, but a URI fragment using JSON Pointer syntax (If you removed the leading "#" then it would be a JSON Pointer). I am assuming that's just a typo- even with that changed I am unclear on the utility here.

[Relequestual]

What @handrews said.

[handrews]

@erosb I'm going to go ahead and submit a PR for json-schema-org/json-schema-spec#109 that will include jsonpointer as a format, because even if this proposal (#141) is accepted, not all jsonpointers are used to refer back into the same instance or to apply any further validation. Just wanted to warn you- when you see a PR for json-schema-org/json-schema-spec#109 that's not me dismissing this issue.

I still would need to see some real use cases for this one (#141) before I could support it, though.

[erosb]

Okey. These 2 proposals can even live together, json-schema-org/json-schema-spec#109 for untyped references and this for typed ones. I will get back to here with some examples when I will have some more time.

[erosb]

Example for using typed JSON Pointers

Lets consider a json format which represents 3D objects in a scenegraph and collisions of elements. So it has 2 root properties:

• "scene" which is arbitary-depth tree, hierarchy of scene objects
• "collisions" which is an array, where each element is an array of colliding objects in the scene.

Each individual collision should be represented with a list of JSON pointers, pointing to the colliding objects in the scene.
These pointers are allowed to denote only objects and no other type of data in the document - and this is the restriction which can be expressed by the proposed JSON pointer type.

So generally speaking, JSON is basically designed to represent hierarchical (tree) data structures. Once the data model becomes a graph and not a tree, the most robust solution to express (possibly circular) references is using JSON Pointers.

This proposal (#141) helps in making these pointers typed. So in draftv4 there is not even a way to make syntactical restrictions on a string to verify that it is a JSON Pointer, and this is the problem being solved by "format":"jsonpointer" proposed in json-schema-org/json-schema-spec#109 . This is much better than having nothing, but the pointers exressed this way are untyped, so even after schema validation, a consuming application needs to manually validate the referred values, in case it has some necessary preconditions regarding their structure (and chances are high it has).

So as an analogy, having only untyped pointers is like working with java.lang.Object-type references in java and casting and instanceof-ing the referred values all the way. So that is not a good practice, and JSON Schema must also provide a mechanism to counter it, even if it is less important in this domain. This analogy is not even that far, if you consider JSON Schema as a type system.

Final thoughts:

• I'm fine if both format: add uritemplate and jsonpointer json-schema-spec#109 and json-schema-org/json-schema-spec#141 becomes a part of the spec, but json-schema-org/json-schema-spec#141 is able to express untyped references too, with "referredSchema":{}
• @handrews wrote "the value you have given for ptrToNum is not actually a JSON Pointer" - well I'd argue with that, "#/settings/width" is a JSON Pointer in its URI fragment identifier representation, according to RFC 6901. But it is a detail being unimportant at the moment.

[handrews]

"the value you have given for ptrToNum is not actually a JSON Pointer" - well I'd argue with that, "#/settings/width" is a JSON Pointer in its URI fragment identifier representation, according to RFC 6901. But it is a detail being unimportant at the moment.

It's actually quite important. "/settings/width" would be {"format": "jsonpointer"} while "#/settings/width" would be {"format": "urireference"}, with the media type of the referenced resource determining whether the fragment can be interpreted as a JSON Pointer or not.

Either type could have constraints on the thing to which it is pointing, although with a URI Reference the constraint can already be described through the "profile" media type parameter and/or link, and the "describedBy" link. In that case you are relying on the referenced resource to provide the correct schema.

[handrews]

To repeat a relevant response to your comment in json-schema-org/json-schema-spec#150

RFC 6901 does not define a representation of a JSON Pointer as a URI fragment. It defines how to encode a JSON Pointer in a URI fragment. (specifically UTF8 + percent encoding).

That is just a URI fragment, and the interpretation of the fragment as a JSON Pointer is controlled by the media type of the document (which is always how fragments are interpreted).

If you want to use a URI with a JSON Pointer fragment, that is just a "uri" or "uriref" format. It is not, by itself, a JSON Pointer.

---

It would be just as reasonable to propose a reference type (with referredSchema) that works with a URI Reference, or that can take either a URI Reference or a JSON Pointer.

For URI References, you need to define the base URI (is it the same base used for $ref?). For JSON Pointers you need to define to which document it applies (the schema? the instance? something else?)

Also keep in mind that most uses of JSON Pointer from within a schema actually need Relative JSON Pointers for the same reason that most uses of URIs are actually URI References: when re-using a bit of schema you may not know where the schema is being used, but you more often know or have defined as sub-schemas what other schemas or instances exist relative to the point at which the schema is used.

[erosb]

@handrews the following came into my mind when reading your comment above: what about defining these references as

• "type" : "reference" (or maybe "pointer")
• "format" : it can be "jsonpointer" or "urireference"
• "referredSchema" : still the same, although it seems a better naming for it is necessary (I originally called it "referredSchema" as a short form to say "the expected schema of the referred value", but it seems to be inaccurate and misleading)

If "format" is "urireference" then it resolved at the instance level in the same way as "$ref"s are resolved in the schema.
If "format" is "jsonpointer" then it is evaluated as defined in RFC6901.

What do you think?

[handrews]

@erosb that seems like a reasonable approach. I'm not entirely sold on whether the use case is in-scope for JSON Schema, but I need to look at it a bit more. Right now I'm focusing on trying to wrap up Draft 06 so I'll probably come back to this once that's sorted.

[handrews]

I just realized that an obvious use case for this proposal is in the meta-schema, placing restrictions on what $ref can reference:

{
    "definitions": {
        "jsonReference": {
            "properties": {
                "$ref": {
                    "type": "reference",
                    "format": "uriref",
                    "referredSchema": {"$ref": "#"}
                }
            }
        }
    }
}

Note "format": "uriref" instead of "format": "jsonpointer" as $ref uses a URI reference, which sometimes happens to be a fragment containing a JSON Pointer, but "format": "jsonpointer" would be a valid option in other situations.

In issue json-schema-org/json-schema-spec#98 (comment) I just noted that this type would be useful for defining the with clause more clearly. The difference between this meta-schema and the one in that comment of issue json-schema-org/json-schema-spec#98 is that "nonValidationSchema" here further restricts $ref to only refer to another non-validation schema:

{
    "definitions": {
        "validationProperties": {
            "enum": ["type", "multipleOf", "maximum", ...]
        },
        "nonValidationSchema": {
            "allOf": [
                {"$ref": "#"},
                {"propertyNames": {"not": {"$ref": "#/definitions/validationProperties"}}},
                {
                    "properties": {
                       "$ref": {
                            "referredSchema": "#/definitions/nonValidationSchema"
                        }
                    }
                }
            ]
        }
    },
    "properties": {
        "$use": {
            "type": "object",
            "properties": {
                "source": {"$ref": "#"},
                "with": {"$ref": "#/definitions/nonValidationSchema"}
            }
        }
    }
}

[erosb]

I'm glad to see you start liking it :)

Hopefully I will have some time to work on this after Christmas.

[epoberezkin]

"type" is used as basic JSON type, with "integer" being the only exclusion... Maybe instead of proliferating the exclusions we should indeed consider dropping "integer" and use format for it? @erosb I don't understand how "format": "jsonpointer" is not sufficient and why do you want to abuse type to achieve it...

[epoberezkin]

If it's an alternative to $data proposal, it seems an unnecessary complex and confusing alternative.

But it seems like it is something else...

I don't think I understand your use-case.

[epoberezkin]

I just realized that an obvious use case for this proposal is in the meta-schema

@handrews you can't really bend the standard for the purpose of writing meta-schemas... Users write schemas, meta-schemas serve the standard, enforce it, not the other way around...