hyper-schema: "messageDefault" for automatic application of a default value

[handrews]

I'm filing this as a result of discussion with @pipobscure in issue #204.

"default" intentionally avoids specifying how the default value is used. It may simply be documentation, for instance, with no run-time effect of any sort. This is important because we do not want to restrict how an application makes use of validation- it may or may not automatically make use of the default.

However, a well-known use case in hyper-schema is to omit values from requests and responses either to make it easier to construct the message or to reduce the weight of the message on the wire.

"messageDefault" behavior

"messageDefault" would be a value that MUST be valid against the containing schema. When a client or server receives a message that omits a field or element with a "messageDefault" set, it MUST treat the instance the same as if the "messageDefault" were present. Clients and servers MAY automatically write in the "messageDefault" value before validating or otherwise processing the instance.

Use with "targetSchema" and response profiles

"targetSchema" is advisory only. The schema given by the response's "profile" media type parameter or link, or the "describedBy" link, is the only authoritative schema. A "messageDefault" that appears in a "targetSchema" but not in the authoritative schema MUST be ignored.

This follows directly from

Clients MUST NOT use the value of this property to aid in the interpretation of the data received in response to following the link

Use with input: "schema" and (if it is accepted) "hrefSchema"

Both of these keywords define schemas that may be used to accept user input. "messageSchema" MUST NOT be used to pre-populate user input. It may only be used after the user takes action. If pre-population schema field is to be defined, it would be the responsibility of the UI Schema vocabulary (#67).

In the case of "schema", "messageDefault" is used on the server side when the server receives the message.

In the case of "hrefSchema" (#179), its role in hypermedia messaging is to determine the URI by filling out the URI Template. If present, "messageDefault" MUST be used if (and only if) no user input was supplied and no instance data is available.

An application MAY use "messageDefault" to document the effect of an empty input field, including in online help such as a tooltip.

"default" and "messageDefault"

A schema may set both "default" and "messageDefault". An application may take advantage of "default"'s unrestricted usage to accomplish some other goal, such as displaying help text or pre-populating an input field.

"messageDefault" MUST NOT be treated as a regular "default" when the schema is used in a context other than a request/response message. (this point is debatable, but I think it makes sense- I'm also thinking ahead to the possibility of a UI Schema defining an "inputDefault" or something similar. If we have several usage-specific defaults, it would be ambiguous which to use as the generic "default" so it is better to keep it separate).

[epoberezkin]

@handrews Not sure I understand the name for this keyword... Can you explain please?

[handrews]

@handrews Not sure I understand the name for this keyword... Can you explain please?

default for use with hypermedia messages. "requestResponseHrefDefault" seemed too wordy :-P
(also, I thought you didn't touch hyper-schema?)

[handrews]

@epoberezkin This proposal is how we implemented "default" usage in the generic hyperclient for my last major project. It was an essential part of our approach to REST, particularly for API calls that were not directly related to web UIs.

However, this usage is not covered by the specification for "default", and some have argued that it is incorrect. Since it was a critically important feature for us, I filed it as an additional keyword, describing the behavior that we implemented and used in many places.

[epoberezkin]

I understand. The name seems a bit confusing, but it's ok. responseDefault maybe? Or it would be incorrect?

[handrews]

@epoberezkin it would be incorrect because it applies to requests and URI Template variable input as well. It behaves consistently across all of those (if this value is missing, fill it in before using).

(if it makes more sense now, any chance I can get you to remove the frowny face?)

[epoberezkin]

:)

[handrews]

It could be "hyperMediaDefault"? That's generic but at least ties it to the vocabulary. "usageDefault" is too generic- could mean anything, not just hyper-schema. I'm open to more suggestions.

[epoberezkin]

maybe exchangeDefault? That's what HTTP spec seems to be calling requests and responses... But also href... communicationDefault? Whatever :) Message is fine probably

[epoberezkin]

hyperMediaDefault is fine too, better probably, particularly taking into account where you're going to with this idea - vocabulary specific default... hyperSchemaDefault

[handrews]

"exchangeDefault" has potential... as do any of the "hyper..." options. I'm not going to push on this until after Draft 06 so let's see if anyone else comes along with an opinion in the meantime :-)

[pipobscure]

how about "defaultValue" it's generic yet by specifying that it's a "Value" as opposed to anything else makes it clear. And it leaves open the purpose !

For example my use case has nothing to do with mesaaging anything or exchanging anything. It's about infoucting a schema for storage. So by coercing something request/response/href related things would be less clear.

As always just my £0.02 feel free to distegard.

[handrews]

@pipobscure the problem with the current "default" is that it is so under-specified that you cannot make assumptions about its purpose. Or, more accurately, people make different assumptions and then tell each other they are wrong.

So the idea is to define different usages clearly. One usage involves hypermedia messages. Specifically, if a particular kind of default value is missing in a hypermedia context, it should be written into the instance before the instance is used. This kind of default is intended to make it easier to construct hyper-media messages. The usage of the default value is mandatory in specific circumstances.

This is different from, for instance, how a default might be used in a UI. In that case, the UI would be pre-populated with the default, but if the user removed the value, the default would not be written into the instance when the data was submitted. This kind of default is intended to give the user the option of accepting a suggested default. However, they may decline it.

It is not possible to write one keyword definition that works in both of these (not to mention other) usage patterns simultaneously. If we tried to use one default in both the UI and hypermedia sense (with a form submission), then we would present the default in the UI, but if the user removed the default value, when we submitted it over hypermedia, we would write the default value back in. That is probably not what the user intended when they removed the default value.

Does this make sense? The whole point is to nail down the purpose, one purpose at a time.

[pipobscure]

@handrews I get that. I was going by the title "automatic application of a default value".

That title made me think that the correct name for a property whose value is "automatically applied" if not otherwise present is exactly a "default value".

While I realize that the specific use case might be message based comms, the title makes a more general claim.

Whence my suggestion to name it "defaultValue" 😄

[pipobscure]

I have also come around to agree that plain "default" could/should be anything. defaultValue on the other hand specifies that it has to be a value.

[slurmulon]

@handrews @pipobscure

• Regarding the inputDefault idea:

  I think this property could be useful even if you consider that messageDefault will almost always be the same as inputDefault. But we should not make an assumption about this likelihood or speculate too much on the potential needs of the application, other than the obvious ones. I believe the middle ground is to clearly define the intended use of the properties we reserve through documentation, such as inputDefault and messageDefault, and then allow users to somehow add their own. It's also important to clearly define the order in which these defaults are applied - for instance, perhaps we can say that the hierarchy is always default < messageDefault < inputDefault.

  Super minor, but I'm proposing formDefault as an alternative to inputDefault, the main reason being that input may initially correlate with the <input> HTML tag in people's minds, where as form is more generic, encompasses all inputs and rings stronger with HTML forms.

• As an alternative to default, messageDefault and inputDefault, how about:

  defaults: {
    base: { ... },
    form: { ... },
    body: { ... }
  }
  

  The main advantage here is that it's clearly expressed that there exists a commonality between these properties and the intent of each value is consistently expressed with its name, partially addressing the "what is default vs. messageDefault?" issue. It also allows us to add new properties / defaults to the object as time goes on and people find new uses for Hyper-Schema.

  Lastly we sort of achieve a harmony in that we can reserve and document properties for the common / reserved use cases, such as for an HTML form or a request body, but we can also allow users to add their own non-reserved properties and use them any way they'd like. This way we don't have to make (as many) assumptions about the different use cases and also don't have to limit the arbitrary / free-range default to the single property; people can add them as needed.

[handrews]

@pipobscure ah, I see what you mean. I believe if we are going to be more clear about a default that can be relied upon as usable, we should also be more clear about when it is applied. It makes a great deal of difference to me as an API developer or user whether a default is consistently filled in any time a message passed between the client and server, vs whether it is only filled in right before being presented to the client, but is never filled in if it is missing from a server message.

One of those (before client presentation only) is useful for pre-populating user interfaces. The other is useful for consistently working with smaller data structures. Those are very different use cases.

[handrews]

@slurmulon I'd like to keep this focused on the hypermedia message use case as much as possible. That keyword belongs in the hyper-media vocabulary, while inputDefault/formDefault belongs in the UI vocabulary (see #67).

If we have multiple such keywords, each should be fully specified on its own, and not need to worry about the presence or absence of others. For instance, "messageDefault" is utilized as close to the message-passing boundary as possible while "formDefault" is used just before presentation to the user. There's no need to disambiguate that.

We have generally avoided making sub-objects. LDOs are a notable exception, as they are meant to also be used on their own and the repetition of the fields requires some sort of grouping.

I really, really, really want to keep Hyper-schema, UI, and other vocabularies independent. I am not necessarily using my hypermedia with UI forms at all. In fact, I am often definitely not connecting them to forms. Likewise, form generation may not connect to hyper-media.

[slurmulon]

@handrews

I'd like to keep this focused on the hypermedia message use case as much as possible.

Ok. I'm going to provide just a few more details and reply to your point. I hope to show how this could be used not only for message but for any default.

I'll think more about refining the purpose and usage of the messageDefault property in particular and will keep on that track from now on.

---

If we have multiple such keywords, each should be fully specified on its own, and not need to worry about the presence or absence of others. For instance, "messageDefault" is utilized as close to the message-passing boundary as possible while "formDefault" is used just before presentation to the user. There's no need to disambiguate that.

Not sure what you mean by "there's no need to disambiguate that" or how introducing a nested object would do this..? Like, how is default.form any different than defaultForm other than it gives you more flexibility and a way of grouping / categorizing the properties? I am mostly interested in that it allows users to define their own unique uses of default, giving them free-range to make any property on the default object that's not already reserved and thoroughly documented. This would eliminate the issue mentioned where we only have one free-range default property, which is default. For instance, if you wanted to provide something like documentationDefault which has different value than default and you're already using default for something else, you're out of luck without this object.

---

We have generally avoided making sub-objects.

Yeah I noticed that and I figured it was to eliminate "special" constructs but they're clearly used in LDOs as you mentioned. In my mind if you're going to use a sub-object (or any concept) in one place then it's better to allow these in other places as well in order to diminish the specialness and uniqueness. If the selection of these places is arbitrary or ad-hoc then it probably just makes things confusing for the user. But, if these properties are only allowed within an LDO, then I argue the issue of specialness is well contained and shouldn't really surprise the user.

---

I really, really, really want to keep Hyper-schema, UI, and other vocabularies independent. I am not necessarily using my hypermedia with UI forms at all. In fact, I am often definitely not connecting them to forms. Likewise, form generation may not connect to hyper-media.

Hm, I fail to see how introducing an object makes these vocabularies any less independent or how this forces you to define things you may not use use, like UI. default by itself, an empty object, means nothing. default.form is completely orthogonal to default.message even though they belong in the same object. Consider something like this:

{
  default: {
    message: ...
    form: ...
    anything_the_user_needs: ...
  }
}

vs.

{
  defaultMessage: ...
  defaultForm: ...
}

The only difference I see is that the parent object gives users more flexibility and freedom while achieving the same goal. Is it simply that the properties sharing an object makes them undesirably related with each other in your mind? If so, why wouldn't the same argument apply to the second example with defaultMessage and defaultForm, they both share an object, the only difference is they're prefixed with the word default and grouped in that matter instead.

[handrews]

@slurmulon The question of nested objects or not is a broader one than this message, and we should not be trying to solve it here. If you want to make a case for nested objects (it's come up in several places, although never convincingly so), then you should file your own issue for that. You can use whatever existing or proposed groupings you want for an example there.

There's nothing whatsoever in this proposal that makes any difference one way or the other whether the specific default keyword I'm talking about is nested or not.

I'm not sure this issue is all that useful at this point. I may slap a shrug label on it, lock it, and come back again later. It does not seem to have made a good impression on anyone.

[slurmulon]

@handrews

The question of nested objects or not is a broader one than this message, and we should not be trying to solve it here

That's fine, but it was not very clear what was going on as people started talking about other related values, particularly default. I'm not going to waste time opening a ticket if people have, for some reason, repeatedly decided against this concept in a generic way. Moving on...

I'm not sure this issue is all that useful at this point. I may slap a shrug label on it, lock it, and come back again later. It does not seem to have made a good impression on anyone.

I think the concept is sound and very useful, I'm not sure where the "bad impression" vibe is coming from. If it's just a matter of defining how / where this is used, well I think it's pretty clear. messageDefault is applied in either an HTTP request or response, in either a client or server, where the contents of the body are compared against the defined messageDefault schema and missing values are filled in accordingly. I'm not saying there isn't a need for formalities or that there aren't additional details to cover and a need for formality, there certainly are, but maybe we should be explicit as to what they should be. What aspects would you like to see fleshed out more here?

[handrews]

@slurmulon This issue is now quite long and going in three or four different directions, at least. I am going to close it and re-file the hypermedia part separately again because, based on past experience, some people will flat-out refuse to read or comment on an issue this long and complicated. That is why I try to keep the issues focused.

Please file your ideas for "formDefault" / "inputDefault" separately- I think we should have a default of that sort but it needs to be discussed separately from the hypermedia default.

Like, how is default.form any different than defaultForm other than it gives you more flexibility and a way of grouping / categorizing the properties?

Stylistically, JSON Schema nearly aways uses multiple keywords with prefixes or suffixes instead of nested objects. For instance, "properties", "patternProperties", "additionalProperties", "propertyNames", "items", "additionalItems", etc. etc. instead of

{
    "properties": {
        "exact": {...},
        "pattern": {...},
        "additional": {...},
        "name": {...},
    }
}

This has come up over and over and over again. See discussions around "if"/"then"/"else", for instance.

Why do we do it this way? No idea. I'm neither defending it nor attacking it. But since we have an established pattern, we should continue to follow the pattern. If you want to change that pattern, that's a discussion that needs to happen separate from any specific keyword proposal.

But, if these properties are only allowed within an LDO, then I argue the issue of specialness is well contained and shouldn't really surprise the user.

It's not that they're allowed within the LDO- keywords inside the LDO are also flat and prefixed e.g. "href" and "hrefSchema". It's just that the "links" keyword itself is a list of objects, since there are multiple links.

The only real place where a nested object occurs is the "media" keyword and we may want to revisit that. It's kind of a glaring exception. But this is not the issue in which to debate this.