From 3ecb2294c71119509e7df66bbdb15dd4b8a2882a Mon Sep 17 00:00:00 2001 From: Gregg Kellogg Date: Fri, 3 May 2019 16:45:42 -0700 Subject: [PATCH 1/5] Rewrite LoadDocumentCallback to include most HTML-specific and profile-specific processing, and use in *expand* and *Context Processing*. --- index.html | 299 +++++++++++++++++++++++++++++++++-------------------- 1 file changed, 186 insertions(+), 113 deletions(-) diff --git a/index.html b/index.html index 15aa8cea..64da0284 100644 --- a/index.html +++ b/index.html @@ -929,7 +929,7 @@

Processor Levels

JSON-LD mostly uses the JSON syntax [[RFC8259]] along with various micro-syntaxes based on XML Schema datatypes [[XMLSCHEMA11-2]]. However, it has become increasingly common to include JSON within - a script element + a script element within an HTML document [[HTML]], as described in . As not all processors operate in an environment which can include HTML, @@ -1111,33 +1111,10 @@

Algorithm

then the processor MUST NOT do a further dereference, and context is set to the previously established internal representation. -
  • Otherwise, dereference context, transforming into the internal representation.
    • -
  • If the retrieved document's Content-Type is text/html: -
      -
    1. If processor is a pure JSON Processor, - a loading remote context failed - error has been detected and processing is aborted.
      2. -
    2. If the original context - contains a fragment identifier, - set source to the textContent - of the script element in context - having an id attribute - that matches the fragment identifier, after decoding percent encoded sequences.
      3. -
    3. Otherwise, if the retrived document has a script element - of type application/ld+json;profile=http://www.w3.org/ns/json-ld#context, - set source to the textContent - of the first element found with that type.
      4. -
    4. Otherwise, if the retrived document has a script element - of type application/ld+json, - set source to the textContent - of the first element found with that type.
      5. -
    5. If no element is found, - a loading remote context failed - error has been detected and processing is aborted.
      6. -
    6. Set context to the result of the Extract Script Content algorithm, - using source, transforming into the internal representation.
      7. -
    -
    • +
  • Otherwise, dereference context using + the LoadDocumentCallback, passing context + for url, + and http://www.w3.org/ns/json-ld#context for profile
  • If context cannot be dereferenced, or cannot be transformed into the internal representation, a loading remote context failed @@ -5001,7 +4978,7 @@

    Extract Script Content Algorithm

    The algorithm extracts the text content a JSON-LD script element into a dictionary or array of dictionaries. A JSON-LD script element is a script element - within an HTML [[HTML]] document with the type attribute set to + within an HTML [[HTML]] document with the type attribute set to application/ld+json.

    The algorithm takes a single required input variable: source, @@ -5131,21 +5108,18 @@

    The JsonLdProcessor Interface

  • Create a new Promise promise and return it. The following steps are then executed asynchronously.
  • If the passed input - is a string representing the IRI of a remote document, dereference it as original input. -
      -
    1. If the retrieved document's Content-Type is text/html, - and the processor is a pure JSON Processor - reject the promise passing a loading document failed error.
      2. -
    2. Otherwise, if the retrieved document's Content-Type is neither - application/json, - application/ld+json, - nor any other media type using a - +json suffix as defined in [[RFC6839]], - reject the promise passing a loading document failed error.
      3. -
    + is a string representing the IRI of a remote document, await and dereference it as remote document + using LoadDocumentCallback, passing input + for url, + the extractAllScripts option for extractAllScripts.
    • +
  • If necessary, transform document + from remote document into the internal representation. + If document cannot be transformed to the internal representation, + reject promise passing a loading document failed error.
  • Initialize a new empty active context. - The base IRI of the active context is set to the IRI of the currently being processed document, if available; + The base IRI of the active context is set to the documentUrl + from remote document, if available; otherwise to null. If set, the base option from options overrides the base IRI.
  • If an expandContext option has been passed, @@ -5153,66 +5127,13 @@

    The JsonLdProcessor Interface

    passing the expandContext as local context. If expandContext is a dictionary having an @context member, pass that member's value instead.
    • -
  • Once input has been retrieved, - the response has an HTTP Link Header [[RFC8288]] using the http://www.w3.org/ns/json-ld#context link relation - and a Content-Type of application/json - or any media type with a +json suffix as defined in [[RFC6839]] - except application/ld+json, +
  • If remote document has a contextUrl, update the active context using the Context Processing algorithm, - passing the context referenced in the HTTP Link Header as local context. - The HTTP Link Header is ignored for documents served as application/ld+json - or text/html. - If multiple HTTP Link Headers using the http://www.w3.org/ns/json-ld#context link relation are found, - the promise is rejected with a JsonLdError whose code is set to multiple context link headers - and processing is terminated.
    • -
  • If the retrieved document's Content-Type is text/html, - the processor is not a pure JSON Processor - and the passed input is - a string representing the IRI of a remote document, - extract the content of the JSON-LD script element(s) into original input: -
      -
    1. Set base IRI to the the Document Base URL - of original input, as defined in [[HTML]], - using the existing base IRI as the document's URL. -
      - The use of the Document Base URL - from [[HTML]] for setting the base IRI of the enclosed JSON-LD - is an experimental feature, which may be changed in a future version of this specification. -
      2. -
    2. If the original passed input parameter - contains a fragment identifier, - set source to the textContent - of the script element in input - having an id attribute - that matches the fragment identifier, after decoding percent encoded sequences. - If no element is found - or the located element is not a script element with type application/ld+json, - reject promise passing an invalid script element error. - Set original input to the result of the Extract Script Content algorithm, - using source.
      3. -
    3. Otherwise, if the extractAllScripts option is not present, or false, - locate the first JSON-LD script element in input, - and set source to its textContent, if found. - Set original input to the result of the Extract Script Content algorithm, - using source. - If no JSON-LD script element is found, set original input to a new empty array.
      4. -
    4. Otherwise, set original input to a new empty array. - For each JSON-LD script element in input: -
        -
      1. Set source to its textContent.
        2. -
      2. Set script content to the result of the Extract Script Content algorithm, - using source.
        3. -
      3. If script content is an array, merge it to the end of original input.
        4. -
      4. Otherwise, append script content to original input.
        5. -
      -
      5. -
    -
    • -
  • If necessary, transform original input into the internal representation. - If input cannot be transformed to the internal representation, - reject promise passing a loading document failed error.
    • + passing the contextUrl as local context.
  • Set expanded output to the result of using the Expansion algorithm, - passing the active context and original input as element, + passing the active context and document + from remote document, or input + if there is no remote document as element, and if passed, the frameExpansion and ordered flags in options.
    • @@ -5612,25 +5533,175 @@

    Remote Document and Context Retrieval

    This section details the parameters of that callback and the data structure used to return the retrieved context.

    -
    +

    LoadDocumentCallback

    The LoadDocumentCallback defines a callback that custom document loaders - have to implement to be used to retrieve remote documents and contexts.

    + have to implement to be used to retrieve remote documents and contexts. + Upon successful completion, the callback returns a Promise containing a RemoteDocument. + On failure, the Promise is rejected with an appropriate error code.

    url
    The URL of the remote document or context to load.
    +
    options
    +
    A set of options to determine + the behavior of the callback. See .
    -

    All errors result in the Promise being rejected with a JsonLdError - whose code is set to loading document failed - or multiple context link headers - as described in the next section.

    +

    The following algorithm describes the default callback and places + requirements on implementations of the callback.

    + +
      +
    1. Create a new Promise promise and return it. + The following steps are then executed asynchronously.
      2. +
    2. Set document to the body retrieved from + the resource identified by url, + or by otherwise locating a resource associated with url. + When requesting remote documents the request MUST prefer Content-Type application/ld+json + followed by application/json. + +

      If requestProfile is set, + it MUST be added as a profile on application/ld+json.

      + +

      Processors MAY include other media types using a +json suffix as defined in [[RFC6839]].

      + +

      A full Processor MUST include text/html at any preference level.

      3. +
    3. Set documentUrl to the location of the retrieved resource + considering redirections (exclusive of HTTP status 303 "See Other" redirects + as discussed in [[?cooluris]]).
      4. +
    4. If the retrieved resource's Content-Type is application/json + or any media type with a +json suffix as defined in [[RFC6839]] + except application/ld+json, + and the response has an HTTP Link Header [[RFC8288]] using the http://www.w3.org/ns/json-ld#context link relation, + set contextUrl to the associated href. +

      If multiple HTTP Link Headers using the http://www.w3.org/ns/json-ld#context link relation are found, + the promise is rejected with a JsonLdError whose code is set to multiple context link headers + and processing is terminated.

      +

      Processors MAY transform document to the internal representation + using the Extract Script Content algorithm.

      +

      The HTTP Link Header is ignored for documents served as application/ld+json + or text/html.

      +
      5. +
    5. Otherwise, if the retrieved resource's Content-Type is text/html: +
        +
      1. If the processor is a pure JSON Processor + the promise is rejected with a JsonLdError whose code is set to loading document failed + and processing is terminated.
        2. +
      2. Set documentUrl to the the Document Base URL + of url, as defined in [[HTML]], + using the existing documentUrl as the document's URL. +
        + The use of the Document Base URL + from [[HTML]] for setting the base IRI of the enclosed JSON-LD + is an experimental feature, which may be changed in a future version of this specification. +
        +
        3. +
      3. If the url parameter + contains a fragment identifier, + set source to the textContent + of the script element in document + having an id attribute + that matches the fragment identifier, after decoding percent encoded sequences. +

        If no such element is found, + or the located element is not a JSON-LD script element, + the promise is rejected with a JsonLdError whose code is set to loading document failed + and processing is terminated.

        4. + +
      4. Otherwise, if the profile + option is specified, + set source to the result of transforming the + textContent + of the first script element in document + having an type attribute + of application/ld+json along with the value of the + profile option, if found.
        5. +
      5. If source is undefined and the extractAllScripts option is not present, or false, + set source to the textContent + of the first JSON-LD script element in document. +

        If no such element is found, + or the located element is not a JSON-LD script element, + the promise is rejected with a JsonLdError whose code is set to loading document failed + and processing is terminated.

        6. +
      6. If source is defined and the, + set document to the result of the + Extract Script Content algorithm, + using source, rejecting promise + with a JsonLdError whose code set from the result, if an error is detected + and processing is terminated. +
        7. +
      7. Otherwise, if source is undefined and the extractAllScripts option is not present, or false, + the promise is rejected with a JsonLdError whose code is set to loading document failed + and processing is terminated.
        8. +
      8. Otherwise, if source is undefined and the extractAllScripts option is true, + set document to a new empty array. + For each JSON-LD script element in input: +
          +
        1. Set source to its textContent.
          2. +
        2. Set script content to the result of the Extract Script Content algorithm, + using source, rejecting promise + with a JsonLdError whose code set from the result, if an error is detected + and processing is terminated.
          3. +
        3. If script content is an array, merge it to the end of document.
          4. +
        4. Otherwise, append script content to document.
          5. +
        +
        9. +
      +
      6. +
    6. Otherwise, if the retrieved document's Content-Type is neither + application/json, + application/ld+json, + nor any other media type using a + +json suffix as defined in [[RFC6839]], + reject the promise passing a loading document failed error.
      7. + +
    7. Create a new RemoteDocument remote document using document, + the returned Content-Type (without parameters) as contentType, + any returned profile parameter, + and any contextUrl.
      8. +
    8. Fulfill the promise passing remote document.
      9. +
    + +

    A custom LoadDocumentCallback set via the + documentLoader option might be used + to maintain a local cache of well-known context documents or to implement + application-specific URL protocols.

    +
    + +
    +

    LoadDocumentOptions

    + +

    The LoadDocumentOptions type is used to pass various options + to the LoadDocumentCallback.

    + +
    + +
    +
    extractAllScripts
    +
    If set to true, + when extracting JSON-LD script elements from HTML, + unless a specific fragment identifier is targeted, + extracts all encountered JSON-LD script elements using an array form, if necessary.
    +
    profile
    +
    When the resulting contentType is text/html, + this option determines the profile to use for selecting a JSON-LD script elements.
    +
    requestProfile
    +
    One or more IRIs to use in the request as a profile parameter. + (See IANA Considerations in [[JSON-LD11]]).
    +
    @@ -5645,6 +5716,7 @@

    RemoteDocument

    USVString documentUrl; any document; USVString contentType; + USVString profile = null; }; --> @@ -5666,7 +5738,10 @@

    RemoteDocument

    This can either be the raw payload or the already parsed document.
    contentType
    The Content-Type - of the loaded document.
    + of the loaded document, exclusive of any optional parameters. +
    profile
    +
    The value of any profile parameter + retrieved as part of the original contentType.
    @@ -5822,7 +5897,7 @@

    JsonLdErrorCode

    invalid script element
    A script element in HTML input which is the target of a fragment identifier - does not have an appropriate type attribute.
    + does not have an appropriate type attribute.
    invalid set or list object
    A set object or list object with disallowed members @@ -5892,10 +5967,6 @@

    Internationalization Considerations

    Open Issues

    The following is a list of issues open at the time of publication.

    -

    Define a "streaming profile" for JSON-LD to help parse data from a stream, rather than require the entire document to be in memory. Also to generate documents that can be so streamed. This would aid in using JSON-LD as a dataset dump format where there are a very large number of quads.

    -

    What if a @context URL response is HTML? - The HTML-based response has the value of potentially providing documentation for the contained context, but we lack a few things and/or need to clarify others. -

    More compact @prefix.

    @@ -6014,6 +6085,8 @@

    Changes since JSON-LD Community Group Final Report

    of type application/ld+json;profile=http://www.w3.org/ns/json-ld#context, or application/ld+json is used as the context for further processing. This allows a mechanism for documenting the content of a context using HTML. +
  • Consolodate RemoteDocument processing into the LoadDocumentCallback + including variations on HTML processing.
    • From 1a4bf00bb2b7212064f62fbfd2a20db261f8cbd8 Mon Sep 17 00:00:00 2001 From: Dave Longley Date: Sat, 4 May 2019 13:22:51 -0700 Subject: [PATCH 2/5] Apply suggestions from code review Committed @dlongley's suggestions. Co-Authored-By: gkellogg --- index.html | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/index.html b/index.html index 64da0284..21eea036 100644 --- a/index.html +++ b/index.html @@ -5110,8 +5110,8 @@

    The JsonLdProcessor Interface

  • If the passed input is a string representing the IRI of a remote document, await and dereference it as remote document using LoadDocumentCallback, passing input - for url, - the extractAllScripts option for extractAllScripts. + for url + and the extractAllScripts option for extractAllScripts.
  • If necessary, transform document from remote document into the internal representation. From 9f11d92bf16d43fa97d6e25c2e9a96ebac68ada9 Mon Sep 17 00:00:00 2001 From: Gregg Kellogg Date: Sat, 4 May 2019 13:50:56 -0700 Subject: [PATCH 3/5] Cleanup some promises language. --- index.html | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/index.html b/index.html index 21eea036..2fbcfa70 100644 --- a/index.html +++ b/index.html @@ -5078,7 +5078,7 @@

    The JsonLdProcessor Interface

    and if passed, the compactArrays and ordered flags in options.
    • -
  • Fulfill the promise passing compacted output +
  • Resolve the promise with compacted output transforming compacted output from the internal representation to a JSON serialization.
    • @@ -5110,8 +5110,8 @@

    The JsonLdProcessor Interface

  • If the passed input is a string representing the IRI of a remote document, await and dereference it as remote document using LoadDocumentCallback, passing input - for url - and the extractAllScripts option for extractAllScripts. + for url, + the extractAllScripts option for extractAllScripts.
  • If necessary, transform document from remote document into the internal representation. @@ -5137,7 +5137,7 @@

    The JsonLdProcessor Interface

    and if passed, the frameExpansion and ordered flags in options.
    • -
  • Fulfill the promise passing expanded output +
  • Resolve the promise with expanded output transforming expanded output from the internal representation to a JSON serialization.
    • @@ -5183,7 +5183,7 @@

    The JsonLdProcessor Interface

    and if passed, the compactArrays and ordered flags in options. -
  • Fulfill the promise passing flattened output +
  • Resolve the promise with flattened output transforming flattened output from the internal representation to a JSON serialization.
    • @@ -5265,7 +5265,7 @@

    The JsonLdProcessor Interface

    as is the support for generalized RDF Datasets and thus the produceGeneralizedRdf option may be also be removed. -
  • Fulfill the promise passing dataset.
    • +
  • Resolve the promise with dataset.
    • @@ -5538,8 +5538,8 @@

    LoadDocumentCallback

    The LoadDocumentCallback defines a callback that custom document loaders have to implement to be used to retrieve remote documents and contexts. - Upon successful completion, the callback returns a Promise containing a RemoteDocument. - On failure, the Promise is rejected with an appropriate error code.

    + The callback returns a Promise resolving to a RemoteDocument. + On failure, the Promise is rejected with an appropriate error code.