Question About JSON Schema

[mighty-sparrow]

DRAFT

This is a draft proposal of a design change.

Overview

While working on a Java façade for the arduino-cli application, I have struggled with identifying commonalities in the JSON responses. For more background info, I started thinking about this after running a simple set of JUnit tests and submitting issue #95 .

Examples

For example, a call to two different lib commands has two different response formats.

Side Note

I have only tested this on the two lib commands that provide a response, search and list --all
This is visible through the first, three levels of a JSON response. The arduino-cli lib search --format=json yields a result with a structure like:

{
  "libraries":[
    {
      "library":{
        "Name":"ESP8266AVRISP",
        "Author":"Kiril Zyapkov",
        "Maintainer":"Kiril Zyapkov \[email protected]\u003e",
        "Sentence":"AVR In-System Programming over WiFi for ESP8266",
          /* ...additional properties */
      },
      "Available":null
    },
    {
      "library":{
        "Name":"ESP8266httpUpdate",
        "Author":"Markus Sattler",
        "Maintainer":"Markus Sattler",
        "Sentence":"Http Update for ESP8266",
          /* ...additional properties */
      },
      "Available":null
    },

Similarly, the arduino-cli lib search yun --format=json command, yields a very different structure:

{
  "libraries":[
    {
      "Name":"SpacebrewYun",
      "Releases":{
        "1.0.0":{
          "Author":"Julio Terra",
          "Version":"1.0.0",
          /* ...additional properties */
        },
        "1.0.1":{
          "Author":"Julio Terra",
          "Version":"1.0.1",
          /* ...additional properties */
        }
      }
    },
    {
      "Name":"Ubidots Arduino YUN",
      "Releases":{
        "2.0.0":{
          "Author":"Ubidots Devel Team \[email protected]\u003e",
          "Version":"2.0.0",
          /* ...additional properties */
        }
      }
    }
  ]
}

Next Steps for Consideration

Propose a location wherein the data structures are published in a consumable format, using standards outlined on json-schema.org.

Example Schema JSON

This is a bit of a long-winded piece of code, but I wanted to include enough of a sample for it to make sense.

{  
  "$id":"https://www.arduino.cc/libraries.schema.json",  
  "$schema":"http://json-schema.org/draft-07/schema#",  
  "description":"An array of available Arduino Libraries.  This may be the result of a search or a 'list' operation from the arduino-cli.",  
  "type":"object",  
  "properties":{  
    "libraries":{  
      "type":"array",  
  "items":{  
        "type":{  
          "$ref":"#/definitions/library"  
  }  
      }  
    }  
  },  
  "definitions":{  
    "library":{  
      "Version":{  
        "$ref":"#/definitions/version"  
  },  
  "ContainerPlatform":{  
        "$ref":"#/definitions/containerPlatform"  
  }  
    },  
  "version":{  
      "type":"object",  
  "properties":{  
        "major":{  
          "type":"integer"  
  },  
  "minor":{  
          "type":"integer"  
  },  
  "build":{  
          "type":"integer"  
  }  
      }  
    },  
  "containerPlatform":{  
      "type":"object",  
  "required":["Version","Resource"],  
  "properties":{  
        "Version":{  
          "$ref":"#/definitions/version"  
  },  
  "Resource":{  
          "$ref":"#/definitions/resource"  
  },  
  "Dependencies":{  
          "$ref":"#definitions/dependency"  
  }  
      },  
  "resource":{  
        "type":"object",  
  "description":"A downloadable resource.",  
  "properties":{  
          "URL":{  
            "type":"string",  
  "description":"A location where this resource may be downloaded."  
  },  
  "ArchiveFilename":{  
            "type":"string",  
  "description":"The full name of the archive to download."  
  },  
  "Checksum":{  
            "type":"string",  
  "description":"A SHA-256 key for validation."  
  },  
  "Size":{  
            "type":"integer",  
  "description":"The size (in bytes) of the downloadable artifact."  
  },  
  "CachePath":{  
            "type":"string",  
  "description":"The directory where the file will be downloaded. This path is relative to your '$ARDUINO_HOME' path."  
  }  
        }  
      },  
  "dependency":{  
        "type":"object",  
  "description":"An array of dependencies required by this Library.",  
  "required":["ToolName","ToolVersion","ToolPackager"],  
  "properties":{  
          "ToolName":{  
            "type":"string",  
  "description":"The name of the tool"  
  },  
  "ToolVersion":{  
            "type":{  
              "$ref":"#/definitions/Version"  
  }  
          },  
  "ToolPackager":{  
            "type":"string",  
  "description":"The packager required at compile-time."  
  }  
        }  
      },  
  "board":{  
        "type":"object",  
  "required":["fqbn","displayName"],  
  "properties":{  
          "fqbn":{  
            "type":"string",  
  "description":"The unique Fully Qualified Board Name"  
  },  
  "displayName":{  
            "type":"string",  
  "description":"A UI-friendly name for this board.  This value may contain spaces or special characters."  
  }  
        }  
      }  
    }  
  }  
}

With a JSON schema published, several tools are available for users to quickly generate object code for their platform of choice. For example, JSON Schema 2 POJO is a simple online tool for Java users. If any changes are made to the API, additional transformation tools may be employed by developers leveraging arduino-cli in their projects.

I think it would be useful to use the current _index.json files as the source for generating a schema. The schema may be used to validate the response to any command-line operations.

I am not a Go programmer, so can't say whether or not any of these Go Validators are suitable for your dev pipeline.

Further Options for Developers

Custom integrations will definitely be unique for anyone leveraging the arduino-cli implementation. Some may want to send the fully loaded list of Libraries. Others may only want to send simple information - like Name, Maintainer, Sentence, Website and Version.

I think it would be helpful to provide developers with the option of consuming a subset of available data.

This may be a predefined, additional flag. For example, consider two options:

arduino-cli lib list [--all] [--format (text|json)] [--fields=(simple|all) | --fields-file=@PathToFile]

The first --fields option provides users with a platform-defined option. The options are all fields or a set of simple fields. The second option is a path to a file with a subset of your published JSON schema.

Your Go code could follow the same pattern; use a JSON schema file packaged with the tool. If a file is provided in the command-line, use that instead of the pre-packaged one.

...clear as mud? 😜

---

Other Example Transformation Tools

• Jolt
• Architect

---

NOTE: I have a separate repository for my jarduino-cli project, but not yet published on GitHub. I can upload examples if it would be helpful.

[philippejadin]

Amen for all, except the option for developers. I would always return the full fields and not make it configurable. The amount of data will never be huge (or could be cached)