Clarification is required on the duplicate Library and LibraryRelease properties. #928

kittaakos · 2020-08-28T07:35:26Z

The following properties are available on both the LibraryRelease and the Library objects. Is this something you’re planning to change later, or the gRPC API is like that by design. If it’s by design, why, and what does it solve?

author,
maintainer,
sentence,
paragraph,
website,
category,
architectures, and
types.

It does not cause any errors, no need to change it right now (if you want to change it at all), but as a gRPC API consumer, I want to understand the why.

Thank you!

arduino-cli/rpc/commands/lib.proto

Lines 135 to 155 in 3c16c05

    
           message LibraryRelease { 
        
               // Value of the `author` field in library.properties. 
        
               string author = 1; 
        
               // Value of the `version` field in library.properties. 
        
               string version = 2; 
        
               // Value of the `maintainer` field in library.properties. 
        
               string maintainer = 3; 
        
               // Value of the `sentence` field in library.properties. 
        
               string sentence = 4; 
        
               // Value of the `paragraph` field in library.properties. 
        
               string paragraph = 5; 
        
               // Value of the `url` field in library.properties. 
        
               string website = 6; 
        
               // Value of the `category` field in library.properties. 
        
               string category = 7; 
        
               // Value of the `architectures` field in library.properties. 
        
               repeated string architectures = 8; 
        
               // The type categories of the library, as defined in the libraries index. 
        
               // Possible values: `Arduino`, `Partner`, `Recommended`, `Contributed`, 
        
               // `Retired`. 
        
               repeated string types = 9;

arduino-cli/rpc/commands/lib.proto

Lines 213 to 232 in 3c16c05

    
           message Library { 
        
               // The library's directory name. 
        
               string name = 1; 
        
               // Value of the `author` field in library.properties. 
        
               string author = 2; 
        
               // Value of the `maintainer` field in library.properties. 
        
               string maintainer = 3; 
        
               // Value of the `sentence` field in library.properties. 
        
               string sentence = 4; 
        
               // Value of the `paragraph` field in library.properties. 
        
               string paragraph = 5; 
        
               // Value of the `url` field in library.properties. 
        
               string website = 6; 
        
               // Value of the `category` field in library.properties. 
        
               string category = 7; 
        
               // Value of the `architectures` field in library.properties. 
        
               repeated string architectures = 8; 
        
               // The type categories of the library. Possible values: `Arduino`, 
        
               // `Partner`, `Recommended`, `Contributed`, `Retired`. 
        
               repeated string types = 9;

The text was updated successfully, but these errors were encountered:

cmaglie · 2020-09-01T11:26:31Z

LibraryRelease contains the metadata that comes from the library_index.json, so it maps the index 1:1 and it's basically the result of the search when you're looking for a not-installed library.
Library contains the metadata that are extracted by scanning the installed libraries and looking into their library.properties. An installed library may not be present in the libraries index (because it may have been manually installed for example).

Some fields overlap (because they are both present in the index and in the library.properties) some other not (for example the URL to download the library is present only in LibaryRelease, or vice-versa for the install_dir is present only in the Library).

So:
LibraryRelease -> a library release in the library_index
Library -> an installed library in the user sketchbook

Now, for completeness let's see some operations on the libraries (I've omitted the parts that are not relevant):

message LibrarySearchResp {
    repeated SearchedLibrary libraries = 1;
}

message SearchedLibrary {
    // Library name.
    string name = 1;
    // The index data for the available versions of the library. The key of the
    // map is the library version.
    map<string, LibraryRelease> releases = 2;
    // The index data for the latest version of the library.
    LibraryRelease latest = 3;
}

The search operation returns all the release of a library as a map<version, LibraryRelease> because these metadata are obtained from the index (since the libraries are not installed yet). It also returns a LibraryRelease with the latest version available.

message LibraryListResp {
    repeated InstalledLibrary installed_library = 1;
}

message InstalledLibrary {
    // Information about the library.
    Library library = 1;
    // When the `updatable` field of the `LibraryList` request is set to `true`,
    // this will contain information on the latest version of the library in the
    // libraries index.
    LibraryRelease release = 2;
}

the list operation instead returns information about the installed libraries. So the structure InstalledLibrary contains a Library but... if an update of the library is available it also links a LibraryRelease (because the update is not installed but comes from the index!).

Hope this clarifies things a bit!

kittaakos · 2020-09-02T09:31:20Z

Thank you! Unfortunately, I still cannot acquire the provides_includes from the available set of CLI commands.

When I run lib search, provides_includes is available:

% ./arduino-cli lib search "AllThingsTalk LoRaWAN SDK" --format json
{
  "libraries": [
    {
      "name": "AllThingsTalk LoRaWAN SDK",
      "releases": {
        "3.0.0": {
          "author": "AllThingsTalk",
          "version": "3.0.0",
          "maintainer": "Alain Cuypers \[email protected]\u003e",
          "sentence": "LoRaWAN SDK for AllThingsTalk Maker.",
          "paragraph": "Compatible with Microchip RN2483 and RN2903 devices.",
          "website": "https://github.com/allthingstalk/arduino-lorawan-sdk",
          "category": "Communication",
          "architectures": [
            "*"
          ],
          "types": [
            "Contributed"
          ],
          "resources": {
            "url": "http://downloads.arduino.cc/libraries/github.com/allthingstalk/AllThingsTalk_LoRaWAN_SDK-3.0.0.zip",
            "archivefilename": "AllThingsTalk_LoRaWAN_SDK-3.0.0.zip",
            "checksum": "SHA-256:b33d17c95e1b4c73cd39c7dab62ca97a8a31fbba37f36d95e4a03d7204aad7b7",
            "size": 52001,
            "cachepath": "libraries"
          },
          "provides_includes": [
            "ABPCredentials.h",
            "LoRaModem.h",
            "CborPayload.h"
          ]
        },
        "3.1.0": {
          "author": "AllThingsTalk",
          "version": "3.1.0",
          "maintainer": "Alain Cuypers \[email protected]\u003e",
          "sentence": "LoRaWAN SDK for AllThingsTalk Maker",
          "paragraph": "Compatible with Microchip RN2483 and RN2903 devices.",
          "website": "https://github.com/allthingstalk/arduino-lorawan-sdk",
          "category": "Communication",
          "architectures": [
            "*"
          ],
          "types": [
            "Contributed"
          ],
          "resources": {
            "url": "http://downloads.arduino.cc/libraries/github.com/allthingstalk/AllThingsTalk_LoRaWAN_SDK-3.1.0.zip",
            "archivefilename": "AllThingsTalk_LoRaWAN_SDK-3.1.0.zip",
            "checksum": "SHA-256:6c76df191f5189a8b80826bde0880dba97a6e3d27861cc8533ffce51f400ea48",
            "size": 484781,
            "cachepath": "libraries"
          },
          "provides_includes": [
            "AllThingsTalk_LoRaWAN.h"
          ]
        },
        "3.1.2": {
          "author": "AllThingsTalk",
          "version": "3.1.2",
          "maintainer": "Vanja \[email protected]\u003e",
          "sentence": "LoRaWAN SDK for AllThingsTalk Maker",
          "paragraph": "Compatible with Microchip RN2483 and RN2903 devices.",
          "website": "https://github.com/allthingstalk/arduino-lorawan-sdk",
          "category": "Communication",
          "architectures": [
            "*"
          ],
          "types": [
            "Contributed"
          ],
          "resources": {
            "url": "http://downloads.arduino.cc/libraries/github.com/allthingstalk/AllThingsTalk_LoRaWAN_SDK-3.1.2.zip",
            "archivefilename": "AllThingsTalk_LoRaWAN_SDK-3.1.2.zip",
            "checksum": "SHA-256:8bcf89a38f8e5f672798158e88351da2335a341ae157342716a3f4705b52b29c",
            "size": 497266,
            "cachepath": "libraries"
          },
          "provides_includes": [
            "AllThingsTalk_LoRaWAN.h"
          ]
        },
        "3.1.3": {
          "author": "AllThingsTalk",
          "version": "3.1.3",
          "maintainer": "Vanja \[email protected]\u003e",
          "sentence": "LoRaWAN SDK for AllThingsTalk Maker",
          "paragraph": "Compatible with Microchip RN2483 and RN2903 devices.",
          "website": "https://github.com/allthingstalk/arduino-lorawan-sdk",
          "category": "Communication",
          "architectures": [
            "*"
          ],
          "types": [
            "Contributed"
          ],
          "resources": {
            "url": "http://downloads.arduino.cc/libraries/github.com/allthingstalk/AllThingsTalk_LoRaWAN_SDK-3.1.3.zip",
            "archivefilename": "AllThingsTalk_LoRaWAN_SDK-3.1.3.zip",
            "checksum": "SHA-256:85f8347d63d33320ce8da291e226277cf4d26c242cbcba58948fa1019329d1bb",
            "size": 499001,
            "cachepath": "libraries"
          },
          "provides_includes": [
            "AllThingsTalk_LoRaWAN.h"
          ]
        },
        "3.1.4": {
          "author": "AllThingsTalk",
          "version": "3.1.4",
          "maintainer": "Vanja \[email protected]\u003e",
          "sentence": "LoRaWAN SDK for AllThingsTalk Maker",
          "paragraph": "Compatible with Microchip RN2483 and RN2903 devices.",
          "website": "https://github.com/allthingstalk/arduino-lorawan-sdk",
          "category": "Communication",
          "architectures": [
            "*"
          ],
          "types": [
            "Contributed"
          ],
          "resources": {
            "url": "http://downloads.arduino.cc/libraries/github.com/allthingstalk/AllThingsTalk_LoRaWAN_SDK-3.1.4.zip",
            "archivefilename": "AllThingsTalk_LoRaWAN_SDK-3.1.4.zip",
            "checksum": "SHA-256:3a136c0cad72d1cbee3c6ff00cd8bf6ab497d7ba4de58c9082a76425ec30b79e",
            "size": 499691,
            "cachepath": "libraries"
          },
          "provides_includes": [
            "AllThingsTalk_LoRaWAN.h"
          ]
        },
        "3.1.5": {
          "author": "AllThingsTalk",
          "version": "3.1.5",
          "maintainer": "Vanja \[email protected]\u003e",
          "sentence": "LoRaWAN SDK for AllThingsTalk Maker",
          "paragraph": "Compatible with Microchip RN2483 and RN2903 devices.",
          "website": "https://github.com/allthingstalk/arduino-lorawan-sdk",
          "category": "Communication",
          "architectures": [
            "*"
          ],
          "types": [
            "Contributed"
          ],
          "resources": {
            "url": "http://downloads.arduino.cc/libraries/github.com/allthingstalk/AllThingsTalk_LoRaWAN_SDK-3.1.5.zip",
            "archivefilename": "AllThingsTalk_LoRaWAN_SDK-3.1.5.zip",
            "checksum": "SHA-256:d01908de7ea13b7e732083f091cae55352c8e180de4f90e4ee69bcf2572e4772",
            "size": 499350,
            "cachepath": "libraries"
          },
          "provides_includes": [
            "AllThingsTalk_LoRaWAN.h"
          ]
        },
        "3.1.6": {
          "author": "AllThingsTalk",
          "version": "3.1.6",
          "maintainer": "Vanja \[email protected]\u003e",
          "sentence": "LoRaWAN SDK for AllThingsTalk Maker",
          "paragraph": "Compatible with Microchip RN2483 and RN2903 devices.",
          "website": "https://github.com/allthingstalk/arduino-lorawan-sdk",
          "category": "Communication",
          "architectures": [
            "*"
          ],
          "types": [
            "Contributed"
          ],
          "resources": {
            "url": "http://downloads.arduino.cc/libraries/github.com/allthingstalk/AllThingsTalk_LoRaWAN_SDK-3.1.6.zip",
            "archivefilename": "AllThingsTalk_LoRaWAN_SDK-3.1.6.zip",
            "checksum": "SHA-256:4d6aac4207415f4449d8d44771dd8167e2f32ef6362d2ba0a41d25051ee71c24",
            "size": 502634,
            "cachepath": "libraries"
          },
          "provides_includes": [
            "AllThingsTalk_LoRaWAN.h"
          ]
        }
      },
      "latest": {
        "author": "AllThingsTalk",
        "version": "3.1.6",
        "maintainer": "Vanja \[email protected]\u003e",
        "sentence": "LoRaWAN SDK for AllThingsTalk Maker",
        "paragraph": "Compatible with Microchip RN2483 and RN2903 devices.",
        "website": "https://github.com/allthingstalk/arduino-lorawan-sdk",
        "category": "Communication",
        "architectures": [
          "*"
        ],
        "types": [
          "Contributed"
        ],
        "resources": {
          "url": "http://downloads.arduino.cc/libraries/github.com/allthingstalk/AllThingsTalk_LoRaWAN_SDK-3.1.6.zip",
          "archivefilename": "AllThingsTalk_LoRaWAN_SDK-3.1.6.zip",
          "checksum": "SHA-256:4d6aac4207415f4449d8d44771dd8167e2f32ef6362d2ba0a41d25051ee71c24",
          "size": 502634,
          "cachepath": "libraries"
        },
        "provides_includes": [
          "AllThingsTalk_LoRaWAN.h"
        ]
      }
    }
  ],
  "status": 1
}

So let's make sure the lib is installed:

% ./arduino-cli lib install "AllThingsTalk LoRaWAN SDK"             
AllThingsTalk LoRaWAN SDK depends on AllThingsTalk LoRaWAN [email protected]
Downloading AllThingsTalk LoRaWAN [email protected]...
AllThingsTalk LoRaWAN [email protected] already downloaded
Installing AllThingsTalk LoRaWAN [email protected]...
Already installed AllThingsTalk LoRaWAN [email protected]

List the installed libraries and get the provides_includes info:

% ./arduino-cli lib list AllThingsTalk_LoRaWAN_SDK --format json
[
  {
    "library": {
      "name": "AllThingsTalk_LoRaWAN_SDK",
      "author": "AllThingsTalk",
      "maintainer": "Vanja \[email protected]\u003e",
      "sentence": "LoRaWAN SDK for AllThingsTalk Maker",
      "paragraph": "Compatible with Microchip RN2483 and RN2903 devices.",
      "website": "https://github.com/allthingstalk/arduino-lorawan-sdk",
      "category": "Communication",
      "architectures": [
        "*"
      ],
      "install_dir": "/Users/akos.kitta/Documents/Arduino/libraries/AllThingsTalk_LoRaWAN_SDK",
      "source_dir": "/Users/akos.kitta/Documents/Arduino/libraries/AllThingsTalk_LoRaWAN_SDK/src",
      "real_name": "AllThingsTalk LoRaWAN SDK",
      "version": "3.1.6",
      "license": "Unspecified",
      "location": 1,
      "layout": 1,
      "examples": [
        "/Users/akos.kitta/Documents/Arduino/libraries/AllThingsTalk_LoRaWAN_SDK/examples/Product Development Kit/TrackAndTrace_Lora",
        "/Users/akos.kitta/Documents/Arduino/libraries/AllThingsTalk_LoRaWAN_SDK/examples/Rapid Development Kit/count-visits",
        "/Users/akos.kitta/Documents/Arduino/libraries/AllThingsTalk_LoRaWAN_SDK/examples/Rapid Development Kit/environmental-sensing",
        "/Users/akos.kitta/Documents/Arduino/libraries/AllThingsTalk_LoRaWAN_SDK/examples/Rapid Development Kit/guard-your-stuff",
        "/Users/akos.kitta/Documents/Arduino/libraries/AllThingsTalk_LoRaWAN_SDK/examples/Rapid Development Kit/pushbutton",
        "/Users/akos.kitta/Documents/Arduino/libraries/AllThingsTalk_LoRaWAN_SDK/examples/SDK Examples/BinaryPayloads",
        "/Users/akos.kitta/Documents/Arduino/libraries/AllThingsTalk_LoRaWAN_SDK/examples/SDK Examples/GPSCBOR",
        "/Users/akos.kitta/Documents/Arduino/libraries/AllThingsTalk_LoRaWAN_SDK/examples/SDK Examples/HelloLora",
        "/Users/akos.kitta/Documents/Arduino/libraries/AllThingsTalk_LoRaWAN_SDK/examples/SDK Examples/IotDataPoint",
        "/Users/akos.kitta/Documents/Arduino/libraries/AllThingsTalk_LoRaWAN_SDK/examples/SDK Examples/LoraOptions",
        "/Users/akos.kitta/Documents/Arduino/libraries/AllThingsTalk_LoRaWAN_SDK/examples/SDK Examples/OTAARetry",
        "/Users/akos.kitta/Documents/Arduino/libraries/AllThingsTalk_LoRaWAN_SDK/examples/SDK Examples/instrumentation"
      ]
    },
    "release": {}
  }

Ouch, release is not there. But no worries, the documentation of the InstalledLibrary says:

When the updatable field of the LibraryList request is set to true, this will contain information on the latest version of the library in the libraries index.

Let's force the updatable flag to access the release property:

% ./arduino-cli lib list AllThingsTalk_LoRaWAN_SDK --updatable --format json
[]%

Ouch, the lib is already up to date, so I cannot get this info other than running lib search and lib list together and make some string matching based on the name of the library. Or is it the real_name I should use? (#932)

kittaakos · 2020-09-02T13:05:12Z

running lib search and lib list together and make some string matching based on the name of the library

It works for user libraries, but it does not work for libs bundled with a core, for instance for EEPROM. EEPROM is not listed with lib search. Can you please recommend a workaround for this limitation, @cmaglie?

ubidefeo · 2020-09-02T13:31:52Z

@kittaakos
you have the --all flag to add libraries in cores to the list

kittaakos · 2020-09-02T13:35:59Z

@kittaakos
you have the --all flag to add libraries in cores to the list

It's about lib search, @ubidefeo and not lib list.

lib search does not support --all:

% ./arduino-cli lib search EEPROM --all
Error: unknown flag: --all

cmaglie · 2020-09-02T13:44:23Z

Do you need includes for the installed libraries?

kittaakos · 2020-09-02T13:55:54Z

Do you need includes for the installed libraries?

I am not sure if I understood it, but I want to clone this feature for the Pro IDE. I need the includes for user libs and libs packaged with cores.

kittaakos · 2020-09-04T11:12:06Z

Hope this clarifies things a bit!

Yes, thank you!

The rest was addressed as part of #934.

github-actions bot added the tracked label Aug 28, 2020

kittaakos mentioned this issue Aug 28, 2020

Added 'lib examples' command #905

Merged

cmaglie mentioned this issue Sep 2, 2020

Added provides_includes field in Library gRPC message #934

Merged

kittaakos closed this as completed Sep 4, 2020

kittaakos mentioned this issue Sep 9, 2020

As a gRPC consumer I want to retrieve LibraryRelease#provides_includes with a LibraryListReq request #902

Closed

per1234 added type: enhancement Proposed improvement conclusion: resolved Issue was resolved topic: code Related to content of the project itself topic: gRPC Related to the gRPC interface labels Jun 18, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clarification is required on the duplicate Library and LibraryRelease properties. #928

Clarification is required on the duplicate Library and LibraryRelease properties. #928

kittaakos commented Aug 28, 2020

cmaglie commented Sep 1, 2020

kittaakos commented Sep 2, 2020

kittaakos commented Sep 2, 2020

ubidefeo commented Sep 2, 2020

kittaakos commented Sep 2, 2020

cmaglie commented Sep 2, 2020

kittaakos commented Sep 2, 2020

kittaakos commented Sep 4, 2020

Clarification is required on the duplicate Library and LibraryRelease properties. #928

Clarification is required on the duplicate Library and LibraryRelease properties. #928

Comments

kittaakos commented Aug 28, 2020

cmaglie commented Sep 1, 2020

kittaakos commented Sep 2, 2020

kittaakos commented Sep 2, 2020

ubidefeo commented Sep 2, 2020

kittaakos commented Sep 2, 2020

cmaglie commented Sep 2, 2020

kittaakos commented Sep 2, 2020

kittaakos commented Sep 4, 2020