[grpc] lib list and board details throw incorrect error when directories.data folder does not exist at daemon startup

[kittaakos]

Bug Report

As a gRPC consumer of the CLI, I receive an incorrect error message from lib list when the directories.data location does not exist before the InitRequest.

Current behavior

I cannot attach code, but this is my workflow:

• Clean directories.data location (on macOS is it rm -rf ~/Library/Arduino15/),
• Create a client with the InitRequest.
• Note: at this point, the Arduino15 folder exists on my filesystem:

Arduino15 % ls -al
total 8
drwxr-xr-x   6 akos.kitta  staff   192 Apr 12 17:02 .
drwx------@ 92 akos.kitta  staff  2944 Apr 12 17:02 ..
-rw-r--r--   1 akos.kitta  staff   104 Apr 12 17:02 inventory.yaml
drwxr-xr-x   3 akos.kitta  staff    96 Apr 12 17:02 packages
drwxr-xr-x   3 akos.kitta  staff    96 Apr 12 17:02 staging
drwxr-xr-x   2 akos.kitta  staff    64 Apr 12 17:02 tmp

• Run a LibraryListRequest

{
  "instance": { "id": 1 },
  "all": true,
  "updatable": false,
  "name": "",
  "fqbn": "arduino:samd:arduino_zero_edbg"
}

• It fails, but with an incorrect error message:

2 UNKNOWN: loading board data: unknown package arduino

Expected behavior

• The LibraryListRequest fails, but with the following error message:

2 UNKNOWN: loading board data: platform arduino:samd is not installed

Environment

• CLI version (output of arduino-cli version): arduino-cli alpha Version: nightly-20210408 Commit: a4ee670a Date: 2021-04-08T01:26:00Z
• OS and platform:

Additional context

• I had my Arduino Zero board attached.
• I used the default CLI config with two additional 3rd party URLs.
• I tried to run a RescanRequest right after the InitRequest and before the LibraryListRequest. The RescanResponse was completely broken on my side: it had one library and multiple platform errors.

[kittaakos]

The same error happens when I do board details instead of lib list.

[kittaakos]

I do not know if this is still valid, but I checked, and the IDE2 code still has a guard against it:

• https://github.com/arduino/arduino-ide/blob/d6a4b0f910e37c3ecd569ca457f91b577734e4df/arduino-ide-extension/src/node/boards-service-impl.ts#L551-L554
• https://github.com/arduino/arduino-ide/blob/d6a4b0f910e37c3ecd569ca457f91b577734e4df/arduino-ide-extension/src/node/library-service-impl.ts#L186-L190

[kittaakos]

This issue works as expected when using the CLI from a terminal:

./arduino-cli version
arduino-cli  Version: git-snapshot Commit: a58d5adb Date: 2023-01-23T09:37:23Z

rm -rf ~/Library/Arduino15
rm -rf ~/Documents/Arduino/libraries
./arduino-cli lib list -b arduino:samd:arduino_zero_edbg --format json
{
  "error": "Error listing Libraries: Unknown FQBN: platform arduino:samd is not installed"
}

rm -rf ~/Library/Arduino15
rm -rf ~/Documents/Arduino/libraries
./arduino-cli board details -b arduino:samd:arduino_zero_edbg --format json
{
  "error": "Error getting board details: Unknown FQBN: platform arduino:samd is not installed"
}

---

When using the gRPC APIs, the libraries and platform index must be updated before the InitRequest. This is precisely what IDE2 does here. The CLI version is here. Otherwise, the core client is not functional. Steps with grpcurl:

grpcurl \
  -plaintext \
  -import-path ./rpc \
  -proto cc/arduino/cli/commands/v1/commands.proto \
  127.0.0.1:50051 \
  cc.arduino.cli.commands.v1.ArduinoCoreService.Create

{
  "instance": {
    "id": 1
  }
}

grpcurl \
  -plaintext \
  -import-path ./rpc \
  -proto cc/arduino/cli/commands/v1/commands.proto \
  -d '{"instance": {"id": 1}}' \
  127.0.0.1:50051 \
  cc.arduino.cli.commands.v1.ArduinoCoreService.UpdateIndex

{
  "downloadProgress": {
    "start": {
      "url": "https://downloads.arduino.cc/packages/package_index.tar.bz2",
      "label": "Downloading index: package_index.tar.bz2"
    }
  }
}
{
  "downloadProgress": {
    "update": {
      "downloaded": "45861",
      "totalSize": "45861"
    }
  }
}
{
  "downloadProgress": {
    "end": {
      "success": true
    }
  }
}

grpcurl \
  -plaintext \
  -import-path ./rpc \
  -proto cc/arduino/cli/commands/v1/commands.proto \
  -d '{"instance": {"id": 1}}' \
  127.0.0.1:50051 \
  cc.arduino.cli.commands.v1.ArduinoCoreService.UpdateLibrariesIndex

{
  "downloadProgress": {
    "start": {
      "url": "https://downloads.arduino.cc/libraries/library_index.tar.bz2",
      "label": "Downloading index: library_index.tar.bz2"
    }
  }
}
{
  "downloadProgress": {
    "update": {
      "downloaded": "1678461",
      "totalSize": "2357079"
    }
  }
}
{
  "downloadProgress": {
    "update": {
      "downloaded": "2357079",
      "totalSize": "2357079"
    }
  }
}
{
  "downloadProgress": {
    "end": {
      "success": true
    }
  }
}
{
  
}

grpcurl \
  -plaintext \
  -import-path ./rpc \
  -proto cc/arduino/cli/commands/v1/commands.proto \
  -d '{"instance": {"id": 1}}' \
  127.0.0.1:50051 \
  cc.arduino.cli.commands.v1.ArduinoCoreService.Init

{
  "initProgress": {
    "taskProgress": {
      "name": "Downloading missing tool builtin:[email protected]"
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "start": {
        "url": "https://downloads.arduino.cc/discovery/mdns-discovery/mdns-discovery_v1.0.6_macOS_64bit.tar.gz",
        "label": "builtin:[email protected]"
      }
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "update": {
        "downloaded": "1612012",
        "totalSize": "2418658"
      }
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "update": {
        "downloaded": "2418658",
        "totalSize": "2418658"
      }
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "end": {
        "success": true
      }
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "completed": true
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "name": "Installing builtin:[email protected]"
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "message": "builtin:[email protected] installed",
      "completed": true
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "name": "Downloading missing tool builtin:[email protected]"
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "start": {
        "url": "https://downloads.arduino.cc/monitor/serial-monitor/serial-monitor_v0.12.0_macOS_64bit.tar.gz",
        "label": "builtin:[email protected]"
      }
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "update": {
        "downloaded": "1321196",
        "totalSize": "1881420"
      }
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "update": {
        "downloaded": "1881420",
        "totalSize": "1881420"
      }
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "end": {
        "success": true
      }
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "completed": true
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "name": "Installing builtin:[email protected]"
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "message": "builtin:[email protected] installed",
      "completed": true
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "name": "Downloading missing tool builtin:[email protected]"
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "start": {
        "url": "https://downloads.arduino.cc/tools/ctags-5.8-arduino11-pm-x86_64-apple-darwin.zip",
        "label": "builtin:[email protected]"
      }
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "update": {
        "downloaded": "118296",
        "totalSize": "118296"
      }
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "end": {
        "success": true
      }
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "completed": true
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "name": "Installing builtin:[email protected]"
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "message": "builtin:[email protected] installed",
      "completed": true
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "name": "Downloading missing tool builtin:[email protected]"
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "start": {
        "url": "https://downloads.arduino.cc/discovery/serial-discovery/serial-discovery_v1.3.3_macOS_64bit.tar.gz",
        "label": "builtin:[email protected]"
      }
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "update": {
        "downloaded": "1182856",
        "totalSize": "1699756"
      }
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "update": {
        "downloaded": "1699756",
        "totalSize": "1699756"
      }
    }
  }
}
{
  "initProgress": {
    "downloadProgress": {
      "end": {
        "success": true
      }
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "completed": true
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "name": "Installing builtin:[email protected]"
    }
  }
}
{
  "initProgress": {
    "taskProgress": {
      "message": "builtin:[email protected] installed",
      "completed": true
    }
  }
}

grpcurl \
  -plaintext \
  -import-path ./rpc \
  -proto cc/arduino/cli/commands/v1/commands.proto \
  -d '{"instance": {"id": 1}, "fqbn": "arduino:samd:arduino_zero_edbg"}' \
  127.0.0.1:50051 \
  cc.arduino.cli.commands.v1.ArduinoCoreService.LibraryList

ERROR:
  Code: NotFound
  Message: Unknown FQBN: platform arduino:samd is not installed

grpcurl \
  -plaintext \
  -import-path ./rpc \
  -proto cc/arduino/cli/commands/v1/commands.proto \
  -d '{"instance": {"id": 1}, "fqbn": "arduino:samd:arduino_zero_edbg"}' \
  127.0.0.1:50051 \
  cc.arduino.cli.commands.v1.ArduinoCoreService.BoardDetails

ERROR:
  Code: NotFound
  Message: Unknown FQBN: platform arduino:samd is not installed

---

I also verified the behavior in IDE2.

• Arduino SAMD Boards (32-bits ARM Cortex-M0+) [v 1.8.13] is installed,
• Select the Arduino Zero (Programming Port),
• Quit IDE2,
• Delete directories.data and directoties.user/libraries,
• Start IDE2

The caught exceptions on the IDE2 side do not contain this unknown error message anymore. Errors were:

{"code":9,"details":"Platform 'arduino:samd' not found: platform arduino:samd is not installed","metadata":{"content-type":["application/grpc"]}}

{"code":5,"details":"Unknown FQBN: platform arduino:samd is not installed","metadata":{"content-type":["application/grpc"]}}

The (gRPC) commands where

• lib list -b
• board details -b
• (and getting the user fields)

---

• I do not know why the responses are not in JSON.
• I do not know why there is an empty object ({}) during the update progress.
• The error message in terminal should use lower case letters: "error": "Error listing Libraries: Unknown FQBN: platform arduino:samd is not installed" should be "error": "Error listing libraries: Unknown FQBN: platform arduino:samd is not installed"

Feel free to close it.

[Bikappa]

@kittaakos regarding the error formatting, that comes from gprcurl. There is a dedicated -format-error option
Try:

grpcurl \
 -plaintext \
 -import-path ./rpc \
 -proto cc/arduino/cli/commands/v1/commands.proto \
 -d '{"instance": {"id": 1}, "fqbn": "arduino:samd:arduino_zero_edbg"}' \
 -format-error \
 127.0.0.1:50051 \
 cc.arduino.cli.commands.v1.ArduinoCoreService.LibraryList

I'll create an issue for each of the other two points