CLI cannot gracefully handle invalid 3rd party URLs when initializing the index

[kittaakos]

Describe the problem

I am working on arduino/arduino-ide#2021 after #2119. I experience error messages when I initialize the directories.data folder via core update-index, and an invalid 3rd party URL is defined. The same problem is present via the gRPC API.

To reproduce

I have a bleeding edge CLI from 1a85d16. Probably, the issue can be reproduced from 62bb4c4.

./arduino-cli version                         
arduino-cli  Version: git-snapshot Commit: 1a85d168 Date: 2023-08-02T10:28:17Z

This is my CLI directories and additional_urls config:

cat ~/.arduinoIDE/arduino-cli.yaml | yq .directories

{
  "builtin": {
    "libraries": "/Users/a.kitta/Library/Arduino15/libraries"
  },
  "data": "/Users/a.kitta/Library/Arduino15",
  "downloads": "/Users/a.kitta/Library/Arduino15/staging",
  "user": "/Users/a.kitta/Documents/Arduino"
}

cat ~/.arduinoIDE/arduino-cli.yaml | yq .board_manager.additional_urls

[
  "https://invalidUrl"
]

My directories.data location is "empty":

ls ~/Library/Arduino15
inventory.yaml

❌ Update the indexes using the CLI config. It fails:

./arduino-cli core update-index --config-file ~/.arduinoIDE/arduino-cli.yaml --format json

{
  "error": "Some indexes could not be updated.",
  "warnings": [
    "Error initializing instance: Loading index file: loading json index file /Users/a.kitta/Library/Arduino15: read /Users/a.kitta/Library/Arduino15: is a directory"
  ]
}

❌ Try to run a platform search. It also fails:

./arduino-cli core search "arduino:avr"  --config-file ~/.arduinoIDE/arduino-cli.yaml --format json

{
  "error": "Some indexes could not be updated.",
  "warnings": [
    "Error initializing instance: Loading index file: loading json index file /Users/a.kitta/Library/Arduino15: read /Users/a.kitta/Library/Arduino15: is a directory"
  ]
}

Expected behavior

The CLI can gracefully handle when a 3rd party package URL is invalid during the index update.
The CLI reports if the URL is invalid. This is working as expected in IDE 2.1.1 with CLI 0.33.1.

If I use the default text output, I see the expected no such host error. But it's missing from the JSON and gRPC levels.

./arduino-cli core search "arduino:avr"  --config-file ~/.arduinoIDE/arduino-cli.yaml             
Downloading index: package_index.tar.bz2 downloaded                                             
Downloading index: . Get "https://invalidUrl": dial tcp: lookup invalidUrl: no such host
Error initializing instance: Loading index file: loading json index file /Users/a.kitta/Library/Arduino15: read /Users/a.kitta/Library/Arduino15: is a directory
Downloading index: package_index.tar.bz2 downloaded                                             
Downloading index: . Get "https://invalidUrl": dial tcp: lookup invalidUrl: no such host
Some indexes could not be updated.

Arduino CLI version

62bb4c4

Operating system

macOS

Operating system version

13.4.1

Additional context

I have noticed that although there is an error message, the primary package and the libraries index files are there, and the tools exist.

rm -rf ~/Library/Arduino15

./arduino-cli core update-index --config-file ~/.arduinoIDE/arduino-cli.yaml --format json
{
  "error": "Some indexes could not be updated.",
  "warnings": [
    "Error initializing instance: Loading index file: loading json index file /Users/a.kitta/Library/Arduino15: read /Users/a.kitta/Library/Arduino15: is a directory"
  ]
}

tree ~/Library/Arduino15
/Users/a.kitta/Library/Arduino15
├── inventory.yaml
├── library_index.json
├── library_index.json.sig
├── package_index.json
├── package_index.json.sig
├── packages
│   └── builtin
│       └── tools
│           ├── ctags
│           │   └── 5.8-arduino11
│           │       └── ctags
│           ├── dfu-discovery
│           │   └── 0.1.2
│           │       ├── LICENSE.txt
│           │       └── dfu-discovery
│           ├── mdns-discovery
│           │   └── 1.0.9
│           │       ├── LICENSE.txt
│           │       └── mdns-discovery
│           ├── serial-discovery
│           │   └── 1.4.0
│           │       ├── LICENSE.txt
│           │       └── serial-discovery
│           └── serial-monitor
│               └── 0.13.0
│                   ├── LICENSE.txt
│                   └── serial-monitor
├── staging
│   └── packages
│       ├── ctags-5.8-arduino11-pm-x86_64-apple-darwin.zip
│       ├── dfu-discovery_v0.1.2_macOS_64bit.tar.gz
│       ├── mdns-discovery_v1.0.9_macOS_64bit.tar.gz
│       ├── serial-discovery_v1.4.0_macOS_64bit.tar.gz
│       └── serial-monitor_v0.13.0_macOS_64bit.tar.gz
└── tmp

16 directories, 19 files

I have also noticed that executing any subsequent core search commands produce the same error message, while core list does not.

./arduino-cli core search "" --config-file ~/.arduinoIDE/arduino-cli.yaml --format json
{
  "error": "Some indexes could not be updated.",
  "warnings": [
    "Error initializing instance: Loading index file: loading json index file /Users/a.kitta/Library/Arduino15: read /Users/a.kitta/Library/Arduino15: is a directory"
  ]
}

./arduino-cli core list --config-file ~/.arduinoIDE/arduino-cli.yaml --format json
[]

Issue checklist

• I searched for previous reports in the issue tracker
• I verified the problem still occurs when using the nightly build
• My report contains all necessary details

[kittaakos]

On the gRPC level, I have noticed that there is the detailsList property of the RPC status object:

{
    "code": 9,
    "message": "Loading index file: loading json index file /var/folders/z1/xkw1yh5n7rz4n8djprp1mdn80000gn/T/d-202372-86048-mewa12.usv9/.testArduinoIDE/data/Arduino15: read /var/folders/z1/xkw1yh5n7rz4n8djprp1mdn80000gn/T/d-202372-86048-mewa12.usv9/.testArduinoIDE/data/Arduino15: is a directory",
    "detailsList": [
        {
            "typeUrl": "type.googleapis.com/cc.arduino.cli.commands.v1.FailedInstanceInitError",
            "value": "CAISlwJMb2FkaW5nIGluZGV4IGZpbGU6IGxvYWRpbmcganNvbiBpbmRleCBmaWxlIC92YXIvZm9sZGVycy96MS94a3cxeWg1bjdyejRuOGRqcHJwMW1kbjgwMDAwZ24vVC9kLTIwMjM3Mi04NjA0OC1tZXdhMTIudXN2OS8udGVzdEFyZHVpbm9JREUvZGF0YS9BcmR1aW5vMTU6IHJlYWQgL3Zhci9mb2xkZXJzL3oxL3hrdzF5aDVuN3J6NG44ZGpwcnAxbWRuODAwMDBnbi9UL2QtMjAyMzcyLTg2MDQ4LW1ld2ExMi51c3Y5Ly50ZXN0QXJkdWlub0lERS9kYXRhL0FyZHVpbm8xNTogaXMgYSBkaXJlY3Rvcnk="
        }
    ]
}

Unfortunately, the base64 decoded value is the incorrect error message:

Loading index file: loading json index file /var/folders/z1/xkw1yh5n7rz4n8djprp1mdn80000gn/T/d-202372-86048-mewa12.usv9/.testArduinoIDE/data/Arduino15: read /var/folders/z1/xkw1yh5n7rz4n8djprp1mdn80000gn/T/d-202372-86048-mewa12.usv9/.testArduinoIDE/data/Arduino15: is a directory