Invalid json when updating index

[drio]

Bug Report

Current behavior

$ arduino-cli  compile --fqbn esp8266:esp8266:generic neo_project
Updating index: package_index.json downloaded
Updating index: package_index.json.sig downloaded
Updating index: package_esp8266com_index.json downloaded
Updating index: package_adafruit_index.json downloaded
Updating index: package_seeeduino_boards_index.json downloaded
Error creating instance: updating the core index: invalid package index in https://raw.githubusercontent.com/Seeed-Studio/Seeed_Platform/master/package_seeeduino_boards_index.json: invalid character ':' after top-level value
make: *** [neo_project/neo_project.esp8266.esp8266.generic.bin] Error 1

Since yesterday, arduino-cli is failing when updating indexes.

Expected behavior

I believe this is not a arduino-cli issue but a problem on the seeed-studio side.

Environment

• arduino-cli Version: 0.13.0 Commit: 693a045eea420c29ca7027e668eee31bce37365d
• osx catalina 10.15.7 (latest)
• Also tested in linux:

$ uname -a 
Linux ironfist 5.4.72-v7l+  #1356 SMP Thu Oct 22 13:57:51 BST 2020 armv7l GNU/Linux

$ cat /etc/issue
Raspbian GNU/Linux 10 

Additional context

I believe this started to fail for me yesterday (Oct 29).

[ubidefeo]

@drio
If you test the seeduino package URL it returns a 404.
Simply visit this URL https://raw.githubusercontent.com/Seeed-Studio/Seeed_Platform/master/package_seeeduino_boards_index.json and see yourself.
Not sure if they moved it or are having server issue, I'd suggest you remove that entry from your configuration file

[per1234]

For some reason they have changed the name of the package index file:
Seeed-Studio/Seeed_Platform@7d8cd4f

The new URL:
https://raw.githubusercontent.com/Seeed-Studio/Seeed_Platform/master/package_legacy_seeeduino_boards_index.json

So the "invalid package index" error is not any fault of Arduino CLI. However, I do think we should consider whether an inaccessible URL breaking the compilation is the correct behavior. In this case it's a permanently dead URL and so the correct thing to do is to remove it or update it. However, sometimes a web server goes down temporarily, and you wouldn't want to remove the Boards Manager URLs from your configuration just so you can continue to use the Arduino CLI for purposes that don't have any need for access to that web server.

It would also be interesting to see if we can improve the error message. I think it would be more helpful for it to give the HTTP error code rather than trying to interpret it as a package index.

[drio]

Thank you all for your answers.

I'd suggest you remove that entry from your configuration file

I forgot to mention that. I tried but arduino-cli keeps trying to pull the index:

 drio@Rufus [~/dev/iot/sketches/neo_project]
   (master *%) $ arduino-cli config init
Config file written to: /Users/drio/Library/Arduino15/arduino-cli.yaml

   drio@Rufus [~/dev/iot/sketches/neo_project]
   (master *%) $ cat  /Users/drio/Library/Arduino15/arduino-cli.yaml
board_manager:
  additional_urls:
  - http://arduino.esp8266.com/stable/package_esp8266com_index.json
  - https://adafruit.github.io/arduino-board-index/package_adafruit_index.json
  - https://raw.githubusercontent.com/Seeed-Studio/Seeed_Platform/master/package_legacy_seeeduino_boards_index.json
daemon:
  port: "50051"
directories:
  data: /Users/drio/Library/Arduino15
  downloads: /Users/drio/Library/Arduino15/staging
  user: /Users/drio/Documents/Arduino
logging:
  file: ""
  format: text
  level: info
telemetry:
  addr: :9090
  enabled: true

Notice the new valid url (Thank you @per1234).
Despite that, arduino cli tries to download the old url. It seems it is storing and using the old url.
How can I get rid of the old url and make acli to use the new one?

[ubidefeo]

@drio
do you have an arduino-cli.yaml somewhere in your current path or folders up?
we have recently reworked how the CLI looks for a configuration file, and I suggest you download a Nightly Build and clean up your configuration files.
You'll be able to do arduino-cli config init --overwrite and it will get you a clean config file, but remember the CLI first looks in the current directory and then goes up one folder until it finds one or reaches the root (this is also in the fix, @silvanocerza can confirm).
If no file is found, the CLI will look for the default one.

You can also initialise a config file somewhere else and tell the CLI to specifically pick that one passing --config-file PATH_TO_FILE as flag

[per1234]

You can find the location of the active Arduino CLI configuration file with this command:

arduino-cli config dump -v

The verbose output will contain a line with something like this:

INFO[0000] Using config file: /Users/drio/Library/Arduino15/arduino-cli.yaml

[drio]

@ubidefeo I used @per1234 cmd to get the config file and modified it to point to the correct url. Unfortunately ACI is overwriting the file. I will try a nightly build next.
I do not have any other config files around, definitely not in the working directory.
Thank you both.

[ubidefeo]

@drio
I'm gonna go through this with @silvanocerza on monday and see what can be the root of the problem.
Your report is very helpful, thank you for contributing to the bug hunt 👍

[drio]

The latest nightly build fixes the issue.

What I did:

1. Download the nightly binary build
2. Modify the config file so we use the correct package json file url (see above for commands and valid url).
3. Update the index: arduino-cli core update-index
4. Reinstall the platform with arduino-cli core install esp8266:esp8266.
5. Rebuild your sketch

Thank you all for your help.

[silvanocerza]

This is strange, I wouldn't expect the latest nightly to fix this issue, I'm still working on a fix for it.

Could you repeat those steps with the latest release? I'd expect it to work.

[drio]

@silvanocerza It seems also fine (using ff3302fc)