Handle Arduino CLI 1.x `core list` command output data format #285

per1234 · 2024-06-13T13:54:44Z

The action parses the output of the arduino-cli core list --format json command.

There were multiple breaking changes to the data format of that output in the 1.0.0 release of Arduino CLI. These caused runs of workflows using the action to fail if the workflow used a platform dependency source other than Boards Manager

Traceback (most recent call last):
  File "/home/runner/work/_actions/arduino/compile-sketches/v1.1.0/compilesketches/compilesketches.py", line 1717, in <module>
    main()  # pragma: no cover
    ^^^^^^
  File "/home/runner/work/_actions/arduino/compile-sketches/v1.1.0/compilesketches/compilesketches.py", line 63, in main
    compile_sketches.compile_sketches()
  File "/home/runner/work/_actions/arduino/compile-sketches/v1.1.0/compilesketches/compilesketches.py", line 219, in compile_sketches
    self.install_platforms()
  File "/home/runner/work/_actions/arduino/compile-sketches/v1.1.0/compilesketches/compilesketches.py", line 334, in install_platforms
    self.install_platforms_from_path(platform_list=platform_list.path)
  File "/home/runner/work/_actions/arduino/compile-sketches/v1.1.0/compilesketches/compilesketches.py", line 522, in install_platforms_from_path
    platform_installation_path = self.get_platform_installation_path(platform=platform)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/_actions/arduino/compile-sketches/v1.1.0/compilesketches/compilesketches.py", line 563, in get_platform_installation_path
    if installed_platform[self.cli_json_key("core list", "ID")] == platform[self.dependency_name_key]:
       ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: string indices must be integers, not 'str'

Change to overall data structure

https://arduino.github.io/arduino-cli/1.0/UPGRADING/#cli-changed-json-output-for-some-lib-core-config-board-and-sketch-commands

arduino-cli core search --format json and arduino-cli core list --format json results are now wrapped under platforms key
{ "platforms": [ {...}, {...} ] }

This is fixed by https://github.com/arduino/compile-sketches/tree/40f6d20f1d0c2fcab12ef7ab9d30439ed1dc79f0

`installed` key renamed to `installed_version`

https://arduino.github.io/arduino-cli/1.0/UPGRADING/#cli-core-list-and-core-search-changed-json-output

CLI core list and core search changed JSON output.

Below is an example of the response containing an object with all possible keys set.

[...]

    "installed_version": "1.6.2",

[...]

This is fixed by https://github.com/arduino/compile-sketches/tree/26705b342943dcb94c56a93270d73c1be2e267ab

Fixes #284

The action parses the output of the `arduino-cli core list --format json` command. There was a breaking change to the data format of that output was changed in the 1.0.0 release of Arduino CLI. The action code is updated to handle the new output, while also retaining backwards compatibility with the pre-1.x data format in order to support workflows that pin the Arduino CLI version to >1.0.0 via the action's "cli-version" input.

The action parses the JSON formatted output of the `arduino-cli core list` command. The action depends on the output having a specific data format, which includes the key names. If the Arduino CLI developers change the name of a key used by the action, it breaks the action. Some time ago, there was a comprehensive review and standardization of the Arduino CLI output key names, which resulted in many changes to the key names. A key name translation system was added to the action code in order to handle that breaking change. Although the action's current code only relies on a couple of keys from a single command, it is planned to use other parts of the output over time (e.g., getting the compilation data now available from the `arduino-cli compile --format json` output instead of parsing the text output as was necessary at the time the action was created). There was an impression of finality to the new key names. For these reasons, the decision was made to implement a comprehensive translation system rather than limiting the scope to only the keys used by the action at the time. Since that time, various key name changes have occurred at various times. Since none of the changed keys were used by the action, this did not cause breakage, but it did render the comprehensive key name translation system outdated. A key used by the action has been renamed once again in the Arduino CLI 1.0.0 release, which requires the translation system to be redesigned and updated. The fact that the comprehensive system has not been kept updated indicates that it is not feasible to maintain such a system. So, rather than investing the significant amount of effort that would be required to review all the changes made to the Arduino CLI output key names and update the comprehensive translation system accordingly, the translation system is trimmed back to only cover the keys actually used by the action. It will be expanded on demand.

The comment was somehow moved to a different line than the code it refers to.

The action parses the output of the `arduino-cli core list --format json` command. The name of one of the keys in that output data used by the action was changed in the 1.0.0 release of Arduino CLI, breaking the action when used with that version of Arduino CLI (which is the default behavior). The action code base already has a key name translation system, which was implemented to handle breakage caused by a previous round of key name changes. However, that system was designed under the assumption that the key names would be stable following the comprehensive redesign that caused the previous changes. It was not designed to handle multiple changes to the name of a given key as has now happened. For this reason, the translation system has been redesigned to support any number of key name changes that might occur.

codecov-commenter · 2024-06-13T13:56:36Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.81%. Comparing base (b35a113) to head (26705b3).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #285   +/-   ##
=======================================
  Coverage   99.81%   99.81%           
=======================================
  Files           2        2           
  Lines        1608     1623   +15     
=======================================
+ Hits         1605     1620   +15     
  Misses          3        3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

per1234 added 4 commits June 13, 2024 05:53

Fix position of comment

a5fd631

The comment was somehow moved to a different line than the code it refers to.

per1234 added type: imperfection Perceived defect in any part of project topic: code Related to content of the project itself labels Jun 13, 2024

per1234 self-assigned this Jun 13, 2024

per1234 merged commit 1bc7c20 into arduino:main Jun 13, 2024
25 checks passed

per1234 deleted the cli-1-format branch June 13, 2024 13:59

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Handle Arduino CLI 1.x `core list` command output data format #285

Handle Arduino CLI 1.x `core list` command output data format #285

per1234 commented Jun 13, 2024

codecov-commenter commented Jun 13, 2024

Handle Arduino CLI 1.x core list command output data format #285

Handle Arduino CLI 1.x core list command output data format #285

Conversation

per1234 commented Jun 13, 2024

Change to overall data structure

installed key renamed to installed_version

CLI core list and core search changed JSON output.

codecov-commenter commented Jun 13, 2024

Codecov Report

Handle Arduino CLI 1.x `core list` command output data format #285

Handle Arduino CLI 1.x `core list` command output data format #285

`installed` key renamed to `installed_version`

CLI `core list` and `core search` changed JSON output.