Skip to content

json output does not properly break up compiler messages, warnings, errors #1121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
3 tasks done
marsfan opened this issue Jan 2, 2021 · 0 comments · Fixed by #2182
Closed
3 tasks done

json output does not properly break up compiler messages, warnings, errors #1121

marsfan opened this issue Jan 2, 2021 · 0 comments · Fixed by #2182
Assignees
@cmaglie
Labels
topic: code Related to content of the project itself type: enhancement Proposed improvement
Milestone
Arduino CLI v1.0.0

Comments

@marsfan
Copy link

marsfan commented Jan 2, 2021 •
edited by per1234
Loading

Describe the request

Compiler output and errors should be broken apart into separate JSON elements so to allow parsing by a computer.

Here is a rough example of how I could see it working. This has a pretty complicated output for the "compiler_out" section, but I wanted to show a potential way of offering a complete breakdown of the output.

Possible JSON output structure 
{
    "compiler_out": [
        {
            "type": "message",
            "msg": "Using board 'uno' from platform in folder: C:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.3"
        },
        {
            "type": "message",
            "msg": "Using core 'arduino' from platform in folder: C:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.3"
        },
        {
            "type": "message",
            "msg": "Detecting libraries used..."
        },
        {
            "type": "command",
            "command": "\"C:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\avr-gcc\\7.3.0-atmel3.6.1-arduino7/bin/avr-g++\"",
            "args": [
                "-c",
                "-g",
                "-Os",
                "-w",
                "-std=gnu++11",
                "-fpermissive",
                "-fno-exceptions",
                "-ffunction-sections",
                "-fdata-sections",
                "-fno-threadsafe-statics",
                "-Wno-error=narrowing",
                "-flto",
                "-w",
                "-x",
                "c++",
                "-E",
                "-CC",
                "-mmcu=atmega328p",
                "-DF_CPU=16000000L",
                "-DARDUINO=10607",
                "-DARDUINO_AVR_UNO",
                "-DARDUINO_ARCH_AVR",
                "\"-IC:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.3\\cores\\arduino\"",
                "\"-IC:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.3\\variants\\standard\"",
                "\"C:\\Users\\Gabe\\AppData\\Local\\Temp\\arduino-sketch-89EE469216900734AB12CA2548271FBB\\sketch\\test.ino.cpp\"",
                "-o",
                "nul"
            ]
        },
        {
            "type": "message",
            "msg": "Generating function prototypes..."
        },
        {
            "type": "command",
            "command": "\"C:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\avr-gcc\\7.3.0-atmel3.6.1-arduino7/bin/avr-g++\"",
            "args": [
                "-c",
                "-g",
                "-Os",
                "-w",
                "-std=gnu++11",
                "-fpermissive",
                "-fno-exceptions",
                "-ffunction-sections",
                "-fdata-sections",
                "-fno-threadsafe-statics",
                "-Wno-error=narrowing",
                "-flto",
                "-w",
                "-x",
                "c++",
                "-E",
                "-CC",
                "-mmcu=atmega328p",
                "-DF_CPU=16000000L",
                "-DARDUINO=10607",
                "-DARDUINO_AVR_UNO",
                "-DARDUINO_ARCH_AVR",
                "\"-IC:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.3\\cores\\arduino\"",
                "\"-IC:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.3\\variants\\standard\"",
                "\"C:\\Users\\Gabe\\AppData\\Local\\Temp\\arduino-sketch-89EE469216900734AB12CA2548271FBB\\sketch\\test.ino.cpp\"",
                "-o",
                "\"C:\\Users\\Gabe\\AppData\\Local\\Temp\\arduino-sketch-89EE469216900734AB12CA2548271FBB\\preproc\\ctags_target_for_gcc_minus_e.cpp\""
            ]
        },
        {
            "type": "command",
            "command": "\"C:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\builtin\\tools\\ctags\\5.8-arduino11/ctags\"",
            "args": [
                "-u",
                "--language-force=c++",
                "-f",
                "-",
                "--c++-kinds=svpf",
                "--fields=KSTtzns",
                "--line-directives",
                "\"C:\\Users\\Gabe\\AppData\\Local\\Temp\\arduino-sketch-89EE469216900734AB12CA2548271FBB\\preproc\\ctags_target_for_gcc_minus_e.cpp\""
            ]
        },
        {
            "type": "message",
            "msg": "Compiling sketch..."
        },
        {
            "type": "command",
            "command": "\"C:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\avr-gcc\\7.3.0-atmel3.6.1-arduino7/bin/avr-g++\"",
            "args": [
                "-c",
                "-g",
                "-Os",
                "-Wall",
                "-Wextra",
                "-std=gnu++11",
                "-fpermissive",
                "-fno-exceptions",
                "-ffunction-sections",
                "-fdata-sections",
                "-fno-threadsafe-statics",
                "-Wno-error=narrowing",
                "-MMD",
                "-flto",
                "-mmcu=atmega328p",
                "-DF_CPU=16000000L",
                "-DARDUINO=10607",
                "-DARDUINO_AVR_UNO",
                "-DARDUINO_ARCH_AVR",
                "\"-IC:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.3\\cores\\arduino\"",
                "\"-IC:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.3\\variants\\standard\"",
                "\"C:\\Users\\Gabe\\AppData\\Local\\Temp\\arduino-sketch-89EE469216900734AB12CA2548271FBB\\sketch\\test.ino.cpp\"",
                "-o",
                "\"C:\\Users\\Gabe\\AppData\\Local\\Temp\\arduino-sketch-89EE469216900734AB12CA2548271FBB\\sketch\\test.ino.cpp.o\"\n"
            ]
        }
    ],
    "compiler_err": [
        {
            "file": "D:\\Desktop\\Example\\test\\test.ino",
            "function": "void setup()",
            "line": 2,
            "column": 5,
            "level": "error",
            "message": "'a' was not declared in this scope",
            "code": "     a++;\r\n     ^\r"
        },
        {
            "file": "D:\\Desktop\\Example\\test\\test.ino",
            "function": "void loop()",
            "line": 6,
            "column": 5,
            "level": "error",
            "message": "'b' was not declared in this scope",
            "code": "     b++\r\n     ^\r\n"
        }
    ],
    "builder_result": {
        "build_path": "C:\\Users\\Gabe\\AppData\\Local\\Temp\\arduino-sketch-89EE469216900734AB12CA2548271FBB"
    },
    "success": false
}

Breaking individual compiler commands and errors into separate elements in the machine readable output formats makes it much easier to parse.

Describe the current behavior

Demonstration Sketch

void setup(){
    a++;
}

void loop(){
    b++
}

(Errors are deliberate to cause compile to fail.)

Demo

$ arduino-cli version
arduino-cli.exe  Version: git-snapshot Commit: b3e8f8a4 Date: 2023-01-17T06:37:36Z

$ arduino-cli compile -b "arduino:avr:uno" -v --warnings all --format json /tmp/FooSketch
{
  "compiler_out": "Using board 'uno' from platform in folder: C:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.6\nUsing core 'arduino' from platform in folder: C:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.6\nDetecting libraries used...\n\"C:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\tools\\\\avr-gcc\\\\7.3.0-atmel3.6.1-arduino7/bin/avr-g++\" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -flto -w -x c++ -E -CC -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10607 -DARDUINO_AVR_UNO -DARDUINO_ARCH_AVR \"-IC:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\hardware\\\\avr\\\\1.8.6\\\\cores\\\\arduino\" \"-IC:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\hardware\\\\avr\\\\1.8.6\\\\variants\\\\standard\" \"C:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Temp\\\\arduino\\\\sketch-2405C4C9F90028537C79B0744BCE536E\\\\sketch\\\\test.ino.cpp\" -o nul\nGenerating function prototypes...\n\"C:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\tools\\\\avr-gcc\\\\7.3.0-atmel3.6.1-arduino7/bin/avr-g++\" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -flto -w -x c++ -E -CC -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10607 -DARDUINO_AVR_UNO -DARDUINO_ARCH_AVR \"-IC:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\hardware\\\\avr\\\\1.8.6\\\\cores\\\\arduino\" \"-IC:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\hardware\\\\avr\\\\1.8.6\\\\variants\\\\standard\" \"C:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Temp\\\\arduino\\\\sketch-2405C4C9F90028537C79B0744BCE536E\\\\sketch\\\\test.ino.cpp\" -o \"C:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Temp\\\\arduino\\\\sketch-2405C4C9F90028537C79B0744BCE536E\\\\preproc\\\\ctags_target_for_gcc_minus_e.cpp\"\n\"C:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\builtin\\\\tools\\\\ctags\\\\5.8-arduino11/ctags\" -u --language-force=c++ -f - --c++-kinds=svpf --fields=KSTtzns --line-directives \"C:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Temp\\\\arduino\\\\sketch-2405C4C9F90028537C79B0744BCE536E\\\\preproc\\\\ctags_target_for_gcc_minus_e.cpp\"\nCompiling sketch...\n\"C:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\tools\\\\avr-gcc\\\\7.3.0-atmel3.6.1-arduino7/bin/avr-g++\" -c -g -Os -Wall -Wextra -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -MMD -flto -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10607 -DARDUINO_AVR_UNO -DARDUINO_ARCH_AVR \"-IC:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\hardware\\\\avr\\\\1.8.6\\\\cores\\\\arduino\" \"-IC:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Arduino15\\\\packages\\\\arduino\\\\hardware\\\\avr\\\\1.8.6\\\\variants\\\\standard\" \"C:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Temp\\\\arduino\\\\sketch-2405C4C9F90028537C79B0744BCE536E\\\\sketch\\\\test.ino.cpp\" -o \"C:\\\\Users\\\\Gabe\\\\AppData\\\\Local\\\\Temp\\\\arduino\\\\sketch-2405C4C9F90028537C79B0744BCE536E\\\\sketch\\\\test.ino.cpp.o\"\n",
  "compiler_err": "C:\\Users\\Gabe\\AppData\\Local\\Temp\\FooSketch\\test.ino: In function 'void setup()':\r\nC:\\Users\\Gabe\\AppData\\Local\\Temp\\FooSketch\\test.ino:2:5: error: 'a' was not declared in this scope\r\n     a++;\r\n     ^\r\nC:\\Users\\Gabe\\AppData\\Local\\Temp\\FooSketch\\test.ino: In function 'void loop()':\r\nC:\\Users\\Gabe\\AppData\\Local\\Temp\\FooSketch\\test.ino:6:5: error: 'b' was not declared in this scope\r\n     b++\r\n     ^\r\n\n",
  "builder_result": {
    "build_path": "C:\\Users\\Gabe\\AppData\\Local\\Temp\\arduino\\sketch-2405C4C9F90028537C79B0744BCE536E",
    "board_platform": {
      "id": "arduino:avr",
      "version": "1.8.6",
      "install_dir": "C:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.6"
    },
    "build_platform": {
      "id": "arduino:avr",
      "version": "1.8.6",
      "install_dir": "C:\\Users\\Gabe\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.6"
    }
  },
  "success": false,
  "error": "Error during build: exit status 1"
}

🙁 It is difficult to parse the machine readable output from the compile command because the individual compiler commands and errors are all combined into the same field.

Arduino CLI version

Original report

0.14.0

Last verified with

b3e8f8a

Operating system

Windows

Operating system version

10, 11

Additional context

This is a pretty big undertaking, and could take time to implement. Ideally, the error output is tackled first, as it is more likely to be useful for the most people.

Issue checklist

  • I searched for previous requests in the issue tracker
  • I verified the feature was still missing when using the latest nightly build
  • My request contains all necessary details
@per1234 per1234 added component/CLI type: enhancement Proposed improvement labels Jan 2, 2021
@per1234 per1234 reopened this Mar 30, 2021
@rsora rsora added topic: CLI Related to the command line interface and removed topic: CLI labels Sep 16, 2021
@per1234 per1234 added topic: code Related to content of the project itself and removed topic: CLI Related to the command line interface labels Mar 25, 2022
@umbynos umbynos added this to the Arduino CLI 1.0 milestone Nov 25, 2022
@cmaglie cmaglie mentioned this issue Apr 26, 2023
Open
3 tasks
@cmaglie cmaglie mentioned this issue May 19, 2023
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cmaglie cmaglie

Labels
topic: code Related to content of the project itself type: enhancement Proposed improvement
Projects
None yet
Milestone
Arduino CLI v1.0.0
Development

Successfully merging a pull request may close this issue.

Report compiler errors in machine readable format cmaglie/arduino-cli
5 participants
@cmaglie @per1234 @marsfan @umbynos @rsora