JSON format not working properly with compile command

[Renaud11232]

When using --format json with the compile command, the output is not properly formatted as JSON in version 0.3.6-alpha.preview

Successful compilation

Running

./arduino-cli compile --fqbn aruino:avr:uno Arduino/Test --format json

Outputs

On stdout

Sketch uses 444 bytes (1%) of program storage space. Maximum is 30720 bytes.
Global variables use 9 bytes (0%) of dynamic memory, leaving 2039 bytes for local variables. Maximum is 2048 bytes.

Failed compilation

Running

./arduino-cli compile --fqbn aruino:avr:uno Arduino/Test --format json

Outputs

On stdout

Build options changed, rebuilding all
{"Message":"Compilation failed.","Cause":"exit status 1"}

On stderr

/home/renaud/Arduino/Test/Test.ino: In function 'void setup()':
/home/renaud/Arduino/Test/Test.ino:3:1: error: 'a' was not declared in this scope
 a
 ^

[Renaud11232]

I created two sketches. One that should compile correctly and one that shouldn't :

cat Success/Success.ino

void setup() {
}

void loop() {
}

cat Fail/Fail.ino

void setup() {
a;
}

void loop() {
}

Then I ran the following commands :

./arduino-cli compile --format json --fqbn arduino:avr:uno Success

which gave me this

Sketch uses 444 bytes (1%) of program storage space. Maximum is 32256 bytes.
Global variables use 9 bytes (0%) of dynamic memory, leaving 2039 bytes for local variables. Maximum is 2048 bytes.

And this :

./arduino-cli compile --format json --fqbn arduino:avr:uno Fail

Which gave me this

/home/renaud/Fail/Fail.ino: In function 'void setup()':
/home/renaud/Fail/Fail.ino:3:1: error: 'a' was not declared in this scope
 a;
 ^
Error during build: exit status 1

The behavior seem to have slightly changed since I originally created this issue but the problem is still present in 0.9.0

$ ./arduino-cli version
arduino-cli Version: 0.9.0 Commit: e4d02b4

[ubidefeo]

@Renaud11232
I think the confusion originates from you expecting output in JSON format for everything, but this isn't the case.
you can get JSON output for a number of things such as
arduino-cli board list(all) --format json
arduino-cli core list --format json
but you won't get the result of compilation/upload formatted into a JSON string.
you only get the compiler's final output (unless you add -v to your command for verbosity) or, in case you have a C/C++ related error, the error from the compiler.
That's why you get the straigh compiler error when it finds you using a symbol which is not defined.

We will remove that --format option from some commands as we move forward, sorry for the confusion

(@rsora this is another one of those sections in which we should update the help text)

[Renaud11232]

Oh OK
I thought it would be available for all commands as it's a global flag.

Now I know :)

Thanks for the clarification

[masci]

Just to clarify: while I confirm all @ubidefeo said, we also have a bug here.

We're not in full control of the output from the lower layers of the CLI and some informations that should be collected by the logger and correctly displayed through the -v flag leak to the stderr/stdout instead.

[ubidefeo]

@masci I had this conversation with @cmaglie in the past when I had some stuff not showing up in my stdout and he said it's because of the way avrdude pipes output to different places.
I believe we should handle the type of output from the compiler such as @Renaud11232 's case, because an integration would need to parse the final output.
That aside, are we able to funnel output from both stdout and stderr into our final output?
There must be some way of issuing a redirect directive (sorry for the cumbersomeness of the terms)

[masci]

Yes we can and should get control of stdout/stderr for any subprocess the CLI starts, there was a tentative mitigation you can see here #364 but we decided not to proceed because the real fix consists in refactoring the low-level module we use to handle info and error messages "from below" (for the curious, I'm talking about this nasty module https://github.com/arduino/arduino-cli/tree/master/legacy/builder/i18n).

[per1234]

I'm happy to report that this bug no longer occurs. I am able to reproduce it with Arduino CLI 0.9.0, but not with the latest build (unfortunately it is not clear which revision(s) provided the fix):

$ ./arduino-cli version
arduino-cli.exe  Version: nightly-20220325 Commit: b86e5cf Date: 2022-03-25T01:33:19Z

$ ./arduino-cli compile --format json --fqbn arduino:avr:uno Success
{
  "compiler_out": "Sketch uses 444 bytes (1%) of program storage space. Maximum is 32256 bytes.\nGlobal variables use 9 bytes (0%) of dynamic memory, leaving 2039 bytes for local variables. Maximum is 2048 bytes.\n",
  "compiler_err": "",
  "builder_result": {
    "build_path": "C:\\Users\\per\\AppData\\Local\\Temp\\arduino-sketch-0712B837E02723B4AB657D1C910CD79F",
    "executable_sections_size": [
      {
        "name": "text",
        "size": 444,
        "max_size": 32256
      },
      {
        "name": "data",
        "size": 9,
        "max_size": 2048
      }
    ],
    "board_platform": {
      "id": "arduino:avr",
      "version": "1.8.5"
    },
    "build_platform": {
      "id": "arduino:avr",
      "version": "1.8.5"
    }
  },
  "success": true
}

$ ./arduino-cli compile --format json --fqbn arduino:avr:uno Fail
{
  "compiler_out": "",
  "compiler_err": "E:\\arduino\\cli\\arduino-cli_nightly\\Fail\\Fail.ino: In function 'void setup()':\r\nE:\\arduino\\cli\\arduino-cli_nightly\\Fail\\Fail.ino:2:1: error: 'a' was not declared in this scope\r\n a;\r\n ^\r\n\n",
  "builder_result": {
    "build_path": "C:\\Users\\per\\AppData\\Local\\Temp\\arduino-sketch-817C41ED7492CE5CA7449AE16AC6D492",
    "board_platform": {
      "id": "arduino:avr",
      "version": "1.8.5"
    },
    "build_platform": {
      "id": "arduino:avr",
      "version": "1.8.5"
    }
  },
  "success": false
}
Error during build: exit status 1

There is still that "Error during build: exit status 1", but this is an error message from Arduino CLI rather than the compiler, so I don't think it really belongs in the compiler_err field. It is printed to stderr, so should not pose any problems for machine parsing of the JSON output from stdout.