Spurious "No such file or directory" errors printed in verbose compile output

[per1234]

Describe the problem

In order to save the user from the need to manually configure the compiler "search path" for the individual library dependencies of each sketch, Arduino CLI's compile command includes a "library discovery" step that automatically finds the paths of the library dependencies and passes them to the compilation command.

The following procedure is used recursively for library discovery:

1. Sketch code file is subjected to C++ preprocessing
2. Error messages are parsed to determine the filenames of header files not found in the search path
3. The path of the library providing the header is added to the search path

So error messages specific to missing header files are an expected and integral part of the library discovery process internally.

🐛 These expected error message are printed in the verbose output of the compile command. Since the users don't know how the library discovery system works internally, these error messages might give them the impression that something went wrong during the compilation process.

To reproduce

$ git clone https://github.com/arduino/arduino-cli

[...]

$ cd arduino-cli

$ git checkout 0585435f~1  # checkout the last commit before the regression

$ task build

[...]

$ ./arduino-cli version

arduino-cli.exe  Version: git-snapshot Commit: 9ce2904b Date: 2023-08-11T08:22:02Z

$ ./arduino-cli lib install Arduino_BuiltIn  # Install an arbitrary library to use for the demo

[...]

$ mkdir /tmp/FooSketch

$ printf "#include <Arduino_BuiltIn.h>\nvoid setup() {}\nvoid loop() {}\n" > "/tmp/FooSketch/FooSketch.ino"

$ ./arduino-cli compile --fqbn arduino:avr:uno --verbose /tmp/FooSketch  # there are no confusing error messages in the output

[...]

Detecting libraries used...
"C:\\Users\\per\\AppData\\Local\\Arduino15\\packages\\arduino\\tools\\avr-gcc\\7.3.0-atmel3.6.1-arduino7/bin/avr-g++" -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -flto -w -x c++ -E -CC -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10607 -DARDUINO_AVR_UNO -DARDUINO_ARCH_AVR "-IC:\\Users\\per\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.6\\cores\\arduino" "-IC:\\Users\\per\\AppData\\Local\\Arduino15\\packages\\arduino\\hardware\\avr\\1.8.6\\variants\\standard" "C:\\Users\\per\\AppData\\Local\\Temp\\arduino\\sketches\\2405C4C9F90028537C79B0744BCE536E\\sketch\\FooSketch.ino.cpp" -o nul
Alternatives for Arduino_BuiltIn.h: [[email protected]]
ResolveLibrary(Arduino_BuiltIn.h)
  -> candidates: [[email protected]]

[...]

$ git checkout master  # checkout the last commit before the regression

$ task build

[...]

$ ./arduino-cli version

arduino-cli.exe  Version: git-snapshot Commit: e9c7d86e Date: 2023-08-11T08:00:04Z

$ ./arduino-cli compile --fqbn arduino:avr:uno --verbose /tmp/FooSketch

Detecting libraries used...
C:\Users\per\AppData\Local\Arduino15\packages\arduino\tools\avr-gcc\7.3.0-atmel3.6.1-arduino7/bin/avr-g++ -c -g -Os -w -std=gnu++11 -fpermissive -fno-exceptions -ffunction-sections -fdata-sections -fno-threadsafe-statics -Wno-error=narrowing -flto -w -x c++ -E -CC -mmcu=atmega328p -DF_CPU=16000000L -DARDUINO=10607 -DARDUINO_AVR_UNO -DARDUINO_ARCH_AVR -IC:\Users\per\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.6\cores\arduino -IC:\Users\per\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.6\variants\standard C:\Users\per\AppData\Local\Temp\arduino\sketches\2405C4C9F90028537C79B0744BCE536E\sketch\FooSketch.ino.cpp -o nul
C:\Users\per\AppData\Local\Temp\FooSketch\FooSketch.ino:1:10: fatal error: Arduino_BuiltIn.h: No such file or directory
 #include <Arduino_BuiltIn.h>
          ^~~~~~~~~~~~~~~~~~~
compilation terminated.
Alternatives for Arduino_BuiltIn.h: [[email protected]]
ResolveLibrary(Arduino_BuiltIn.h)
  -> candidates: [[email protected]]

[...]

Sketch uses 444 bytes (1%) of program storage space. Maximum is 32256 bytes.
Global variables use 9 bytes (0%) of dynamic memory, leaving 2039 bytes for local variables. Maximum is 2048 bytes.

Used library    Version Path
Arduino_BuiltIn 1.0.0   C:\Users\per\Documents\Arduino\libraries\Arduino_BuiltIn

Used platform Version Path
arduino:avr   1.8.6   C:\Users\per\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.8.6

🐛 A confusing error message appeared in the output even though the compilation was successful and completely normal:

C:\Users\per\AppData\Local\Temp\FooSketch\FooSketch.ino:1:10: fatal error: Arduino_BuiltIn.h: No such file or directory
 #include <Arduino_BuiltIn.h>
          ^~~~~~~~~~~~~~~~~~~
compilation terminated.

Expected behavior

The expected error messages generated by the recipe.preproc.macros command invocations during library discovery are not printed in the compile command output.

Unexpected error messages generated by the recipe.preproc.macros command should still be printed to the command output in order to facilitate troubleshooting.

Arduino CLI version

e9c7d86

Operating system

Windows

Operating system version

Windows 11

Additional context

I bisected the regression to 0585435 (does not occur when using the build from the previous commit 9ce2904).

0585435 is described as a refactoring that is not expected to change behavior (#2194), which leads me to believe this was unintentional.

Issue checklist

• I searched for previous reports in the issue tracker
• I verified the problem still occurs when using the nightly build
• My report contains all necessary details