Compiling for different board variants in different build-dirs does not reuse already compiled objects

[n-peugnet]

Describe the problem

I am trying to build the same sketch for different variants of the nano33ble board. For the sense Rev2 board I added this flag --build-property 'compiler.cpp.extra_flags=-DSENSE_REV2' to include some additional parts of the code. When alternating between two builds (with and without this extra flag) the compilation is very long as all the libraries are recompiled each time. So I tried to set a build dir with --build-cache-path and --build-path options for each configuration, so that the already compiled files can be reused. But it continues to recompile everything each time I switch from one command to another. Am I doing something wrong?

To reproduce

Here are the two compile commands I am using:

arduino-cli compile -b arduino:mbed_nano:nano33ble --build-cache-path build-nano33ble --build-path build-nano33ble
arduino-cli compile -b arduino:mbed_nano:nano33ble --build-cache-path build-nano33blesense2 --build-path build-nano33blesense2 --build-property 'compiler.cpp.extra_flags=-DSENSE_REV2'

And here is a console log:
arduino-cli-compile.log

Expected behavior

When using different --build-path and --build-cache-path, multiple variant of the same sketch for the same board should be able to be built without recompiling everything when switching from one to another.

Arduino CLI version

arduino-cli Version: nightly-20230817 Commit: 459fe76 Date: 2023-08-17T01:23:47Z

Operating system

Linux

Operating system version

Linux SMP PREEMPT_DYNAMIC Debian 6.4.4-2 (2023-07-30) x86_64

Additional context

No response

Issue checklist

• I searched for previous reports in the issue tracker
• I verified the problem still occurs when using the nightly build
• My report contains all necessary details

[cmaglie]

Hi @n-peugnet, thanks for the bug report, I've examined the situation and I can confirm the bug.

It seems that the problem is that arduino-cli is not able to handle multiple build dirs inside the sketch folder because the cli recognize a previous stray build dir as part of the sketch.
One of the first steps of the compilation is to prepare the sketch build dir by copying the sketch source code in the build path. Since a previous build-dir may be in the sketch, the CLI will copy the content of that dir in the build path:

~/Arduino/Blink$ tree
.
├── Blink.ino
├── build-nano33ble
│    [...]
└── build-nano33blesense2
    ├── Blink.ino.bin
    ├── Blink.ino.elf
    ├── Blink.ino.hex
    ├── Blink.ino.map
    ├── Blink.ino.with_bootloader.bin
    ├── Blink.ino.with_bootloader.hex
    ├── build.options.json
    ├── compile_commands.json
    ├── core
    │   [...]
    ├── includes.cache
    ├── libraries
    ├── libraries.cache
    ├── linker_script.ld
    └── sketch
        ├── Blink.ino.cpp
        ├── Blink.ino.cpp.d
        ├── Blink.ino.cpp.o
        └── build-nano33ble
            ├── build.options.json
            ├── compile_commands.json
            └── sketch
                ├── Blink.ino.cpp
                └── build-nano33blesense2
                    ├── build.options.json
                    ├── compile_commands.json
                    └── sketch
                        ├── Blink.ino.cpp
                        └── build-nano33ble
                            ├── build.options.json
                            ├── compile_commands.json
                            └── sketch
                                ├── Blink.ino.cpp
                                └── build-nano33blesense2
                                    ├── build.options.json
                                    ├── compile_commands.json
                                    └── sketch
                                        ├── Blink.ino.cpp
                                        └── build-nano33ble
                                            ├── build.options.json
                                            ├── compile_commands.json
                                            └── sketch
                                                ├── Blink.ino.cpp
                                                └── build-nano33blesense2
                                                    ├── build.options.json
                                                    ├── compile_commands.json
                                                    └── sketch
                                                        ├── Blink.ino.cpp
                                                        └── build-nano33ble
                                                            ├── build.options.json
                                                            ├── compile_commands.json
                                                            └── sketch
                                                                └── Blink.ino.cpp

As you can see, at each compile round, a new level is added to the build path (that in turn will be copied on the next compile recursively). This has two side effects:

1. the files in the build dir take extra disk space (even if the consumed disk space is negligible, this should be fixed)
2. adding a new file in the sketch triggers a full rebuild.

I'm going to prepare a patch to fix these problems.

Luckily, this bug does not affect the compilation result, it is still correct because the deeper Bilnk.ino.cpps are not compiled, it only takes more space/time to compile.

[n-peugnet]

It seems that the problem is that arduino-cli is not able to handle multiple build dirs inside the sketch folder because the cli recognize a previous stray build dir as part of the sketch.
One of the first steps of the compilation is to prepare the sketch build dir by copying the sketch source code in the build path. Since a previous build-dir may be in the sketch, the CLI will copy the content of that dir in the build path:

Great find ! I didn't see it but it's there:

└── sketch
    ├── arduino-ptt.ino.cpp
    ├── arduino-ptt.ino.cpp.d
    ├── arduino-ptt.ino.cpp.o
    ├── build-nano33ble
    │   ├── build.options.json
    │   ├── compile_commands.json
    │   └── sketch
    │       ├── arduino-ptt.ino.cpp
    │       ├── build-nano33blesense2
    │       │   ├── build.options.json
    │       │   ├── compile_commands.json
    │       │   └── sketch
    │       │       ├── arduino-ptt.ino.cpp
    │       │       ├── build-nano33ble
    │       │       │   ├── build.options.json
    │       │       │   ├── compile_commands.json
    │       │       │   └── sketch
    │       │       │       ├── arduino-ptt.ino.cpp
    │       │       │       └── README.md
    │       │       └── README.md
    │       └── README.md
    └── README.md

And I can confirm it works as expected on latest master. Thank you!