esptool path conflict when both ESP8266 & ESP32 cores installed

[ImpulseAdventure]

Great work on releasing this interface -- it is incredibly helpful for regressions.

I have run into what appears to be a transient conflict in esptool definition for arduino-cli when both ESP32 and ESP8266 cores have been installed. When either core has been installed, no issue is observed. When both cores have been installed, the arduino-cli compile process fails (about a half of the time) as the path to esptool is corrupted.

Please note that it wasn't entirely clear to me whether the source of the issue was within arduino-cli or due to something more specific to the espressif cores, though it seems to me that the arduino-cli is the likely source of the path in question below.

Steps to reproduce (approximate)

• Installed arduino-cli (0.3.1-alpha.preview) on Xubuntu 16.04.3 VM from https://downloads.arduino.cc/arduino-cli/arduino-cli-0.3.1-alpha.preview-linux64.tar.bz2 (renamed to arduino-cli from this point onwards)
• Create .cli-config.yml with

board_manager:
  additional_urls:
    - http://arduino.esp8266.com/stable/package_esp8266com_index.json
    - https://dl.espressif.com/dl/package_esp32_index.json

• rm -rf ~/.arduino15/staging/packages/* (to ensure clean package download)
• ./arduino-cli core update-index
• ./arduino-cli core install esp8266:esp8266
• ./arduino-cli core install esp32:esp32
• Create a dummy sketch, eg: basic/basic.ino:

void setup() {
  pinMode(10,OUTPUT);
}
void loop() {
  digitalWrite(10,HIGH);
  delay(1000);
  digitalWrite(10,LOW);
  delay(1000);
}

• Compile the same same sketch multiple times in a row (either from shell script or command prompt)

./arduino-cli compile --fqbn esp32:esp32:esp32 "basic" --verbose | tee log1
./arduino-cli compile --fqbn esp32:esp32:esp32 "basic" --verbose | tee log2
./arduino-cli compile --fqbn esp32:esp32:esp32 "basic" --verbose | tee log3

Results

Out of ten compiles, I often encounter about five that die because esptool can't be found. Comparing the log files between passing & failing, the content is largely identical except for some variation in the compile file ordering and the path to esptool.

In the passing cases (for ESP32), I see:

Linking everything together...
/home/cal/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/xtensa-esp32-elf-gcc ...
python /home/cal/.arduino15/packages/esp32/hardware/esp32/1.0.0/tools/gen_esp32part.py ....
python /home/cal/.arduino15/packages/esp32/tools/esptool/2.3.1/esptool.py --chip esp32 elf2image ...
...

In the failing cases (for ESP32), I see:

Linking everything together...
/home/cal/.arduino15/packages/esp32/tools/xtensa-esp32-elf-gcc/1.22.0-80-g6c4433a-5.2.0/bin/xtensa-esp32-elf-gcc ...
python /home/cal/.arduino15/packages/esp32/hardware/esp32/1.0.0/tools/gen_esp32part.py ....
python /home/cal/.arduino15/packages/esp8266/tools/esptool/0.4.13/esptool.py --chip esp32 elf2image 
python: can't open file '/home/cal/.arduino15/packages/esp8266/tools/esptool/0.4.13/esptool.py': [Errno 2] No such file or directory
Error: exit status 2
Compilation failed.
...

Other observations:

• Without really looking too deeply in the code, I assume the above issue is due to {tools.esptool.path} getting corrupted (perhaps originating from esptool's runtime InstallDir) but I'm not sure.
• The same problem will occur during compiles using --fqbn esp8266:esp8266:nodemcu, as the ESP32 esptool path gets selected instead. The resulting error (for ESP8266 cases) then becomes:

/home/cal/.arduino15/packages/esp8266/hardware/esp8266/2.4.2/tools/sdk/libc/xtensa-lx106-elf/lib -Teagle.flash.4m1m.ld -Wl...
/home/cal/.arduino15/packages/esp32/tools/esptool/2.3.1/esptool -eo ...
Error: fork/exec /home/cal/.arduino15/packages/esp32/tools/esptool/2.3.1/esptool: no such file or directory

• When I uninstall the esp8266 or esp32 core (ie. only have a single arduino core installed), the issue no longer occurs.
• I have not encountered this problem from within the Arduino IDE (wherein both cores are installed) -- only from arduino-cli.
• During the compile log I also observe the warning earlier Unable to cache built core, please tell esp32:[email protected] maintainers to follow ...

Thanks for any assistance

[cmaglie]

@ImpulseAdventure thanks for bug report and for the detailed instructions to reproduce the issue, that helped a lot to spot the bug! The fix in #82 if you want to try it out.

[ImpulseAdventure]

Fantastic... thanks for figuring this one out, @cmaglie !
The multi-core build regressions are working well for me now using the binary attached to #82.