Xtensa absolute filename too long

[Elektrik1]

Hi there,

First compile on Windows 10 Arduino IDE 1.8.6 works fine, but on recompile such error appears:
Compiling core...

cmd.exe /c mkdir "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\core" & (echo #define ARDUINO_ESP8266_GIT_VER 0x00000000 & echo #define ARDUINO_ESP8266_GIT_DESC win-2.5.0-dev ) > "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\core\core_version.h"
A subdirectory or file C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\core already exists.

Using precompiled core: C:\Users\Elektrik\AppData\Local\Temp\arduino_cache_896884\core\core_arduino-esp8266_esp8266_nodemcuv2_CpuFrequency_80,VTable_flash,FlashSize_4M1M,LwIPVariant_v2mss536,Debug_Disabled,DebugLevel_None____,FlashErase_none,UploadSpeed_921600_4e056916d9045c6bc547a3a7ce5e20ae.a
Linking everything together...
"C:\Dropbox\Arduino\Sketches\hardware\arduino-esp8266\esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc" -CC -E -P -DVTABLES_IN_FLASH "C:\Dropbox\Arduino\Sketches\hardware\arduino-esp8266\esp8266/tools/sdk/ld/eagle.app.v6.common.ld.h" -o "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145/eagle.app.v6.common.ld"
"C:\Dropbox\Arduino\Sketches\hardware\arduino-esp8266\esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-gcc" -Wl,-Map "-Wl,C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145/HTTP-server.ino.map" -g -w -Os -nostdlib -Wl,--no-check-sections -u app_entry -u printf_float -u scanf_float -Wl,-static "-LC:\Dropbox\Arduino\Sketches\hardware\arduino-esp8266\esp8266/tools/sdk/lib" "-LC:\Dropbox\Arduino\Sketches\hardware\arduino-esp8266\esp8266/tools/sdk/ld" "-LC:\Dropbox\Arduino\Sketches\hardware\arduino-esp8266\esp8266/tools/sdk/libc/xtensa-lx106-elf/lib" -Teagle.flash.4m1m.ld -Wl,--gc-sections -Wl,-wrap,system_restart_local -Wl,-wrap,spi_flash_read -o "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145/HTTP-server.ino.elf" -Wl,--start-group "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\sketch\HTTP-server.ino.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESP8266WiFi\ESP8266WiFi.a" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\dscKeybusInterface\dscKeybusInterface.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\dscKeybusInterface\dscKeybusPrintData.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\dscKeybusInterface\dscKeybusProcessData.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESP8266mDNS\ESP8266mDNS.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ArduinoOTA\ArduinoOTA.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESPAsyncTCP\AsyncPrinter.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESPAsyncTCP\ESPAsyncTCP.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESPAsyncTCP\ESPAsyncTCPbuffer.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESPAsyncTCP\SyncClient.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESPAsyncTCP\tcp_axtls.c.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESPAsyncWebServer\AsyncEventSource.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESPAsyncWebServer\AsyncWebSocket.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESPAsyncWebServer\SPIFFSEditor.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESPAsyncWebServer\WebAuthentication.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESPAsyncWebServer\WebHandlers.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESPAsyncWebServer\WebRequest.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESPAsyncWebServer\WebResponses.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\ESPAsyncWebServer\WebServer.cpp.o" "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145\libraries\Hash\Hash.a" "C:\Users\Elektrik\AppData\Local\Temp\arduino_cache_896884\core\core_arduino-esp8266_esp8266_nodemcuv2_CpuFrequency_80,VTable_flash,FlashSize_4M1M,LwIPVariant_v2mss536,Debug_Disabled,DebugLevel_None__,FlashErase_none,UploadSpeed_921600_4e056916d9045c6bc547a3a7ce5e20ae.a" -lhal -lphy -lpp -lnet80211 -llwip2 -lwpa -lcrypto -lmain -lwps -lbearssl -laxtls -lespnow -lsmartconfig -lairkiss -lwpa2 -lstdc++ -lm -lc -lgcc -Wl,--end-group "-LC:\Users\Elektrik\AppData\Local\Temp\arduino_build_391145"
xtensa-lx106-elf-gcc: error: C:\Users\Elektrik\AppData\Local\Temp\arduino_cache_896884\core\core_arduino-esp8266_esp8266_nodemcuv2_CpuFrequency_80,VTable_flash,FlashSize_4M1M,LwIPVariant_v2mss536,Debug_Disabled,DebugLevel_None____,FlashErase_none,UploadSpeed_921600_4e056916d9045c6bc547a3a7ce5e20ae.a: No such file or directory

File is present but it can't be opened. I think it's because of long file name (>255 chars), since I cannot open this path with any text editor:

C:\Users\Elektrik\AppData\Local\Temp\arduino_cache_896884\core\core_arduino-esp8266_esp8266_nodemcuv2_CpuFrequency_80,VTable_flash,FlashSize_4M1M,LwIPVariant_v2mss536,Debug_Disabled,DebugLevel_None____,FlashErase_none,UploadSpeed_921600_4e056916d9045c6bc547a3a7ce5e20ae.a

Could file name be shortened ?

[d-a-v]

The arduino IDE is building that name, based on your temp directory probably itself based on the windows variable TEMP.
Can you try and change it ?

[Elektrik1]

File name is not associated with TEMP variable, and consists 208 chars in this case:

core_arduino-esp8266_esp8266_nodemcuv2_CpuFrequency_80,VTable_flash,FlashSize_4M1M,LwIPVariant_v2mss536,Debug_Disabled,DebugLevel_None____,FlashErase_none,UploadSpeed_921600_4e056916d9045c6bc547a3a7ce5e20ae.a

Why can't it be shorter like "esp8266_4e056916d9045c6bc547a3a7ce5e20ae.a" ?

[d-a-v]

File name is not associated with TEMP variable

it seems to.

Why can't it be shorter like "esp8266_4e056916d9045c6bc547a3a7ce5e20ae.a" ?

We don't control the arduino builder. After googling, it is advised to both change %TEMP% and give arduino-1.9beta a try. Can you try changing %TEMP% to something short ?

[devyte]

Is the issue the filename length or the length of path+filename?

[devyte]

On windows the max of path+filename is 260 (by default, the max can be disabled in the registry), so @d-a-v is right: change temp to point elsewhere.

[5chufti]

C:\Users\Elektrik\AppData\Local\Temp\

is the windows standard setting for %temp% and it is dangerous to recommend changing it as it may have ill sideeffects.

It is more likely to be cureable by using less "extra settings" with shorter names and values as long as Arduino IDE doesn't change the way of naming files (put extra settings in filename),

[devyte]

In that case I suggest editing the registry to allow longer path+filename length, and test whether the Arduino IDE works with that.

[5chufti]

I am not sure how many applications will be able to handle filenames longer than the traditional maximum. I wouldn't be surprised if windows itself has problems because this "novelty" is not respected in every aspect.

[Elektrik1]

PATH edit is not a great solution, since the filename is the issue.

Yes, windows can allocate 260 max chars absolute path, it's normally more than enough if reasonable filenames are selected. Filename itself is limited to 260 too, in my case it does not reach this limit yet, but if more options will be available it can exceed and this limit.

Non elegant, more like hack setting Enable win32 long paths in gpeditor or via registry does not fix this issue, perhaps xtensa compiler itself cannot handle such long paths/filenames.

Temporarily manually deleting .a file between compilation helps, but full recompile is done every time.

[d-a-v]

I have an unrelated coming update of the board generator. I'll try to reduce the internal option names in it since windows use them as part of the library name (which is odd?).

[5chufti]

seeing this, it seems clear why ide rebuilds everything if only so little as the upload speed is changed.
Interestingly the change of the port does not show in the name and thus not initiate a "rebuild all".
Maybe there is some "magic" how / which option(names) are not considered "essential"?
Then options like port, upload speed, flash_erase, reset_method could be omitted...
Others like LwIPVariant, CrystalFreq or CpuFrequency could easily be shortened to just LwIP,Xtal and CpuClk etc. This would definitely help w/o loosing essential information or risking the stability of the (windows)system or useability for NOOBS.
On some other issue I recently got reminded (by igrrr) that Arduino is for noobs (so everything more technical than the power-switch should be hidden), so do you really suggest now they should tinker with the windows registry?

[d-a-v]

It is unclear to me why the arduino builder includes these option names with the windows version, and not with the linux version. The same file under linux is like .../xtensa-lx106-elf-gcc [...] /tmp/arduino_build_129936/core/core.a -lhal -lphy [...]. Uniqueness is given by the subdirectory in linux's %TEMP% equivalent (which is /tmp).

[5chufti]

I don't think it is about uniqueness - the random part at the end should suffice, rather about detecting changed build options? Why this wouldn't be necessary under unix - I don't know. Or maybe it is just an error in the Windows version?
maybe an occasional visitor (@PaulStoffregen) has more insight into the Arduino buildsystem?

edit: just checked, it does the same with arduino board selected, so no real esp-core issue, but esp core is prone to file name length overflow because of the plethora of buildoptions.

[d-a-v]

@5chufti does arduino-1.9 on windows have the same behavior ?

[Elektrik1]

Yes, 1.9 beta has exact behavior.
Seems to be arduino issue indeed, turining off Aggressively cache compiled core in preferences fixes arduino_cache_xxx folder creation and caching of .a file.

[d-a-v]

As a workaround, I am trying to find nice short names for hidden strings in boards.txt, updating #5103.
Not an easy task

[d-a-v]

@Elektrik1 Can you try it ?

[5chufti]

ok, disabling the Aggressively cache compiled core removes the options from the filename and should alleviate Elektrik1's problem.
As this setting does not prevent the IDE from rebuilding everything when as little as the upload baudrate is changed [but this is an other (IDE) issue] so one can leave it disabled.

Edit: as the "port" setting does not incur a rebuild, maybe we can get more "exceptions"?

[PaulStoffregen]

maybe an occasional visitor (@PaulStoffregen) has more insight into the Arduino buildsystem?

Not really. Truth is I've not followed the build system stuff closely since they moved it out of the Java codebase and made the Google Go language arduino-builder program.

But I do believe these issues on Arduino's repositories are related:

arduino/Arduino#7902

arduino/arduino-builder#284

I might also suggest a short-term workaround to at least ease user suffering. Seems like many of the internal strings ESP8266 uses (in boards.txt) to define the menus are unnecessarily long. Simply shortening those names could bring a lot of short-term relief while waiting for the Arduino developers to improve the builder.

EDIT: oh, I see that's exactly what #5100 does. Please feel free to ignore everything I've said! ;-)

[d-a-v]

@PaulStoffregen Thanks for confirming that there is currently no other way than this short-term workaround.
(It is quite annoying since we hardly recognize anything in our boards.txt :)

[d-a-v]

@Elektrik1 Please use latest git and report back whether the fix is enough for this issue

[yoursunny]

I've been affected by this issue for two weeks. I have Arduino 1.8.6 installed from Windows Store and "Aggressively cache compiled core" is turned on. It seems that cc0bfa0 fixes this.

[Elektrik1]

I do confirm latest commit fixes it, log:

Compiling core...

cmd.exe /c mkdir "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_505278\core" & (echo #define ARDUINO_ESP8266_GIT_VER 0x00000000 & echo #define ARDUINO_ESP8266_GIT_DESC win-2.5.0-dev ) > "C:\Users\Elektrik\AppData\Local\Temp\arduino_build_505278\core\core_version.h"
A subdirectory or file C:\Users\Elektrik\AppData\Local\Temp\arduino_build_505278\core already exists.

Using precompiled core: C:\Users\Elektrik\AppData\Local\Temp\arduino_cache_703280\core\core_arduino-esp8266_esp8266_nodemcuv2_xtal_80,vt_flash,eesz_4M1M,ip_lm2,dbg_Disabled,lvl_None____,wipe_none,baud_921600_4e056916d9045c6bc547a3a7ce5e20ae.a
Linking everything together...

Thank you!

[d-a-v]

@Elektrik1 thanks for reporting !

[jjsuwa]

Now, 1.8.7 for Windows (or later) makes cached core .a filename MD5-hashed. Solved completely.