-
-
Notifications
You must be signed in to change notification settings - Fork 7k
Compilation fails due to overly long command generated by recipe.c.combine.pattern #6751
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Summary of the issue: The cause of the compilation failure, as found by @bobc in the mentioned issue report, is that the large number of filenames in The Marlin sketch folder structure was recently reorganized to put most of the source files in subfolders and this caused the command length to significantly increase. I believe this version is found in the bugfix-2.0.x-new-layout branch. When I compile the tip of that branch using the hourly build of the Arduino IDE, Windows 7, for Arduino/Genuino Mega I still only end up with a |
I believe that, on Windows, the total length of the command (not the number of arguments, like on Linux) is relevant. This means that the length of the path to the sketchbook might be relevant, but more importantly the length of the path to the build / temp folder. I think this might normally be in the "user" directory, so that would suggest the username length is relevant too. As a workaround, you could probably set |
Setting |
But should the user be required to manually set a temp directory in order to build a sketch ? Am I the only one who sees this as a flaw ? |
Nope, it either just uses that path if it is set, or generates a new path for each sketch when it is empty.
It's certainly not a proper fix, but it is useful to mention the workaround, to at least get people unstuck. The actual problem here is the limited length of a commandline, combined with a sketch (and library) combined with a lot of files. I can't see a perfect solution for this problem, unfortunately. I do have some ideas. You could link together groups of .o files, and then link the resulting groups together into the final binary. A common approach for this would be to use archives (.a files), which contain multiple .o files. This already happens for the core, and libraries that have enabled this in their libraries.properties file. It would be obvious to create an archive for the core, sketch and each library separately. In practice this would probably work, but in theory, this could still break when a single library or sketch contains enough files to trip the limit. To really solve this, the final linker command should be automatically split, based on the actual commandline length. Another downside of using .a files is that it slightly changes the linking strategy: .o files specified on the command file will be included in the link, while .o files inside .a files are only included when they are needed. Due to subsequent optimization steps, the difference between these is minimal, but it might break some corner cases. An alternative would be to merge .o files into a new .o file. Apparently the -r option can be used for this, though I have not tried this. One caveat is that splitting the linker command will very likely need additions to the recipes in One option that would be perfect is if you could specify the list of files to compile through stdin or through a tempory file instead of on the commandline. Looking at the gcc commandline, it supports specifying One point of attention is that the contents of the Summarizing: There are solutions, but nothing that is so simple it could be very quickly implemented. The |
Exactly, it uses the same path for all sketches you might compile IF it is set. There is no per-sketch setting. Using @file seems the simplest and most scalable fix. Clearly this needs to be an option, if there are targets that don't use gcc. If I knew Go, I would propose a patch. I suspect though, anyone who encounters this issue has outgrown the Arduino IDE, which is only really usable for small sketches, and should be using a more professional tool. |
There could be a third option:
This should shorten the commandline A LOT, without any drawback (if the split doesn't recognize a filename, using the absolute path makes no harm) A test commit is here (https://github.com/arduino/arduino-builder/compare/master...facchinm:relative_paths?expand=1) I'm also attaching a Windows build of the builder, if anyone at Marlin still has a copy of the problematic branch it would be nice to test it live 😄 |
@facchinm - dropping in your arduino-builder debug build with your Windows command-line length work-workaround and previously-failing build now succeeds 👍 However, it's unclear how because the output seems to still include the full paths (avr-gcc link command truncated at 32k in the build window): linker section of build log window
|
Hi @fiveangle , Also, I don't know if it's a good idea to print the mangled commandline since you need to |
@facchinm, sounds like a practical approach, though it won't really solve the problem (just raise the bar for making the problem to occur). What command to show is indeed an issue: I would argue that showing a different command from what's actually being run is problematic when debugging some problems. The instant-copy-pasteability of full paths is an advantage, though, so I'm not sure what side to pick (probably show the actual command being run, especially if this happens only for long commandlines). One caveat is that there might be targets that do not like relative paths (on some of its commandline options perhaps). I don't have any examples ready, though. OTOH, if this fix is applied only for too-long commands, then these targets will only break when the build would be broken anyway. |
Trying to download the arduino-builder.zip fails via antivirus https://www.microsoft.com/en-us/wdsi/threats/malware-encyclopedia-description?Name=Trojan:Win32/Fuerboos.A!cl |
I got the same so downloaded on my Mac and scanned with clamd and determined it was just a heuristics based false positive. Regardless, after applying the debug it resolved the path length issues but then later broke building for Uno target (I suspect boards support in debug files isn’t complete). Looking forward to a long term solution. Curious, does Arduino 1.9.0 beta include the proposed Windows command length work-around ? |
Hi @fiveangle , 1.9.0 beta contains the fix with 30000 characters limit (arduino/arduino-builder@f2cd784) AND the |
For anyone curious, I also had sucess fixing this problem by upgrading to 1.9.0 beta Arduino IDE. |
Fixed in 1.8.9. |
Thanks a lot, this suggestion allowed me to finally compile latest Marlin firmware on Windows with Arduino IDE! For those stuck as well with Marlin, edit the file
It does not work: |
i im having a issue with arduino paths being to long and dont let me compile Marlin-bugfix-2.0.x-new-layout firmware there was new arrangments on marlin and seems that now there is a problem in windows and arduino paths being to long
this issue seems to afect alot people
the issue discution is on this link MarlinFirmware/Marlin#7707 (comment)
is there any way to fix this problem?
The text was updated successfully, but these errors were encountered: