failed to compile Arduino_core_STM32 V1.8.0 based projects. V1.7.0 works fine

[engrziarehman]

I have 4.3.1 sloeber installed, all other repositories are compiling fine, except Arduino_Core_STM32, its 1.8.0 version installed.
I have created empty arduino project and selected Bluepill as controller, then after compilation it is giving error in the end while 'starting combiner' appears.

D:/sloeber431/sloeber/arduinoPlugin/tools/make/make --no-print-directory main-build
'Starting combiner'
"D:\sloeber431\sloeber\arduinoPlugin\packages\STM32\tools\xpack-arm-none-eabi-gcc\9.2.1-1.1/bin/arm-none-eabi-gcc" -mcpu=cortex-m3 -mthumb -Os --specs=nano.specs -Wl,--defsym=LD_FLASH_OFFSET=0x2000 -Wl,--defsym=LD_MAX_SIZE=65536 -Wl,--defsym=LD_MAX_DATA_SIZE=20480 -Wl,--cref -Wl,--check-sections -Wl,--gc-sections -Wl,--entry=Reset_Handler -Wl,--unresolved-symbols=report-all -Wl,--warn-common "-TD:\sloeber431\sloeber\arduinoPlugin\packages\STM32\hardware\stm32\1.8.0\variants\PILL_F103XX/ldscript.ld" "-Wl,-Map,Z:/Arduino/EclipseWorkspace/ArduinoCoreSTM32_test/Release/ArduinoCoreSTM32_test.map" "-LD:\sloeber431\sloeber\arduinoPlugin\packages\STM32\tools\CMSIS\5.5.1/CMSIS/DSP/Lib/GCC/" -larm_cortexM3l_math -o "Z:/Arduino/EclipseWorkspace/ArduinoCoreSTM32_test/Release/ArduinoCoreSTM32_test.elf" "-LZ:/Arduino/EclipseWorkspace/ArduinoCoreSTM32_test/Release" -Wl,--start-group .\ArduinoCoreSTM32_test.cpp.o .\core\variant\PeripheralPins.c.o .\core\variant\variant.cpp.o Z:/Arduino/EclipseWorkspace/ArduinoCoreSTM32_test/Release/arduino.ar -lc -Wl,--end-group -lm -lgcc -lstdc++
d:/sloeber431/sloeber/arduinoplugin/packages/stm32/tools/xpack-arm-none-eabi-gcc/9.2.1-1.1/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe: Z:/Arduino/EclipseWorkspace/ArduinoCoreSTM32_test/Release/arduino.ar(startup_stm32yyxx.S.o): in function LoopFillZerobss': makefile:90: recipe for target 'ArduinoCoreSTM32_test.elf' failed (.text.Reset_Handler+0x26): undefined reference to SystemInit'
makefile:69: recipe for target 'all' failed
d:/sloeber431/sloeber/arduinoplugin/packages/stm32/tools/xpack-arm-none-eabi-gcc/9.2.1-1.1/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe: Z:/Arduino/EclipseWorkspace/ArduinoCoreSTM32_test/Release/arduino.ar(main.cpp.o): in function premain()': main.cpp:(.text.startup._Z7premainv+0x4): undefined reference to HAL_NVIC_SetPriorityGrouping'
d:/sloeber431/sloeber/arduinoplugin/packages/stm32/tools/xpack-arm-none-eabi-gcc/9.2.1-1.1/bin/../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/bin/ld.exe: Z:/Arduino/EclipseWorkspace/ArduinoCoreSTM32_test/Release/arduino.ar(board.c.o): in function init': board.c:(.text.init+0x0): undefined reference to hw_config_init'
collect2.exe: error: ld returned 1 exit status
make[1]: *** [ArduinoCoreSTM32_test.elf] Error 1
make: *** [all] Error 2

[jantje]

I don't see a 1.8 version

[ethomas997]

I found that builds with stm32/1.8.0 will fail, whereas changing to stm32/1.7.0 will succeed. This is with building a simple 'Blink' sketch for a BluePill board.

[jantje]

Which json file do you use? I do not have 1.8

[ethomas997]

The current json from stm32duino has 1.8.0:
https://raw.githubusercontent.com/stm32duino/BoardManagerFiles/master/STM32/package_stm_index.json

[jantje]

indeed doesn't seem to work.
No clue why. Someone knowing the stm32 should take a look and see what is wrong.

[ethomas997]

It looks like the ArduinoIDE using stm32 is on v1.8.0 and is able successfully do builds. Seems like the problem is specific to Sloeber-Eclipse.

[fpistm]

Hi,
Issue has been reported on stm32duino forum:
https://www.stm32duino.com/viewtopic.php?f=18&t=173&p=1268#p1268

It seems Sloeber doesn't handle properly the prebuild recipe from the platform.txt.
https://github.com/stm32duino/Arduino_Core_STM32/blob/f1d475a7ce9a73464bcf41b57d22803af8bc371a/platform.txt#L117-L120

[jantje]

@fpistm
Thanks for the investigation. I see 2 problems with the hooks right now.
The first command fails due to a / \ problem. Strangely enough the command works fine from the command line. So workaround : run the command from the command line. This is only needed once (and after removal of the release folder.
The second is more confusing to me.
The platform.txt contains
recipe.hooks.prebuild.3.pattern.windows=cmd /c echo #include ^<SrcWrapper.h^> > "{build.src_wrapper.path}"
As we are on windows Sloeber should create the key value environment variable
A.RECIPE.HOOKS.PREBUILD.3.PATTERN=cmd /c echo #include ^<SrcWrapper.h^> > "{build.src_wrapper.path}"
But on my system it is
A.RECIPE.HOOKS.PREBUILD.3.PATTERN=cmd /c echo
If found that this is caused because sloeber considers everything after # as a comment where only lines starting with # should be seen as comment lines https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5-3rd-party-Hardware-specification

This should be 2 relatively easy fixes.

[jantje]

the # comment problem only existed in non-boards related files like platform.txt.
The other fix is pretty low-level so I'm running the CreateAndCompileDefaultInoOnAllBoardsTest before releasing.

[fpistm]

welcome @jantje
I know how it is hard to maintain such project so when we can help 😉

[jantje]

As I expected the test failed

[jantje]

I'm hitting a CDT issue that I see as very unlikely to be fixed.
https://bugs.eclipse.org/bugs/show_bug.cgi?id=560330
I fear I'll have to come up with another way to fix this.

[jantje]

Now the 2 issues in Sloeber are fixed there seems to be a remaining issue.
The platform.txt contains
recipe.hooks.prebuild.3.pattern.windows=cmd /c echo #include ^<SrcWrapper.h^> > "{build.src_wrapper.path}"
I guess the ^is some kind of escaping for the < and > which are redirecting thingies. I really don't understand what they are trying to do nor why they are doing it the way they do (I think they are kind of stoned) but changing the line to below makes it work in the latest nightly in sloeber
recipe.hooks.prebuild.3.pattern.windows=cmd /c echo #include "SrcWrapper.h" > "{build.src_wrapper.path}"

[fpistm]

@jantje

I guess the ^is some kind of escaping for the < and > which are redirecting thingies. I really don't understand what they are trying to do nor why they are doing it the way they do (I think they are kind of stoned) but changing the line to below makes it work in the latest nightly in sloeber
recipe.hooks.prebuild.3.pattern.windows=cmd /c echo #include "SrcWrapper.h" > "{build.src_wrapper.path}"

Unfortunately, this is not possible else this not produce the correct output with Arduino IDE:

sketch\SrcWrapper.cpp:1:10: error: #include expects "FILENAME" or <FILENAME>
    1 | #include SrcWrapper.h
      |          ^~~~~~~~~~
exit status 1

I've tested a lot of combination, the only one working is using the '^' escape character.

[jantje]

The problem is that -on windows- arduino uses java to run windows commands via cygwin (a linux emulator) where Sloeber uses java to run a windows command that starts a make (there are several makes each with their pro's and con's) which currently starts a windows sh (again their are several sh's each with their pro's and con's).
This means the more unusual a command; the more it is likely to behave differently. I have never before seen ^ as a escape character before, so to me this command is a extremely unusual command.
Googling didn't tell me much about using ^as a escape character.
I wouldn't be surprised that this works because there is a bug somewhere or it is a arduino ide behaviour.
Anyway: have you tried?
recipe.hooks.prebuild.3.pattern.windows=cmd /c echo "#include <SrcWrapper.h>" > "{build.src_wrapper.path}"

What I'm most curiously about is:
Why do you need to create 2 static files (one empty header 😲 ) during the build process? Why can't they just be part of the core or variant?

[fpistm]

Googling didn't tell me much about using ^as a escape character.

https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/echo#remarks

To display a pipe (|) or redirection character (< or >) when you are using echo, use a caret (^) immediately before the pipe or redirection character (for example, ^|, ^>, or ^<). To display a caret, type two carets in succession (^^).

Anyway: have you tried?
recipe.hooks.prebuild.3.pattern.windows=cmd /c echo "#include <SrcWrapper.h>" > "{build.src_wrapper.path}"

As said I've tried several syntax, with this one:

sketch\SrcWrapper.cpp:1:1: error: expected unqualified-id before string constant
    1 | "#include <SrcWrapper.h>"
      | ^~~~~~~~~~~~~~~~~~~~~~~~~
exit status 1

Why do you need to create 2 static files (one empty header 😲 ) during the build process? Why can't they just be part of the core or variant?

• Build_opt.h is not an header file. It is a command-line options for gcc which allows to customize some core definitions or compiler options. See:
  https://github.com/stm32duino/wiki/wiki/Customize-build-options-using-build_opt.h
  As Arduino does not allows other extensions than *.h, *.hpp, *.c, *.cpp and *.ino (maybe other but don't know them), I use the .h.
  So, an user can add his own command-line options at sketch level. If not, it is necessary to provide on empty.
• About including the SrcWrapper.h, it is required to include the built-in library SrcWrapper.
  Reason for this:
  Remove use of whole-archive stm32duino/Arduino_Core_STM32#677
  It could not be included in the core else the library path is not populated to all builds item... Arduino Library management limitation. If you know an other way then I would be happy to try 😉

[jantje]

https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/echo#remarks

Seems like a windows specific thing. Strange as this goes against my understanding arduino IDE uses cygwin and not native windows.

1 | "#include <SrcWrapper.h>"

Again this is native windows behaviour and goes against the idea arduino IDE uses cygwin.
And yes.....cygwin is removed from arduino IDE in 2015 😅 my knowledge of arduio IDE seems pretty outdated.

But then I do not understand why
recipe.hooks.prebuild.3.pattern.windows=cmd /c echo #include "SrcWrapper.h" > "{build.src_wrapper.path}"
doesn't work. Does the arduino IDE removes the "'s? If so Why?

[fpistm]

This is due also to cmd /c usage which implies to protect args of the echo command to execute

[jantje]

I have been investigating these prebuild commands and now I understand the complexities you face and how it works.
However ... these prebuild commands are purely aiming at the arduino IDE build process. It is very unlikely that any other tool will work properly with these prebuild commands.

Problem 1:
The fact the scrWrapper library gets picked up by sloeber is (in my eyes) a bug in CDT https://bugs.eclipse.org/bugs/show_bug.cgi?id=560405

Problem2:
As soon as sloeber processes these prebuild commands properly and you would start using build_opt.h sloeber fails. This because you assume build_opt.h is copied during the build process. Something Sloeber doesn't do.

Alternative idea 1:
Why not creating build_opt.h and src_wrapper.cpp in the sketch folder. Some people need build_opt.h to be there. This doesn't make any assumptions about the build process.

Alternative idea 2:
Why not have 2 source files build_opt.h.org and src_wrapper.cpp.org and copyrename them. This way you can put content in the files explaining what they are about and how (not) to use them.
The pre build commands become lots easier as you only need to copy (don't overwrite) the files.
Linux : cp -n myoldfile.txt mycopiedfile.txt
windows: echo n | copy /-y

[jantje]

This is due also to cmd /c usage which implies to protect args of the echo command to execute

[fpistm]

Well, copy can be an option. Comment in build_opt.h not as this will not be accepted by gcc.
The best way is you try on your side then give me the one which work and then I could test also with Arduino Ide.

[jantje]

This works in windows in sloeber nightly because {build.source.path} was not yet defined

# Pre and post build hooks
extra.path={runtime.platform.path}/extra
build.opt.path={build.source.path}/build_opt.h
build.opt.org.path={extra.path}/build_opt.h

build.src_wrapper.path={build.source.path}/SrcWrapper.cpp
build.src_wrapper.org.path={extra.path}/SrcWrapper.cpp

# Create empty {build.opt} if not exists in the sketch dir
recipe.hooks.prebuild.1.pattern= cp -n "{build.opt.org.path}" "{build.opt.path}" 
recipe.hooks.prebuild.1.pattern.windows=robocopy "{extra.path}" "{build.source.path}" build_opt.h /XC /njh /njs /ndl &

# Force include of SrcWrapper library
recipe.hooks.prebuild.2.pattern= cp -n "{build.src_wrapper.org.path}" "{build.src_wrapper.path}"
recipe.hooks.prebuild.2.pattern.windows=robocopy "{extra.path}" "{build.source.path}" SrcWrapper.cpp /XC /njh /njs /ndl & 

I put the files in a extra folder

[jantje]

Thinking about the define ...
As you are fixing Arduino IDE with the SrcWrapper.cpp creation you should use a arduino define and not a Sloeber/eclipse define.

[fpistm]

Unfortunately, if a library is not included at sketch level this will not work and ScrWrapper is a library.

[jantje]

Unfortunately, if a library is not included at sketch level this will not work and SrcWrapper is a library.

What I'm saying is that your core is dependent on the inclusion of the SrcWrapper library but nowhere mentions it.
Normally you should put it under the core folder but then it is in the archive which you don't want.
So the next option would be to put the library in the variants folder. But there are plenty of variant folders causing code duplication (which is probably the best solution but ....)
So the decision was taken to put SrcWrapper in a library. But the way it was done all source references to SrcWrapper have been removed from core and variant.
This way no tool based on source code processing can detect the SrcWrapper library code is needed.
By simply adding #include "SrcWrapper.h" to Arduino.h this problem is resolved.

If I understood correctly Arduino IDE will not pick this up but Sloeber will (and other advanced IDE's as well) and maybe Arduino IDE will in a later stage.
Also if the workaround for arduino IDE fails, at least you know that something is wrong/missing and it relates to SrcWrapper.

[engrziarehman]

@jantje yes i have followed your workaround, by deleting -mthumb "@{build.opt.path}" from platform.txt, also added blank build_opt.h in sketch root folder, and included "SrcWrapper.h" in .ino file. but the combiner is still giving me error,

arm-none-eabi-gcc: error: .\libraries\SrcWrapper\src\HAL\stm32yyx_hal_spi_ex.c.o: No such file or directory

and i have searched in Release\libraries\SrcWrapper\src\HAL all files have prefix of stm32yyxx, means double yy and double xx, and spi file do exist but with double xx, and combiner is searching with single x, am i thinking right?

[jantje]

I don't understand what is going on at your system.
I'm working on #1145 which makes stm32 is currently not welcome on my system (not building out of the box)
I'll take a look after #1145 work is done.

[jantje]

@fpistm

Unfortunately, if a library is not included at sketch level this will not work and ScrWrapper is a library.

I assumed one could put a subfolder in the variants folder, but that was wrong. Arduino IDE ignores subfolders in the variants folder 😞

[fpistm]

@fpistm

I assumed one could put a subfolder in the variants folder, but that was wrong. Arduino IDE ignores subfolders in the variants folder 😞

Yes, I continue to think about this and try to find a way to answer all constraint. If I found one I will come back to you.

[jantje]

@fpistm
Are you aware other board vendors put the usage of build_opt.h in the menu.
Arduino IDE takes the first option as default. if that is no build_opt.h this should work out of the box both in Sloeber and Arduino IDE.
In this case you can reference the build_opt.h in the sketch folder and it doesn't have to be a .h file.
So no copy needed in prebuild

[fpistm]

No, could you point which one, please?
Anyway, build_opt.h is not the main issue as with 1.7.0 it works

[jantje]

Still searching for the build_opt. I'm running a lot of tests (600+ boards) and I saw it pass by.
Have you tried adding -include SrcWrapper.h to the command line?
This might trick the arduino IDE and maybe it doesn't hurt to much.

[jantje]

I found the boards that set the build ops on/of. It workes with a define which may not be acceptable for you.

[jantje]

The -include SrcWrapper.h doesn't work in sloeber the way I had hoped.
the indexer gets the error but does not translate it to a unresolved include

c:\eclipse\arduinoplugin\packages\stm32\tools\xpack-arm-none-eabi-gcc\9.2.1-1.1\bin\../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/include/c++/9.2.1/backward
 c:\eclipse\arduinoplugin\packages\stm32\tools\xpack-arm-none-eabi-gcc\9.2.1-1.1\bin\../lib/gcc/arm-none-eabi/9.2.1/include
 c:\eclipse\arduinoplugin\packages\stm32\tools\xpack-arm-none-eabi-gcc\9.2.1-1.1\bin\../lib/gcc/arm-none-eabi/9.2.1/include-fixed
 c:\eclipse\arduinoplugin\packages\stm32\tools\xpack-arm-none-eabi-gcc\9.2.1-1.1\bin\../lib/gcc/arm-none-eabi/9.2.1/../../../../arm-none-eabi/include
End of search list.
cc1plus.exe: fatal error: SrcWrapper.h: No such file or directory
compilation terminated.

[jantje]

I added some very special workaround for these boards to sloeber on windows.
If a #include "SrcWrapper.h" statement is added to arduino.h the boards work out of the box.
If you do not do so you'll need to add SrcWrapper library manually.

Note that the workaround disables build_opt and the prebuild commands

[jantje]

@fpistm
I tried adding a #include "SrcWrapper.h" statement to arduino.h in Arduino IDE and arduino IDE did detect and add ScrWrapper Library and then failed for the core build.
The reason is that Arduino IDE does not add the library include folders to the include path during core build. Resulting in ScrWrapper.h to be a unresolved include.
Apart from the include I see no differences between core and sketch compile that could be used to work around this problem. 😒

The only solution I can think of to make ScrWrapper work in both Arduino IDE and Sloeber is to put the parts that can not be archived of scrWrapper in all the variants folders and the rest in the core folder.
As a test I put the content of the src folder of the srcWrapper lib in the variant folder of Nucleo-32 and Arduino IDE processed that one fine (see log)
As you can see from the log there are quite some files in the link command.

arduino build scrwrapper in variant.txt

[fpistm]

The reason is that Arduino IDE does not add the library include folders to the include path during core build. Resulting in ScrWrapper.h to be a unresolved include.

Right. Thant's why I didn't add it as an include in Arudino.h. This would be to easy... 😁

The only solution I can think of to make ScrWrapper work in both Arduino IDE and Sloeber is to put the parts that can not be archived of scrWrapper in all the variants folders and the rest in the core folder.
As a test I put the content of the src folder of the srcWrapper lib in the variant folder of Nucleo-32 and Arduino IDE processed that one fine (see log)

Unfotunately, I don't think I will do that. this will duplicate a lot of files.

[jantje]

@fpistm
Then I would go for a different solution for Sloeber and Arduino IDE
By adding the following to Arduino.h

#ifdef __IN_ECLIPSE__
#include "SrcWraper.h"
#endif

we solve the ScrWrapper issue in Sloeber without impacting Arduino IDE

[fpistm]

I've made a PR with your proposition. Let me know if it is ok.
Note: I've fixed your addition as it misses an 'p' in the file name 😉

[jantje]

Seems fine to me

[hasikp1]

Is there a solution for this? I am getting
In file included from ..\sloeber.ino.cpp:7: C:\STM32Toolchain\eclipse\arduinoPlugin\packages\STM32\hardware\stm32\1.9.0\cores\arduino/Arduino.h:33:12: fatal error: SrcWrapper.h: No such file or directory 33 | #include "SrcWrapper.h" | ^~~~~~~~~~~~~~ compilation terminated. subdir.mk:18: recipe for target 'sloeber.ino.cpp.o' failed make[1]: *** [sloeber.ino.cpp.o] Error 1 makefile:76: recipe for target 'all' failed make: *** [all] Error 2 "C:/STM32Toolchain/eclipse/arduinoPlugin/tools/make/make all" terminated with exit code 2. Build might be incomplete.

[jantje]

@hasikp1
As Sloeber 4.4 is released this should work out of the box.

[rtek1000]

It still persists:

Arduino.h:33:12: fatal error: SrcWrapper.h: No such file or directory

(eclipse/embedcpp-2022-03/eclipse/arduinoPlugin/packages/STMicroelectronics/hardware/stm32/2.4.0)

P.S.: Since it failed to automatically install the STM32 core, so I copied the STM32 functional installation folder in the Arduino IDE to the eclipse folder.

[rtek1000]

Why can't Sloeber use the Arduino IDE functional installation files?

Wouldn't it be possible to indicate a folder path to use the same files? Since the Arduino IDE installation is functional?

[jantje]

@rtek1000 please do not resurrect old issues.