Skip to content

Unable to compile arduino as idf component #5319

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tsctrl opened this issue Jun 23, 2021 · 16 comments
Closed

Unable to compile arduino as idf component #5319

tsctrl opened this issue Jun 23, 2021 · 16 comments
Assignees
Labels
Milestone

Comments

@tsctrl
Copy link

tsctrl commented Jun 23, 2021

Hardware:
Board: ESP32 Dev Module
Core Installation version: ESP-IDF v4.4-dev-1594-g1d7068e4b-dirty
IDE name: IDF component
Flash Frequency: 40Mhz
PSRAM enabled: no
Upload Speed: 115200
Computer OS: Windows 10

Description:
Unable to compile arduino as idf component.

I am doing library update from 4.2 to the current master 4.4 for idf and arduino but having issue with the arduino esp32 lib to compile with error:
I am able to replicate this issue using simple blink project with one additional idf components.
Arduino WiFi / Server need to be call from idf component cpp file.

Step to reproduce:

  1. create new blink project
  2. create components folder
  3. create new .cpp app_main file and replace .c file
  4. download the arduino-esp32
  5. copy the sdkconfig from tools folder
  6. create new component
  7. create new clinkage .cpp static file in components with header
  8. call arduino WiFi.begin / Server.begin in component static function
  9. call component static function from main function

Observation:
can compile and Serial.println works in main files, also work if added from component.
compile failed if i add WiFi.begin() in the component files.

This issue not happening on the 4.2 release

build the project with error as below:

Building in: ~\eclipse-workspace\blink\build
cmake --build . -- -v
[1/9] cmd.exe /C "cd /D ~\eclipse-workspace\blink\build\bootloader && ~\.espressif\tools\cmake\3.16.4\bin\cmake.exe --build ."
[1/1] cmd.exe /C "cd /D ~\eclipse-workspace\blink\build\bootloader\esp-idf\esptool_py && python C:/esp-idf/components/partition_table/check_sizes.py --offset 0x8000 bootloader 0x1000 ~/eclipse-workspace/blink/build/bootloader/bootloader.bin"
Bootloader binary size 0x4300 bytes. 0x2d00 bytes (67%) free.
[2/7] ccache ~\.espressif\tools\xtensa-esp32-elf\esp-2020r3-8.4.0\xtensa-esp32-elf\bin\xtensa-esp32-elf-g++.exe  -DHAVE_CONFIG_H -DMBEDTLS_CONFIG_FILE=\"mbedtls/esp_config.h\" -Iconfig -I../components/test-comp/include -IC:/esp-idf/components/newlib/platform_include -IC:/esp-idf/components/freertos/include -IC:/esp-idf/components/freertos/port/xtensa/include -IC:/esp-idf/components/esp_hw_support/include -IC:/esp-idf/components/esp_hw_support/include/soc -IC:/esp-idf/components/esp_hw_support/port/esp32/. -IC:/esp-idf/components/heap/include -IC:/esp-idf/components/log/include -IC:/esp-idf/components/lwip/include/apps -IC:/esp-idf/components/lwip/include/apps/sntp -IC:/esp-idf/components/lwip/lwip/src/include -IC:/esp-idf/components/lwip/port/esp32/include -IC:/esp-idf/components/lwip/port/esp32/include/arch -IC:/esp-idf/components/soc/include -IC:/esp-idf/components/soc/esp32/. -IC:/esp-idf/components/soc/esp32/include -IC:/esp-idf/components/hal/esp32/include -IC:/esp-idf/components/hal/include -IC:/esp-idf/components/esp_rom/include -IC:/esp-idf/components/esp_rom/esp32 -IC:/esp-idf/components/esp_common/include -IC:/esp-idf/components/esp_system/include -IC:/esp-idf/components/esp_system/port/soc -IC:/esp-idf/components/esp_system/port/public_compat -IC:/esp-idf/components/esp32/include -IC:/esp-idf/components/driver/include -IC:/esp-idf/components/driver/esp32/include -IC:/esp-idf/components/esp_pm/include -IC:/esp-idf/components/esp_ringbuf/include -IC:/esp-idf/components/efuse/include -IC:/esp-idf/components/efuse/esp32/include -IC:/esp-idf/components/xtensa/include -IC:/esp-idf/components/xtensa/esp32/include -IC:/esp-idf/components/vfs/include -IC:/esp-idf/components/esp_wifi/include -IC:/esp-idf/components/esp_event/include -IC:/esp-idf/components/esp_netif/include -IC:/esp-idf/components/esp_eth/include -IC:/esp-idf/components/tcpip_adapter/include -IC:/esp-idf/components/esp_phy/include -IC:/esp-idf/components/esp_phy/esp32/include -IC:/esp-idf/components/app_trace/include -IC:/esp-idf/components/esp_timer/include -I../components/arduino/variants/esp32 -I../components/arduino/cores/esp32 -I../components/arduino/libraries/ArduinoOTA/src -I../components/arduino/libraries/AsyncUDP/src -I../components/arduino/libraries/BLE/src -I../components/arduino/libraries/BluetoothSerial/src -I../components/arduino/libraries/DNSServer/src -I../components/arduino/libraries/EEPROM/src -I../components/arduino/libraries/ESP32/src -I../components/arduino/libraries/ESPmDNS/src -I../components/arduino/libraries/FFat/src -I../components/arduino/libraries/FS/src -I../components/arduino/libraries/HTTPClient/src -I../components/arduino/libraries/HTTPUpdate/src -I../components/arduino/libraries/LITTLEFS/src -I../components/arduino/libraries/NetBIOS/src -I../components/arduino/libraries/Preferences/src -I../components/arduino/libraries/RainMaker/src -I../components/arduino/libraries/SD_MMC/src -I../components/arduino/libraries/SD/src -I../components/arduino/libraries/SimpleBLE/src -I../components/arduino/libraries/SPIFFS/src -I../components/arduino/libraries/SPI/src -I../components/arduino/libraries/Ticker/src -I../components/arduino/libraries/Update/src -I../components/arduino/libraries/WebServer/src -I../components/arduino/libraries/WiFiClientSecure/src -I../components/arduino/libraries/WiFi/src -I../components/arduino/libraries/WiFiProv/src -I../components/arduino/libraries/Wire/src -IC:/esp-idf/components/spi_flash/include -IC:/esp-idf/components/mbedtls/port/include -IC:/esp-idf/components/mbedtls/mbedtls/include -IC:/esp-idf/components/mbedtls/esp_crt_bundle/include -IC:/esp-idf/components/mdns/include -IC:/esp-idf/components/console -IC:/esp-idf/components/esp_adc_cal/include -IC:/esp-idf/components/wifi_provisioning/include -IC:/esp-idf/components/protocomm/include/common -IC:/esp-idf/components/protocomm/include/security -IC:/esp-idf/components/protocomm/include/transports -IC:/esp-idf/components/bt/common/osi/include -IC:/esp-idf/components/bt/include/esp32/include -IC:/esp-idf/components/bt/host/bluedroid/api/include/api -IC:/esp-idf/components/nvs_flash/include -IC:/esp-idf/components/nghttp/port/include -IC:/esp-idf/components/nghttp/nghttp2/lib/includes -mlongcalls -Wno-frame-address   -ffunction-sections -fdata-sections -Wall -Werror=all -Wno-error=unused-function -Wno-error=unused-variable -Wno-error=deprecated-declarations -Wextra -Wno-unused-parameter -Wno-sign-compare -ggdb -mfix-esp32-psram-cache-issue -mfix-esp32-psram-cache-strategy=memw -Og -fstack-protector -fmacro-prefix-map=~/eclipse-workspace/blink=. -fmacro-prefix-map=C:/esp-idf=IDF -fstrict-volatile-bitfields -Wno-error=unused-but-set-variable -std=gnu++11 -fexceptions -fno-rtti -D_GNU_SOURCE -DIDF_VER=\"v4.4-dev-1594-g1d7068e4b-dirty\" -DESP_PLATFORM -DARDUINO=10812 -DARDUINO_ESP32_DEV -DARDUINO_ARCH_ESP32 -DARDUINO_BOARD=\"ESP32_DEV\" -DARDUINO_VARIANT=\"esp32\" -DESP32 -MD -MT esp-idf/test-comp/CMakeFiles/__idf_test-comp.dir/test-comp.cpp.obj -MF esp-idf\test-comp\CMakeFiles\__idf_test-comp.dir\test-comp.cpp.obj.d -o esp-idf/test-comp/CMakeFiles/__idf_test-comp.dir/test-comp.cpp.obj -c ../components/test-comp/test-comp.cpp
[3/7] cmd.exe /C "cd . && ~\.espressif\tools\cmake\3.16.4\bin\cmake.exe -E remove esp-idf\test-comp\libtest-comp.a && ~\.espressif\tools\xtensa-esp32-elf\esp-2020r3-8.4.0\xtensa-esp32-elf\bin\xtensa-esp32-elf-ar.exe qc esp-idf\test-comp\libtest-comp.a  esp-idf/test-comp/CMakeFiles/__idf_test-comp.dir/test-comp.cpp.obj && ~\.espressif\tools\xtensa-esp32-elf\esp-2020r3-8.4.0\xtensa-esp32-elf\bin\xtensa-esp32-elf-ranlib.exe esp-idf\test-comp\libtest-comp.a && cd ."
[4/7] cmd.exe /C "cd /D ~\eclipse-workspace\blink\build\esp-idf\esp32 && python C:/esp-idf/tools/ldgen/ldgen.py --config ~/eclipse-workspace/blink/sdkconfig --fragments C:/esp-idf/components/spi_flash/linker.lf	C:/esp-idf/components/esp_gdbstub/linker.lf	C:/esp-idf/components/espcoredump/linker.lf	C:/esp-idf/components/esp_phy/linker.lf	C:/esp-idf/components/esp_system/linker.lf	C:/esp-idf/components/esp_system/app.lf	C:/esp-idf/components/hal/linker.lf	C:/esp-idf/components/esp_event/linker.lf	C:/esp-idf/components/esp_wifi/linker.lf	C:/esp-idf/components/lwip/linker.lf	C:/esp-idf/components/log/linker.lf	C:/esp-idf/components/heap/linker.lf	C:/esp-idf/components/soc/linker.lf	C:/esp-idf/components/esp_hw_support/linker.lf	C:/esp-idf/components/esp_pm/linker.lf	C:/esp-idf/components/esp_ringbuf/linker.lf	C:/esp-idf/components/driver/linker.lf	C:/esp-idf/components/xtensa/linker.lf	C:/esp-idf/components/esp_common/common.lf	C:/esp-idf/components/esp_common/soc.lf	C:/esp-idf/components/freertos/linker.lf	C:/esp-idf/components/newlib/esp32-spiram-rom-functions-c.lf	C:/esp-idf/components/newlib/newlib.lf	C:/esp-idf/components/newlib/system_libs.lf	C:/esp-idf/components/app_trace/linker.lf	C:/esp-idf/components/bt/linker.lf --input C:/esp-idf/components/esp32/ld/esp32.project.ld.in --output ~/eclipse-workspace/blink/build/esp-idf/esp32/ld/esp32.project.ld --kconfig C:/esp-idf/Kconfig --env-file ~/eclipse-workspace/blink/build/config.env --libraries-file ~/eclipse-workspace/blink/build/ldgen_libraries --objdump ~/.espressif/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-objdump.exe"
[5/7] cmd.exe /C "cd . && ~\.espressif\tools\xtensa-esp32-elf\esp-2020r3-8.4.0\xtensa-esp32-elf\bin\xtensa-esp32-elf-g++.exe  -mlongcalls -Wno-frame-address   @CMakeFiles\blink.elf.rsp  -o blink.elf  && cd ."
FAILED: blink.elf 
cmd.exe /C "cd . && ~\.espressif\tools\xtensa-esp32-elf\esp-2020r3-8.4.0\xtensa-esp32-elf\bin\xtensa-esp32-elf-g++.exe  -mlongcalls -Wno-frame-address   @CMakeFiles\blink.elf.rsp  -o blink.elf  && cd ."
~/.espressif/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: esp-idf/test-comp/libtest-comp.a(test-comp.cpp.obj):(.literal._ZN8testcomp4funcEv+0xc): undefined reference to `WiFi'
~/.espressif/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: esp-idf/test-comp/libtest-comp.a(test-comp.cpp.obj):(.literal._ZN8testcomp4funcEv+0x10): undefined reference to `WiFiSTAClass::begin(char const*, char const*, int, unsigned char const*, bool)'
~/.espressif/tools/xtensa-esp32-elf/esp-2020r3-8.4.0/xtensa-esp32-elf/bin/../lib/gcc/xtensa-esp32-elf/8.4.0/../../../../xtensa-esp32-elf/bin/ld.exe: esp-idf/test-comp/libtest-comp.a(test-comp.cpp.obj): in function `testcomp::func()':
~\eclipse-workspace\blink\build/../components/test-comp/test-comp.cpp:16: undefined reference to `WiFiSTAClass::begin(char const*, char const*, int, unsigned char const*, bool)'
collect2.exe: error: ld returned 1 exit status
ninja: build stopped: subcommand failed.
Build complete (4 errors, 0 warnings): ~\eclipse-workspace\blink\build
Total time taken to build the project: 16,746 ms

~some path detail are omitted from the logs

edit: some correction and formatting.

thanks!

@chegewara
Copy link
Contributor

How CMakeLists.txt looks like in test-comp?

@tsctrl
Copy link
Author

tsctrl commented Jun 23, 2021

idf_component_register(SRCS "test-comp.cpp" 
REQUIRES        arduino INCLUDE_DIRS "include")

@tsctrl
Copy link
Author

tsctrl commented Jun 23, 2021

issue happens with arduino loop() or app_main with clinkage

Arduino Revision: 21947eb
Author: Clemens Kirchgatterer [email protected]
Date: 6/18/2021 18:10:40

edit: arduino revision

@me-no-dev
Copy link
Member

can you please create a minimal example project (exclude the arduino component) zip and upload it here?

@tsctrl
Copy link
Author

tsctrl commented Jun 27, 2021

sample_project.zip

hi @me-no-dev thank you for your reply.
please find attached sample project. project was created and build on 4.3. should be the same for 4.4 master
issue is in 4.4/master

esptool.py v3.1-dev
Generated sample_project/build/main.bin
Build complete (0 errors, 0 warnings) :sample_project\build
Total time taken to build the project: 32,601 ms

I (319) component1: [component1]: component1 RUNNING
component2 RUNNING!
I (329) component1: [component1]: component1 RUNNING
app_main started!

@chegewara
Copy link
Contributor

I can confirm that using WiFi (and probably other arduino libs) in component is broken. I think it is something in new toolchain.

@me-no-dev
Copy link
Member

this is really confusing... why there is no issue calling Serial.print(), but it is an issue to call WiFi.begin()? Both should be just the same ...

@me-no-dev
Copy link
Member

@tsctrl you've gone all wild with the extern "C" into the files :D

You can call WiFi.begin inside the component, as long as you call any WiFi function in the main component. Sounds strange but it works :)

@chegewara
Copy link
Contributor

That make sense sort of.
WiFi is global variable and probably adding extern into component could work too. Nice catch @me-no-dev

@me-no-dev
Copy link
Member

@tsctrl here is your sample project cleaned and working

sample_project.zip

@me-no-dev
Copy link
Member

Alright :) here is the fix:
You need to add the following line AFTERidf_component_register in the component's CMakeLists.txt

set_property(TARGET ${COMPONENT_LIB} APPEND PROPERTY LINK_INTERFACE_MULTIPLICITY 3)

End result should look something like:

idf_component_register(
SRCS "component2.cpp" 
INCLUDE_DIRS "include"
REQUIRES arduino-esp32
)
set_property(TARGET ${COMPONENT_LIB} APPEND PROPERTY LINK_INTERFACE_MULTIPLICITY 3)

There is some effort to make this line not required, but for right now, this is a solution :)

@me-no-dev me-no-dev added the Type: For reference Common questions & problems label Jun 29, 2021
@tsctrl
Copy link
Author

tsctrl commented Jun 29, 2021

thanks @me-no-dev

@VojtechBartoska VojtechBartoska added this to the 2.0.0 milestone Jul 14, 2021
@tsctrl
Copy link
Author

tsctrl commented Jul 16, 2021

hi, few more wish to have to this:

i hope that arduino as component can be initialize as previous setup, without depending on setup and loop function. the loop and setup should be optional. i am not using setup and loop as it require additional larga stack. its looks redundant on top of the main task with existing stack as we could run while(;;) loop in it. should have the ability to disable the stack on top of the auto start loop in config.

i never got issue running while loop in the main function so not sure adding setup and loop add benefits per ram usage that required. the new changes where loop is extern c also not favorable to me. its add limits then the user add the extern them self.

cc, @me-no-dev

thanks.

@igrr
Copy link
Member

igrr commented Jul 16, 2021

@tsctrl do you have CONFIG_AUTOSTART_ARDUINO enabled in menuconfig? I think if you don't enable it, then you can simply call initArduino from your app_main function and then proceed with any other initialization steps.

igrr added a commit that referenced this issue Jul 16, 2021
Until this commit, Kconfig options (e.g. CONFIG_TINYUSB_ENABLED) were
used in conditions preceding idf_component_register to determine which
components need to be added to `arduino` component requirements.
However the Kconfig options aren't known at the early expansion stage,
when the component CMakeLists.txt files are expanded the first time
and requirements are evaluated. So all the conditions evaluated as if
the options were not set.
This commit changes the logic to only add these components as
dependencies when the Kconfig options are known. Dependencies become
"weak", which means that if one of the components isn't included into
the build for some reason, it is not added as a dependency.
This may happen, for example, if the component is not present in the
`components` directory or is excluded by setting `COMPONENTS` variable
in the project CMakeLists.txt file.
This also ensures that if the component is not present, it will not be
added as a dependency, and this will allow the build to proceed.

Follow-up to #5391.
Closes #5319.
@igrr igrr added Status: In Progress ⚠️ Issue is in progress Type: Bug 🐛 All bugs and removed Type: For reference Common questions & problems labels Jul 16, 2021
igrr added a commit that referenced this issue Jul 16, 2021
Until this commit, Kconfig options (e.g. CONFIG_TINYUSB_ENABLED) were
used in conditions preceding idf_component_register to determine which
components need to be added to `arduino` component requirements.
However the Kconfig options aren't known at the early expansion stage,
when the component CMakeLists.txt files are expanded the first time
and requirements are evaluated. So all the conditions evaluated as if
the options were not set.
This commit changes the logic to only add these components as
dependencies when the Kconfig options are known. Dependencies become
"weak", which means that if one of the components isn't included into
the build for some reason, it is not added as a dependency.
This may happen, for example, if the component is not present in the
`components` directory or is excluded by setting `COMPONENTS` variable
in the project CMakeLists.txt file.
This also ensures that if the component is not present, it will not be
added as a dependency, and this will allow the build to proceed.

Follow-up to #5391.
Closes #5319.
me-no-dev pushed a commit that referenced this issue Jul 16, 2021
…5404)

Until this commit, Kconfig options (e.g. CONFIG_TINYUSB_ENABLED) were
used in conditions preceding idf_component_register to determine which
components need to be added to `arduino` component requirements.
However the Kconfig options aren't known at the early expansion stage,
when the component CMakeLists.txt files are expanded the first time
and requirements are evaluated. So all the conditions evaluated as if
the options were not set.
This commit changes the logic to only add these components as
dependencies when the Kconfig options are known. Dependencies become
"weak", which means that if one of the components isn't included into
the build for some reason, it is not added as a dependency.
This may happen, for example, if the component is not present in the
`components` directory or is excluded by setting `COMPONENTS` variable
in the project CMakeLists.txt file.
This also ensures that if the component is not present, it will not be
added as a dependency, and this will allow the build to proceed.

Follow-up to #5391.
Closes #5319.
@tsctrl
Copy link
Author

tsctrl commented Jul 17, 2021

hi @igrr

yup, i disable the auto start and i guess it does not register the task and did not run any initialization.

but still, there is arduino stack size option in the config. i an able to reduce the size but just hope it does not impacting any functionality.

@igrr
Copy link
Member

igrr commented Jul 17, 2021

If autostart option is disabled then no stack is created by Arduino core.

We can make this Kconfig option hidden when autostart is disabled, so it doesn't confuse anybody.

@VojtechBartoska VojtechBartoska removed the Status: In Progress ⚠️ Issue is in progress label Jul 19, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants