ARDUINO_BOARD not defined correctly on compile command line

[liebman]

Describe the bug
In an ESP8266 project the compile command does not include proper quoting for the ARDUINO_BOARD macro. Using the arduino IDE its passed as "-DARDUINO_BOARD=\"ESP8266_GENERIC\"" and with sloeber its passed as -DARDUINO_BOARD="ESP8266_NODEMCU" This causes a compilation error when using the ESP8266mDNS library as the value of ARDUINO_BOARD is expected to be a string.

make all 
Building file: /Users/chris.l/Dropbox/git/ESPButton/hardware/esp8266com/esp8266/libraries/ESP8266mDNS/ESP8266mDNS.cpp
Starting C++ compile
"/Users/chris.l/Dropbox/git/ESPButton/hardware/esp8266com/esp8266/tools/xtensa-lx106-elf/bin/xtensa-lx106-elf-g++" -D__ets__ -DICACHE_FLASH -U__STRICT_ANSI__ "-I/Users/chris.l/Dropbox/git/ESPButton/hardware/esp8266com/esp8266/tools/sdk/include" "-I/Users/chris.l/Dropbox/git/ESPButton/hardware/esp8266com/esp8266/tools/sdk/lwip2/include" "-I/Users/chris.l/Dropbox/git/ESPButton/hardware/esp8266com/esp8266/tools/sdk/libc/xtensa-lx106-elf/include" "-I/Users/chris.l/Dropbox/git/ESPButton/HTTPServerTest/Release/core" -c -Wall -Wextra -Os -g -mlongcalls -mtext-section-literals -fno-exceptions -fno-rtti -falign-functions=4 -std=c++11 -MMD -ffunction-sections -fdata-sections -DF_CPU=160000000L -DLWIP_OPEN_SRC -DTCP_MSS=536 -DLWIP_FEATURES=0 -DLWIP_IPV6=0 -DDEBUG_ESP_PORT=Serial -DDEBUG_ESP_HTTP_SERVER -DARDUINO=10802 -DARDUINO_ESP8266_NODEMCU -DARDUINO_ARCH_ESP8266 -DARDUINO_BOARD="ESP8266_NODEMCU"   -DESP8266   -I"/Users/chris.l/Dropbox/git/ESPButton/hardware/esp8266com/esp8266/cores/esp8266" -I"/Users/chris.l/Dropbox/git/ESPButton/hardware/esp8266com/esp8266/variants/nodemcu" -I"/Users/chris.l/Dropbox/git/ESPButton/hardware/esp8266com/esp8266/libraries/ESP8266WebServer/src" -I"/Users/chris.l/Dropbox/git/ESPButton/hardware/esp8266com/esp8266/libraries/ESP8266WiFi/src" -I"/Users/chris.l/Dropbox/git/ESPButton/hardware/esp8266com/esp8266/libraries/ESP8266mDNS" -MMD -MP -MF"libraries/ESP8266mDNS/ESP8266mDNS.cpp.d" -MT"libraries/ESP8266mDNS/ESP8266mDNS.cpp.o" -D__IN_ECLIPSE__=1 -x c++ "/Users/chris.l/Dropbox/git/ESPButton/hardware/esp8266com/esp8266/libraries/ESP8266mDNS/ESP8266mDNS.cpp"  -o  "libraries/ESP8266mDNS/ESP8266mDNS.cpp.o"
/Users/chris.l/Dropbox/git/ESPButton/hardware/esp8266com/esp8266/libraries/ESP8266mDNS/ESP8266mDNS.cpp: In member function 'void MDNSResponder::enableArduino(uint16_t, bool)':
<command-line>:0:15: error: 'ESP8266_NODEMCU' was not declared in this scope
/Users/chris.l/Dropbox/git/ESPButton/hardware/esp8266com/esp8266/libraries/ESP8266mDNS/ESP8266mDNS.cpp:972:44: note: in expansion of macro 'ARDUINO_BOARD'
   addServiceTxt("arduino", "tcp", "board", ARDUINO_BOARD);
                                            ^
make: *** [libraries/ESP8266mDNS/ESP8266mDNS.cpp.o] Error 1

To Reproduce
Steps to reproduce the behavior:

1. Create an esp8266 project (use any of the ESP8266mDNS library examples.
2. Try to compile
3. See error

Expected behavior
expect no error

Desktop (please complete the following information):

• OS: macOS 10.14.1

[jantje]

This is a duplicate of #1028
Workaround is described there

[liebman]

I don't find that workaround. in #1046, acceptable as it requires modifying a library not in my control, its part of ESP8266/Ardunio core. I'd then have to make this same change every time I updated. Maybe a better solution would be to escape any " in values as the Arduino IDE does.

[jantje]

I would prefer escaping the quotes but it is not simple for several reasons.
The main reason Arduino IDE gets away with it is because they do not run native on the windows prompt but in a cygwin runtime (which is a linux simulator for windows).
If you see Mac as a Linux variant this makes they only have to support Linux where Sloeber supports windows and linux.
This makes the problem far more complex.
For instance escaping and quotes work completely different in windows and linux.
Secondly there are several levels of excaping
The first level of escaping is ->the os gives the quote to build tool
The second level of escaping is -> the build tool gives the quote to the "define"
Sloeber does the first level and not the second.

So when a quote is found Sloeber needs to find out whether it needs to be escaped (not needed for file paths) and if it needs escaping whether it is a first level escape or a second level escape
Then Sloeber needs to find the closing quotes. (if quotes are in between the quotes-> process them properly).
Then Sloeber needs to modify the quoted text (per os directives) to achieve the correct escaping level.

All these things are hard. For instance I don't even know how to do the escaping levels in windows let alone linux or mac.
Normally I would look at the arduino ide but as they use cygwin ....

[liebman]

In other words this is a known incompatibility with Arduino and won't be fixed? :-(

[jantje]

In other words this is a known incompatibility with Arduino and won't be fixed? :-(

Unless someone comes up with a great idea, yes

[liebman]

I don't have windows available to test but I believe you could escape the double quotes with ^ on windows and \ on both linux and macOS. That said I recall that on windows globbing and quoting are handled by the executable being run so its possible that \ cold possibly work.

[jantje]

That works for the first level and is implemented by Sloeber.
It is the second level that is a problem as described above

[jantje]

rereading I see

"-DARDUINO_BOARD="ESP8266_GENERIC"" and with sloeber its passed as -DARDUINO_BOARD="ESP8266_NODEMCU"

The difference ESP8266_NODEMCU versus ESP8266_GENERIC is probably because you do not have the exact same setup in both cases. probably a different board

[liebman]

True, I did not set the board for a compile test with Arduino IDE - it also compiles fine with nodemcu selected - Sloeber still fails with either board selection.

[jantje]

To get to the dirty details This is what the recipe is supposed to be as to the platform.txt file

## Compile c++ files
recipe.cpp.o.pattern="{compiler.path}{compiler.cpp.cmd}" {compiler.cpreprocessor.flags} {compiler.cpp.flags} -DF_CPU={build.f_cpu} {build.lwip_flags} {build.debug_port} {build.debug_level} -DARDUINO={runtime.ide.version} -DARDUINO_{build.board} -DARDUINO_ARCH_{build.arch} -DARDUINO_BOARD="\"{build.board}\"" {build.led} {compiler.cpp.extra_flags} {build.extra_flags} {includes} "{source_file}" -o "{object_file}"

On windows §like you say on linux the is translated to -DARDUINO_{build.board}
"-DARDUINO_BOARD="ESP8266_NODEMCU""

In contrast to what I believed this command works on windows natively.
But where do the extra quotes come from?
When are they needed?

[liebman]

Whats needed is to escape the quotes around the board type -DARDUINO_BOARD="\"{build.board}\"" so that whatever is parsing the command options (shell on linux/macos and the command on windows) will not strip them. On linux/macos somehow not stripping the \ should work also.

zod: 07:01:17 device chris.l$ echo xxx="\"foo\""
xxx="foo"

Since I don't know how Sloeber is processing these commands, and what quotes may be stripped before it hits a shell on linux/macos, I don't know how much I can help other than when the command is finally passed to the shell to execute it needs to have both " on the value escaped so that they are not stripped by the shell.

[jantje]

You can help by running tests on linux :-)
If I change something fundamental like changing the platfomr.txt and boards.txt i want to run extensive tests. These are not hard to start but take hours to run

[liebman]

I would be glad to. (at least for macOS)

[jantje]

Investigation shows it is more complicated than seems :-s
The platform.txt download from espressif contains
-DARDUINO_BOARD="{build.board}"

At install time sloeber runs following code
// on Windows fix esp8266 platform.txt file // replace -DARDUINO_BOARD="{build.board}" with // -DARDUINO_BOARD="\"{build.board}\"" if (SystemUtils.IS_OS_WINDOWS) { if ("esp8266".equals(platform.getArchitecture()) && "esp8266".equals(platform.getName())) { //$NON-NLS-1$ //$NON-NLS-2$ FileModifiers.replaceInFile(platform.getPlatformFile(), false, "-DARDUINO_BOARD=\"{build.board}\"", //$NON-NLS-1$ "-DARDUINO_BOARD=\"\\\"{build.board}\\\"\""); //$NON-NLS-1$ } }
Which boils down to replace -DARDUINO_BOARD="{build.board}" with -DARDUINO_BOARD="\"{build.board}\"" in the platform.txt for esp8266 on windows.

If I look at the platform.txt of esp8266 (all versions I have installed) I see
-DARDUINO_BOARD="\"{build.board}\""

and I also get
-DARDUINO_BOARD=""ESP8266_NODEMCU""
in the command which works with the ESP8266mDNS library.

So it looks like the "windows fix" is also needed for macOS and maybe even for Linux?
Can you test by doing a global find and replace of -DARDUINO_BOARD="{build.board}" with ` -DARDUINO_BOARD="" in the platform.txt. save the file
open the project properties of the impacted project and select apply ok.
Then build?

[jantje]

Seems like esp32 needs this as well. https://www.patreon.com/posts/23070587?utm_medium=post_notification_email&utm_source=post_link&utm_campaign=patron_engagement
So maybe this "windows fix" needs to be applied all the time.

[jantje]

As it looks that the environment variable ARDUINO_BOARD is only used in ESP826 and ESP32 (I didn't find any other platform using this) the fix is pretty safe.
To get the fix update to the nightly (from tomorrow onwards)
Using the boardsmanager
deinstall the esp platform
install the esp platform
open the project properties of the project having this issue->arduino->aplly and OK

[Crazor]

Just chiming in to let you know that yes, the fix is definitely needed on macOS, and probably Linux, as well. Haven't checked the nightly for this, but applied the fix to my platform.txt manually.

[jantje]

It seems ARDUINO_VARIANT is another one causing issues with ESP32