section type conflict with __c

[ewasscher]

Basic Infos

With (a modified version of) the arduino-dsmr library (https://github.com/matthijskooijman/arduino-dsmr) building the included examples for ESP8266 fails with "section type conflict with __c"

Hardware

Hardware: Any ESP8266
Core Version: 2.2.0

Description

With (a modified version of) the arduino-dsmr library (https://github.com/matthijskooijman/arduino-dsmr) building the included examples for ESP8266 fails with "section type conflict with __c" when using PROGMEM string literals (e.g. INVALID_UNIT) in one of the library's headers. Should this be considered a bug in the ESP core? I can work around this, but if there's a bug in the core involved I'd prefer not to do so and wait a bit more for the bugfix.

Settings in IDE

Not appliccable

Sketch

See the "parse" example sketch in the attached library

Debug Messages

In file included from /home/ewasscher/Arduino/libraries/arduino-dsmr/src/dsmr.h:35:0,
                 from /home/ewasscher/Arduino/arduino-dsmr-read-test/arduino-dsmr-read-test.ino:22:
/home/ewasscher/Arduino/libraries/arduino-dsmr/src/dsmr/parser.h:160:31: error: dsmr::INVALID_UNIT causes a section type conflict with __c
 static constexpr char PROGMEM INVALID_UNIT[] = "Invalid unit";
                               ^
In file included from /home/ewasscher/.arduino15/packages/esp8266/hardware/esp8266/2.2.0/cores/esp8266/Arduino.h:244:0,
                 from /tmp/build8e00daea58ed4d885aed86a35f556178.tmp/sketch/arduino-dsmr-read-test.ino.cpp:1:
/home/ewasscher/.arduino15/packages/esp8266/hardware/esp8266/2.2.0/cores/esp8266/pgmspace.h:21:51: note: '__c' was declared here
 #define PSTR(s) (__extension__({static const char __c[] PROGMEM = (s); &__c[0];}))
                                                   ^
/home/ewasscher/.arduino15/packages/esp8266/hardware/esp8266/2.2.0/cores/esp8266/WString.h:38:76: note: in definition of macro 'FPSTR'
 #define FPSTR(pstr_pointer) (reinterpret_cast<const __FlashStringHelper *>(pstr_pointer))
                                                                            ^
/home/ewasscher/.arduino15/packages/esp8266/hardware/esp8266/2.2.0/cores/esp8266/WString.h:39:34: note: in expansion of macro 'PSTR'
 #define F(string_literal) (FPSTR(PSTR(string_literal)))
                                  ^
/home/ewasscher/Arduino/arduino-dsmr-read-test/arduino-dsmr-read-test.ino:110:20: note: in expansion of macro 'F'
       Serial.print(F(": "));

arduino-dsmr-esp.zip

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[igrr]

/cc @Makuna

[matthijskooijman]

@ewasscher, does this also occur with a simple sketch that just has a single PROGMEM global and a single F() string?

[igrr]

This compiles fine:

static constexpr char PROGMEM INVALID_UNIT[] = "Invalid unit";

void setup() {
  Serial.println(INVALID_UNIT);
  Serial.println(F(": "));
}

void loop() {
}

[matthijskooijman]

Here's a related thread that @ewasscher posted in the PR that prompted this issue before: http://stackoverflow.com/questions/35091862/inline-static-data-causes-a-section-type-conflict

IIUC, the section attribute (which is what PROGMEM expands to through the ICACHE_RODATA_ATTR macro) is not supposed to be used on local variables, only globals. I'm not entirely sure if that is the cause of this particular issue, though it might be.

The section conflict seems to mean that two variables end up in the same section, but having different linkage (or perhaps not just different, but also conflicting linkage that would result in different section-wide attributes).

I suspect the location of the F() is relevant, and just putting it in setup() doesn't require different enough linkage to break things. Looking at the original code, perhaps this does break:

static constexpr char PROGMEM INVALID_UNIT[] = "Invalid unit";

struct Printer {
  template<typename Item>
  void apply(Item i) {
    Serial.print(F(": "));
  }
};

void setup() {
  Serial.print(INVALID_UNIT);
  Printer().apply(1);
}

void loop() {
}

Being a template method, apply() might end up with some weird linkage. Perhaps the struct can be removed as well, just a global template function might also break I believe.

[ewasscher]

The sketch from the previous post does not build for ESP8266 2.2.0 core, it does for Nano and Due though.

sketch_may31a:1: error: INVALID_UNIT causes a section type conflict with __c
 static constexpr char PROGMEM INVALID_UNIT[] = "Invalid unit";
                               ^
In file included from /home/ewasscher/.arduino15/packages/esp8266/hardware/esp8266/2.2.0/cores/esp8266/Arduino.h:244:0,
                 from /tmp/builddcb292e80a208e3f29f1a50e37dee40e.tmp/sketch/sketch_may31a.ino.cpp:1:
/home/ewasscher/.arduino15/packages/esp8266/hardware/esp8266/2.2.0/cores/esp8266/pgmspace.h:21:51: note: '__c' was declared here
 #define PSTR(s) (__extension__({static const char __c[] PROGMEM = (s); &__c[0];}))
                                                   ^
/home/ewasscher/.arduino15/packages/esp8266/hardware/esp8266/2.2.0/cores/esp8266/WString.h:38:76: note: in definition of macro 'FPSTR'
 #define FPSTR(pstr_pointer) (reinterpret_cast<const __FlashStringHelper *>(pstr_pointer))
                                                                            ^
/home/ewasscher/.arduino15/packages/esp8266/hardware/esp8266/2.2.0/cores/esp8266/WString.h:39:34: note: in expansion of macro 'PSTR'
 #define F(string_literal) (FPSTR(PSTR(string_literal)))
                                  ^
/home/ewasscher/Arduino/sketch_may31a/sketch_may31a.ino:6:18: note: in expansion of macro 'F'
     Serial.print(F(": "));
                  ^
exit status 1
INVALID_UNIT causes a section type conflict with __c

[Makuna]

@ewasscher the thread @matthijskooijman referenced sums up the issue.

Due to the use of the templates function for apply() in the struct, GCC will not support it and it creates this error.

Work around is to move the definition to a global or out of a function that relies on the linker to place it. This has another side effect in that if you had several uses in your code of the same string, before they would have each had a unique memory address, by making one global and using it, you save a little memory.

[ewasscher]

@matthijskooijman do you understand and agree? I'm not sure I do.

[matthijskooijman]

@Makuna, yeah that seems about right. What puzzles me, though, is why this does work for AVR. AFAIK PROGMEM is also defined as a section attribute there, but this section conflict does not seem to occur there...

[ewasscher]

@matthijskooijman on AVR PROGMEM
expands to attribute((progmem))

[Makuna]

The AVR doesn't use the GCC compiler.
Attribute support and nuances changes from compiler to compiler.

__progmem__ is a really a ATMEL/AVR specific attribute, we tried to simulate it with ESP8266 the best we could, but there are always going to be unique things that we just can't control; and I think we hit one here.

[matthijskooijman]

on AVR PROGMEM expands to attribute((progmem))

Right, I misremembered that, then. That would explain why this works on AVR.

The AVR doesn't use the GCC compiler.

Huh? It certainly does, it calls avr-gcc (which is compiled from regular gcc sources).

progmem is a really a ATMEL/AVR specific attribute, we tried to simulate it with ESP8266 the best we could, but there are always going to be unique things that we just can't control; and I think we hit one here.

Right, I can't really think of any way to work around this from the core, while keeping the section attribute, and I guess I'll have to just work around this in my library (which, admittedly, isn't like your average Arduino library with its usage of templates). I looked a bit more at the objects mgenerated by the ESP compiler and the problem is indeed the same as in the stackoverflow: The template static variable gets the "GROUP" attribute (whatever that means exactly, couldn't find any docs about that), while the global one does not. It does not become "unique" as suggested by the article, so the route to getting this "GROUP" attribute is probably slightly different.

However, looking more closely at the code, like how pgm_read_byte() is implemented, is all this PROGMEM-stuff even needed for ESP at all? It seems all that pgm_read_byte() really does is 32-bit-aligning reads from flash, and PROGMEM apparently forces things into flash. But can't the compiler do this automatically for const variables? Or is it too tricky to guarantee 32-bit aligned accesses, so it resorts to just copying constants to memory anyway?

[Makuna]

Some platforms don't have separate code and ram restrictions at all. AVR it was different asm calls to access it, esp8266, it just has a restriction that it must be read by 32bits calls only. PROGMEM was to abstract "if and how" it does it. So its needed for compatibility, if you want your code to run across platforms; and for many libraries this is desired.

And no, there is no guarantee that const will force it into flash.

[igrr]

@matthijskooijman consider the following code fragment

char get_first_char() {
  const char* p = get_some_string();
  return p[0];
}

In general, there is no way for the compiler to figure out where p will point to at runtime (flash or RAM).
It is possible to force the compiler to always use 32-bit loads instead of 8-bit and 16-bit ones, but it obviously comes with a cost.
There is a version of gcc for xtensa which introduces this option, we don't use it because it may break compatibility with pre-built binary libraries from the SDK in some cases.

[matthijskooijman]

Thanks for the explanations and answers, I indeed think this is something that's just impossible to fix and needs to be worked around. @ewasscher already submitted a PR for my library to prevent this issue there. Thanks!

[ssilverman]

@Makuna The teensy puts all const-declared variables into flash somehow, so maybe it's possible here? (Regarding your comment, "And no, there is no guarantee that const will force it into flash.")

[Makuna]

@ssilverman Tensy is ARM. Esp8266 does not; I suspect due to the restrictions on reads being 32bit only for flash of which ARM may not have these. But its a mute point, for compatibility reasons you should stick to the PROGMEM functions.