Getting back the old Serial handling?

[JonesWest]

Hi all,

We've upgraded the esp arduino library from 2.0.0 to 2.2.0 recently, only to realize that the UART buffer handling has been changed from SW buffers to HW buffers. Resulting in a RX-buffer limit on 128bytes (because it's hardware buffers). Well, we need at least 2048bytes, so we basicly need the old Serial / UART handling again? Can we change the Serial libraries back to the old version, and keep the ESP updated code?

(Yes, we've tried to empty the hardware buffer fast as possible but it is not fast enough - it takes around 8ms to fill up the HW buffer, and we have some tasks taking up to 10-12ms)

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

[miky2k]

The old software implementation is a software as your code .
You must listen buffer full event if aviable and start dma transfer

[igrr]

You may try placing older HardwareSerial class into the latest core. No guarantees that it will work without issues though. A few places in the code now assume that Serial functions are reentrant, and it is safe to call them from any task. This is not true with the older HardwareSerial, so some things (like debugging output from certain libraries) may cause unexpected behavior.

[Duality4Y]

is there an interrupt for when the buffer is empty then you could fill it when it's empty :)

[JonesWest]

Okay thanks!

I've tried to just replace HardwareSerial.h, HardwareSerial.cpp and cbuf.h in the latest core with the files from 2.0.0.. It seems to work fine :)

[JonesWest]

I get an exception though. I have a HOST microcontroller that sends data chunks of 1024bytes over the serial to the ESP, for it to send through a WiFi socket.

Sometimes it works (the whole file is 25kb), but every other time it throws the following stack trace:

0x40209068: HardwareSerial::isRxEnabled() at C:\Users\jwa\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.2.0\cores\esp8266/HardwareSerial.cpp line 550
0x4022d3f6: ram_restart_cal at ?? line ?
0x40210fe6: ieee80211_ht_updateparams at ?? line ?
0x40211a25: ieee80211_parse_beacon at ?? line ?
0x4010576c: ets_timer_arm_new at ?? line ?
0x40106bf4: uart_interrupt_handler(uart_*) at C:\Users\jwa\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.2.0\cores\esp8266/HardwareSerial.cpp line 541
0x40207fbe: HardwareSerial::_rx_complete_irq(char) at C:\Users\jwa\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.2.0\cores\esp8266/HardwareSerial.cpp line 541
0x40106c28: uart_interrupt_handler(uart_*) at C:\Users\jwa\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.2.0\cores\esp8266/HardwareSerial.cpp line 541
0x4020dba0: pm_get_sleep_type at ?? line ?
0x40105a12: spi_flash_read at ?? line ?
0x40106f90: pvPortZalloc at C:\Users\jwa\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.2.0\cores\esp8266/heap.c line 33
0x4020d671: pm_set_sleep_time at ?? line ?
0x4020db06: pm_get_sleep_type at ?? line ?
0x4020a48c: pp_noise_test at ?? line ?
0x4020dbb3: pm_get_sleep_type at ?? line ?
0x4021f289: ets_timer_handler_isr at ?? line ?
0x4021f2ce: ets_timer_handler_isr at ?? line ?

Could this have anything to do with the fact that I've replaced the HardwareSerial?

[igrr]

Could you please look up what code is at 0x40209068?

[JonesWest]

bool HardwareSerial::isRxEnabled(void) {
    if(_uart == 0)    <---------- line 550
        return false;
    return _uart->rxEnabled;
}

[igrr]

The code which gets called from ISR has to be marked with ICACHE_RAM_ATTR. I'm not sure how it worked at 2.0.0 times, we had different linker scripts, maybe that wasn't an issue back then.

So the fix would be to add ICACHE_RAM_ATTR to all stuff in HardwareSerial.cpp which gets called from ISR. Not that many functions if i remember correctly.

[JonesWest]

I ended up preceding every function in HardwareSerial.cpp with ICACHE_RAM_ATTR, and it actually did the job!! PERFECT! thanks for that.

Though, I just discovered that it only works when the wifi mode is WiFi.mode(WIFI_AP_STA);, I am using it as station connected to access point, but as soon as I change the mode to WiFi.mode(WIFI_STA) I get the following stack trace:

0x40207a30: cbuf::write(char) at C:\Users\jwa\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.2.0\cores\esp8266/cbuf.h line 86
0x401076ad: uart_interrupt_handler(uart_*) at C:\Users\jwa\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.2.0\cores\esp8266/HardwareSerial.cpp line 124
0x40210a3a: ieee80211_ht_updateparams at ?? line ?
0x40222420: dhcp_fine_tmr at ?? line ?
0x40107680: uart_interrupt_handler(uart_*) at C:\Users\jwa\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.2.0\cores\esp8266/HardwareSerial.cpp line 118
0x4010760e: HardwareSerial::_rx_complete_irq(char) at C:\Users\jwa\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.2.0\cores\esp8266/HardwareSerial.cpp line 643
0x40105629: ets_timer_disarm at ?? line ?
0x4020d5f4: pm_get_sleep_type at ?? line ?
0x40105a12: spi_flash_read at ?? line ?
0x40107a00: pvPortZalloc at C:\Users\jwa\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.2.0\cores\esp8266/heap.c line 33
0x4020d0c5: pm_set_sleep_time at ?? line ?
0x4020d55a: pm_get_sleep_type at ?? line ?
0x4022bbd8: sys_check_timeouts at ?? line ?
0x4020d607: pm_get_sleep_type at ?? line ?
0x4021ecdd: ets_timer_handler_isr at ?? line ?
0x4021ed22: ets_timer_handler_isr at ?? line ?

I've tried to use the ICACHE_RAM_ATTR macro here as well:

        size_t ICACHE_RAM_ATTR write(char c) {   <----- line 86
            if(room() == 0) return 0;

            *_end = c;
            if(++_end == _bufend) _end = _buf;
            return 1;
        }

But I get a compiler error:

C:\Users\user\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.2.0\cores\esp8266\cbuf.h:86:32: error: size_t cbuf::write(char) causes a section type conflict with void uart_ignore_char(char)

         size_t ICACHE_RAM_ATTR write(char c) {

                                ^

C:\Users\jwa\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.2.0\cores\esp8266\HardwareSerial.cpp:400:22: note: 'void uart_ignore_char(char)' was declared here

 void ICACHE_RAM_ATTR uart_ignore_char(char c) {

                      ^

exit status 1
Error compiling.

Any ideas? :)

[devyte]

@JonasWestAlro is this issue still valid with latest git?

[earlephilhower]

We're now using the HW FIFO + a SW buffer in the latest UART code, a significant improvement on the original SW-only handling.