Serial.Available() updates delayed for hardware serial

[mribble]

Basic Infos

Hardware

Hardware: ESP8266 module
Core Version: 2.4.0

Description

I am running the hardware serial at 74880 baud. Each byte of serial data is 9 bits (8 bits of data and 1 stop bit). This means the fastest I should be able to receive 128 bytes of data is 15 milliseconds. I have logs showing checking Serial.available changes from 0 bytes to 128 bytes in 5ms. So it seems the available function isn't getting updated as often as new bytes of data are arriving.

I took a look at the code and got confused so I'm hoping someone with knowledge of how hardware serial works on esp8266 can help me. It looks like in ESP8266 2.1.0 you switched from the standard interrupt based serial approach to using a hardware buffer that is 128 bytes.

Reading through the serial documentation (http://esp8266.github.io/Arduino/versions/2.3.0/doc/reference.html) it seems you have 128 byte FIFO buffer shared for RX and TX. Then you have an additional 256 bytes for TX and 256 bytes for RX. So this means depending on your combination of RX and TX data the max RX data you might receive before starting to drop bytes is 384, but it is also possible that it could start dropping bytes after 256 if you fill the hardware FIFO with TX data. Is my understanding correct?

While I am not seeing any data dropped, I worried about delays in getting the RX data and I want a way to detect when the buffer overflows. Can someone explain to me when Serial.available() actually is notified that there is data? Would it be reasonable to manually go get the data out of the hardware FIFO during Serial.available() to prevent extra latency and prevent confusion like I'm having?

Also there is no query to see how big the serial buffer is, right? In my code I'm trying to detect if that buffer ever overflows. Previously I thought that limit was 128 bytes, but now I'm not sure what to set it to. I guess the safe value is 256 unless my understanding above is incorrect and then it would be 384. Do I have a better option for detecting RX serial buffer overflow?

[mribble]

I started trying things. I found if I reduce UCFFT from 127 to 7 in this line of uart.c:

USC1(uart->uart_nr) = (127 << UCFFT) | (0x02 << UCTOT) | (1 <<UCTOE );

That works around the problem. So if I understand things that reduces the size of the RX fifo from 128 to 8 bytes. I don't want to do that, but it does help confirm my theory above that the fifo is filling up and the Serial.available call doesn't take into account what's in fifo.

What I would rather do is flush out the hardware fifo during uart_rx_available(), but I haven't figured out how to do that yet.

Is there any document that describes esp8266_peri.h? I'm trying to understand what the units for UCTOT are. We set that to 2, but I don't know what 2 means in this case.

[mribble]

I found a document that gives most of the answers I was looking for here: https://www.espressif.com/sites/default/files/documentation/esp8266-technical_reference_en.pdf

I was able to answer most of my questions above and now understand how uart is working. The reason my use case has problems is because you are only draining the rx fifo when it is full or when a timeout occurs and a uart rx timeout only happens if rx is idle, which never happens in my case.

I think what you are doing in the interrupt is valid, but the problem is when you peek or read chars from serial there can be a delay. This delay could be quite large. For example if streaming at 9600 baud to the esp8266 data would take 120 ms. If I wanted to construct a bad case I could put a 2ms delay between each byte before transmitting and that would give a worst case of 396 ms.

The fix is to add this code to uart_rx_available() which will cover the all read and peek cases. It also addresses the issue I was seeing with Serial.available() not reporting the correct number of bytes available (or at least having latency in reporting the correct number).

`
ETS_UART_INTR_DISABLE();

while((USS(uart->uart_nr) >> USRXC) & 0x7F){

    uint8_t data = USF(uart->uart_nr);

    size_t nextPos = (uart->rx_buffer->wpos + 1) % uart->rx_buffer->size;

    if(nextPos != uart->rx_buffer->rpos) {

        uart->rx_buffer->buffer[uart->rx_buffer->wpos] = data;

        uart->rx_buffer->wpos = nextPos;

    }

}

ETS_UART_INTR_ENABLE();

`

I will create a merge request with this change.

[devyte]

Closing via #4328 .