ESP32-S3 USB CDC doesn't work on every computer, "serial port is closed" errors

[marcocipriani01]

Board

ESP32-S3-WROOM-1U

Device Description

Self-powered ESP32-S3-WROOM-1U custom board with USB connected on pins IO19 and IO20. The power comes from a barrel jack and a 3.3V LDO regulator.
There is a USB Type-C connector and a TVS diode array (designed for USB 2.0 usage). VUSB doesn't go anywhere except for the TVS diode.

Hardware Configuration

Version

v2.0.15

IDE Name

Arduino IDE

Operating System

Windows 10, 11

Flash frequency

QIO 80MHz

PSRAM enabled

no

Upload speed

921600, upload via native USB

Arduino settings

Just USB CDC on boot:

Description

In my company, we developed a product based on the ESP32-S3-WROOM-1U that uses USB CDC on a Type-C connector to communicate with the computer. During prototyping we didn't have major issues with USB, but now we're testing the first production batch of boards on a large sample of computers and in various conditions. Unfortunately, about ¼ of the PCs in our production test have trouble connecting to the ESP32-S3, and the issue is unclear to us. These are some of the behaviors we observed:

• On a few PCs, complete inability to connect to the serial port: every software we use to connect to the ESP32 says either "the serial port is closed" or simply "failed to connect". This includes the Arduino Serial Monitor, Visual Studio Code's Serial Monitor extension, our custom C# software and more.
  • No other info is provided by these software, and we're unable to debug the situation since we can't even connect once.
  • Putting the board in bootloader mode using GPIO0 correctly enters bootloader and these same PCs can successfully upload new sketches to the ESP32 using Arduino IDE.
• On other PCs, including my workstation, the issue only happens sporadically (probably ⅓ of the times). The rest of the times, everything works fine. Downgrading to the old arduino-esp32 core 2.0.12 seems to decrease the frequency of the issue on the affected PCs, but doesn't solve it.
• The rest of computers have absolutely no trouble connecting to the ESP32-S3 board under any circumstance. We can use our custom C# software to do all the things we need and USB never stops working,

While testing, I also discovered that opening and closing the serial monitor rapidly a few times (open-close-open-close-...) makes the USB CDC stop working. For example, I created a sketch that prints "Hello" and does one Serial.read() every second in the loop() function. After opening and closing the serial port a few times, I no longer see "Hello", I receive nothing.

The issues happens on all the 30 boards in our first production batch. They've been professionally reflow soldered by another company. I manually inspected all of them under a microscope for soldering defects around the USB Type-C connector and the ESP32 module. It is a 6 layer board and the USB differential pair is length-matched and impedance-controlled to 90Ω according to USB standard.

Sketch

Any Arduino sketch that uses Serial.print(), Serial.println(), Serial.read().

Debug Message

"The serial port is closed" on some software.

Other Steps to Reproduce

Unclear. Depends on the PC where testing.

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[lbernstone]

Have you tested v3.0.0 and IDF to see if the behavior is the same?

[marcocipriani01]

Arduino-ESP32 v3.0.0 breaks our production firmware, I will try to make it work on v3 and let you know.

[Jason2866]

You can try latest version from branch https://github.com/espressif/arduino-esp32/tree/release/v2.x
HWCDC fixes/changes have been backported

[lbernstone]

Arduino-ESP32 v3.0.0 breaks our production firmware, I will try to make it work on v3 and let you know.

Well, then also check that a minimal sketch with the same hardware settings causes the issue. This seems like a core issue, not something in your code, but it will be useful to narrow that down.

[marcocipriani01]

I was able to upgrade to Arduino-ESP32 v3.0.0-RC1 and, while the situation is better, I'm still having a lot of trouble with USB on affected computers. In particular, on my current workstation, with Arduino-ESP32 v3, the following happens:

• Serial.begin() becomes stuck sometimes, i.e. blocks the microprocessor from continuing.
  • Since CDC on boot is enabled, Serial.begin() isn't actually needed, so I commented the line.
• The USB port seems to be in fact more reliable, improvements are clearly showing:
  • I no longer get the occasional error during an Arduino sketch upload that I used to get before (rare, but annoying). EDIT: see comment below
  • No more "the port is closed" errors
• Right after a new Arduino sketch upload, the USB CDC serial port works perfectly from every software, but just once:
  • Any further attempt to connect to the serial port does succeed (no errors), but no data is sent or received over the serial port. The ESP32 doesn't read or send anything.
  • This is especially bad if the serial monitor is set to send DTR and RTS signals. Otherwise, there is a small chance of the ESP32 working for another one or two connection attempts.

Here is an example screenshoot, taken from the Visual Studio Code Serial Monitor extension (by Microsoft). The ESP32 is programmed to echo any received string, which it doesn't after the first connection:

When this issue happens, Arduino IDE's serial monitor shows this, immediately after connection (and the ESP32 doesn't echo the strings, just like in VS Code):

Any suggestion or things I can try?

[marcocipriani01]

Now, for some reason, I can't even upload without setting the board in bootloader mode, and the echo sketch works only sometimes even after upload. COM7 exists in Device Manager and the Arduino Serial Monitor can connect to it.

[Jason2866]

Do you can provide a small example sketch where the problem arrives?

[marcocipriani01]

It's the one in the screenshoot:

void setup() {
  Serial.println("Boot!");
  // Comment or not?
  //Serial.begin(115200);
  Serial.print("Serial.begin() done.");
}

void loop() {
  if (Serial.available() > 0)
    Serial.print((char) Serial.read());
}

[Jason2866]

Setup is not as recommended, but this should not the reason.

void setup() {
  Serial.begin(115200);  
  Serial.println("Boot!");
  Serial.print("Serial.begin() done.");
}

void loop() {
  if (Serial.available() > 0)
    Serial.print((char) Serial.read());
}

[marcocipriani01]

Yes I know, but it doesn't matter. CDC on boot is enabled so Serial.begin() should do nothing, plus I even commented that line. Even if I leave setup() empty, the result is the same.
If Serial.begin() really does nothing, why did it halt my ESP32-S3 a few times?

[Jason2866]

Can you try this example https://github.com/espressif/arduino-esp32/blob/master/libraries/ESP32/examples/HWCDC_Events/HWCDC_Events.ino

It does as intended with my S3 CDC device. Tested with Mac. It could be a problem in combination with Windows.

[marcocipriani01]

Sure, I'll try it Monday when I go back to the office. May I ask what is the difference between using Serial (what I'm doing) and HWCDC HWCDCSerial (the example you linked)?

[Jason2866]

The way to use the HWCDC port as serial UART.

[marcocipriani01]

Yes I know, but in the case of the ESP32-S3, Serial is already the native USB CDC port (while Serial0 is UART). What is the HWCDCSerial object then, just an alias for Serial?

[lbernstone]

Yes I know, but in the case of the ESP32-S3, Serial is already the native USB CDC port (while Serial0 is UART). What is the HWCDCSerial object then, just an alias for Serial?

Other way around. HWCDCSerial is the "physical" object. Serial is aliased to that. This is why you don't need to initialize it. HWCDC initializes it b/c you have chosen that option in the menu. As far as why the port doesn't exist, this happens when the HWCDC reinitializes quickly. The PC makes another connection, but the previous one hasn't been torn down yet, so it grabs the next handle. You need to be patient with it. It is not great as a debugging tool- HWCDC is more for ease of programming production devices.

[SuGlider]

• This is especially bad if the serial monitor is set to send DTR and RTS signals. Otherwise, there is a small chance of the ESP32 working for another one or two connection attempts.

@marcocipriani01 - please keep in mind that DTR and RTS signals of the USB CDC port actually control "reset" and "boot mode" signals of the ESP32-S3.

It may be putting the S3 into download mode... "freezing/halting" the S3 firmware execution...
Please read this: #6762 (comment)

[SuGlider]

Yes I know, but it doesn't matter. CDC on boot is enabled so Serial.begin() should do nothing, plus I even commented that line. Even if I leave setup() empty, the result is the same. If Serial.begin() really does nothing, why did it halt my ESP32-S3 a few times?

This is not possible when using HWSerial CDC mode.
There is an issue about it related to USB OTG CDC.
Please read #9532

[SuGlider]

Yes I know, but in the case of the ESP32-S3, Serial is already the native USB CDC port (while Serial0 is UART). What is the HWCDCSerial object then, just an alias for Serial?

In the Arduino Menu for the S3 there are 2 option:

When USB CDC On Boot is disabled, Serial is the UART0 port.
When USB CDC On Boot is enabled, Serial is the USB CDC port.
The USB CDC port can be of 2 exclusive different hardware/software modes: Hardware CDC and JTAG or USB-OTG (tinyUSB)

The Hardware CDC and JTAG creates an object from the HWCDC class (https://github.com/espressif/arduino-esp32/blob/master/cores/esp32/HWCDC.h)
The USB-OTG (tinyUSB) creates an obejct from the USBCDC class (https://github.com/espressif/arduino-esp32/blob/master/cores/esp32/USBCDC.h).

From the comment that says that Serial.begin() doesn't matter, I guess that your are using the USB-OTG (tinyUSB) mode.
Please confirm it.

[SuGlider]

As for a potential question like "what is the difference between HWCDC and USBCDC":

Using HWCDC, the USB port can only be used for CDC/ACM or JTAG port.
This is a HW/Silicon implementation of USB CDC + JTAG.

Using USBCDC, the USB port is a real USB implementation with up to 5 IN/OUT + 1 IN endpoints. It can be used as USB Composite Device and allow the S3 to be USB device using at the same time, for instance, HID + CDC.
It also can work as a USB OTG / Host (using IDF + Arduino - at this time).

[marcocipriani01]

Can you try this example https://github.com/espressif/arduino-esp32/blob/master/libraries/ESP32/examples/HWCDC_Events/HWCDC_Events.ino

Sure, I tried that example and got this result, pretty much identically to the Serial echo sketch I provided:

• I frequently get "Failed to open the serial port COM7" or "the port is busy" when connecting to a serial monitor (VS Code, Arduino, others)
• When the serial monitor connects, I correctly get [xx] connected every second.
• Arduino IDE fails to upload the example sketch via native USB, if that example is running on that board already
  • A fatal error occurred: Could not open COM7, the port doesn't exist while COM7 clearly exists and is not connected to a serial monitor.
  • Setting the board to bootloader mode is needed to upload again.
• I tried modifying the example a little bit to echo received chars, but it doesn't echo anything:

  if (HWCDCSerial) {
    HWCDCSerial.printf("  [%ld] connected\n\r", counter);
      while (HWCDCSerial.available()) {
        HWCDCSerial.write(HWCDCSerial.read());
    }
  }

---

Other way around. HWCDCSerial is the "physical" object. Serial is aliased to that.

If Serial is just the alias of HWCDCSerial, then that doesn't solve my issue anyway. Hardware CDC just doesn't work reliably on every computer. There must be a bug in the Arduino-ESP32 or IDF frameworks.

---

Please keep in mind that DTR and RTS signals of the USB CDC port actually control "reset" and "boot mode" signals of the ESP32-S3.

I have the exact same issues with serial port signals disabled.

---

When USB CDC On Boot is disabled, Serial is the UART0 port.
When USB CDC On Boot is enabled, Serial is the USB CDC port.

Yes I was aware of that.

---

From the comment that says that Serial.begin() doesn't matter, I guess that your are using the USB-OTG (tinyUSB) mode.
Please confirm it.

I am not using USB-OTG (TinyUSB). In my testing initial testing a few months ago, that mode was less reliable than hardware CDC because, obviously, it is a software implementation.
I said that Serial.begin() technically isn't needed for Hadware CDC mode because Serial.print() and Serial.read() work just fine without it. I'm no expert, however, so feel free to correct me on that. Looking at the HWCDC.cpp source, I see some code in begin(), so it must be doing something...

---

If you think could help, I am available to set up TeamViewer or AnyDesk so you can test it yourself on affected computers. We're delaying our product's launch because of this issue so any help is greatly appreciated.

[Jason2866]

We had issues with Windows and the current Arduino HWCDC driver too. We fixed it for our project Tasmota by commenting lines 312 and 313

//deinit(NULL);
//delay(10);  // USB Host has to enumerate it again

Windows seems to have big trouble by doing that.

and inserting after line 334

usb_serial_jtag_ll_clr_intsts_mask(USB_SERIAL_JTAG_LL_INTR_MASK);

so the code block looks like this

// Enable USB pad function
  USB_SERIAL_JTAG.conf0.usb_pad_enable = 1;
  usb_serial_jtag_ll_disable_intr_mask(USB_SERIAL_JTAG_LL_INTR_MASK);
  usb_serial_jtag_ll_clr_intsts_mask(USB_SERIAL_JTAG_LL_INTR_MASK);
  usb_serial_jtag_ll_ena_intr_mask(
    USB_SERIAL_JTAG_INTR_SERIAL_IN_EMPTY | USB_SERIAL_JTAG_INTR_SERIAL_OUT_RECV_PKT | USB_SERIAL_JTAG_INTR_BUS_RESET | USB_SERIAL_JTAG_INTR_SOF
  );

This is not a solution which can be used in general! Code like if(Serial) will not work with this change.

[marcocipriani01]

Good to know I'm not the only one with the issue, I'm trying your workaround. Luckily I don't need if (Serial).

[marcocipriani01]

@Jason2866 your fix appears to be working on my workstation, I will test it on all the affected computers (to be safe) and let you know. Thanks a lot!

@SuGlider now that hardware CDC seems to work, regarding your note:

please keep in mind that DTR and RTS signals of the USB CDC port actually control "reset" and "boot mode" signals of the ESP32-S3.
It may be putting the S3 into download mode... "freezing/halting" the S3 firmware execution...

I can say now it isn't the case. In fact, I get the following in the serial monitor when connecting with serial port signals enabled:

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x15 (USB_UART_CHIP_RESET),boot:0x23 (DOWNLOAD(USB/UART0))
Saved PC:0x40041a76
waiting for download

Which is to be expected when entering bootloader mode. I didn't get this before applying the fix.

[SuGlider]

We had issues with Windows and the current Arduino HWCDC driver too. We fixed it for our project Tasmota by commenting lines 312 and 313

//deinit(NULL);
//delay(10);  // USB Host has to enumerate it again

Windows seems to have big trouble by doing that.

and inserting after line 334

usb_serial_jtag_ll_clr_intsts_mask(USB_SERIAL_JTAG_LL_INTR_MASK);

so the code block looks like this

// Enable USB pad function
  USB_SERIAL_JTAG.conf0.usb_pad_enable = 1;
  usb_serial_jtag_ll_disable_intr_mask(USB_SERIAL_JTAG_LL_INTR_MASK);
  usb_serial_jtag_ll_clr_intsts_mask(USB_SERIAL_JTAG_LL_INTR_MASK);
  usb_serial_jtag_ll_ena_intr_mask(
    USB_SERIAL_JTAG_INTR_SERIAL_IN_EMPTY | USB_SERIAL_JTAG_INTR_SERIAL_OUT_RECV_PKT | USB_SERIAL_JTAG_INTR_BUS_RESET | USB_SERIAL_JTAG_INTR_SOF
  );

This is not a solution which can be used in general! Code like if(Serial) will not work with this change.

Yes... this is a potential problem and conflict... Follow Arduino USB Serial specification (with if (Serial)) and force users to use BOOT button, or let it fail and get esptool.py do its automatic job. I guess that for the C6 and H2 we may have a new alternative as it allow us to detect DTR/RTS signals within HWCDC. But this issue will continue for the C3 and S3 HW USB Serial.

@me-no-dev -- something to discuss for a next meeting.

[marcocipriani01]

I've tested the fix on other affected computers and I can say it's working reliably. My suggestion, since you'll be releasing Arduino-ESP32-v3 with breaking changes, and since this issue only affects the ESP32-S3 and -C3, is to use a pre-processor directive to apply the fix only for affected devices:

#if (!defined(CONFIG_IDF_TARGET_ESP32S3)) && (!defined(CONFIG_IDF_TARGET_ESP32C3))
    deinit(NULL);
    delay(10);  // USB Host has to enumerate it again
#endif

// ... other code

#if defined(CONFIG_IDF_TARGET_ESP32S3) || defined(CONFIG_IDF_TARGET_ESP32C3)
    usb_serial_jtag_ll_clr_intsts_mask(USB_SERIAL_JTAG_LL_INTR_MASK);
#endif

Then you could document somewhere that if (Serial) is deprecated for the -S3 and -C3 MCUs. if (Serial) should then always return true for the -S3 and -C3 to avoid halting people's code when they don't expect it.

[Jason2866]

@marcocipriani01 Tested PR #9628 does work for us.

[lucasssvaz]

Fixed in #9628 Please take a look if this fixes your issues

[marcocipriani01]

Great, as soon as the next release candidate is available I will test it out on affected computers to confirm!

[SuGlider]

@marcocipriani01 - Please check the fix #9660 for Arduino Core 2.0.x

[VojtechBartoska]

Closing this as solved, 2.0.17 was released today.