Serial Monitor crashes when receiving invalid UTF-8

[hoekma]

Similar to issue #1293

When connecting to the serial port on my macBook Pro using both ports /dev/cu.SLAB_USBtoUART and /dev/cu.usbserial-0001 I do not see any serial output. When connecting, I see the follwing:

[Starting] Opening the serial port - /dev/cu.usbserial-0001
or
[Starting] Opening the serial port - /dev/cu.SLAB_USBtoUART

Then no serial output. Serial output does work when connected through the Arduino app. Both are connecting at 115200.

Note: I CAN see the led on my device flash off and then back on when connecting, suggesting it is communicating with the device via serial, however my Serial.println() and Serial.print() output is not displaying in the output.

vsCode: 1.59.1
Arduino plugin: v0.0.4 Preview
Device: ESP32

[FuryMartin]

Same Problem with v0.4.4, vscode 1.60.0

[Ezekiel-DA]

I was having the same issue, until I noticed this in the documentation for the extension settings:

arduino.defaultBaudRate | Default baud rate for the serial port monitor. The default value is 115200.

Changing my Serial.begin() to match the 115200 default appears to make the Serial Console work.

Interestingly, changing the defaultBaudRate setting to match a baud rate other than 115200 set with Serial.begin() does not seem to work for me. Perhaps there is a bug with this setting and it is being ignored?

[cexshun]

Same issue here even using the default 115200 baud rate.

[Starting] Opening the serial port - COM4

No other output. I hit reset on the esp8266, and no output. With the monitor still running, I open the Arduino IDE and the serial monitor in the IDE connects and works fine. If vscode were actually connecting to the COM port, the IDE should be giving an error that the port is already in use.

[hoekma]

@adiazulay FYI - I created this issue based on your request in the issue linked in my OP. Because of this issue, I have haven't used VSCode for my Arduino development for 3 months now. However, I'd love to get back to using it again. Can you comment?

FYI my environment is now:

vsCode: 1.62.3
Arduino plugin: v0.4.8
Device: ESP32

[adiazulay]

@hoekma Thanks for reaching out, I'm actually no longer the maintainer of this repo, but @benmcmorran might be able to help you.

[Rembunator]

@benmcmorran

I see the same issue happening on Linux. The serial monitor connects, but shows no data from the serial port.
My device is a Wemos D1 mini (esp8266).

However, when I close the serial monitor, reset my serial device and then reopen serial monitor, I do get output.
I guess normally the device is reset just after the monitor has started, causing it to crash.

Looking at this code I found the call used to start the monitor:
this._child = child_process_1.spawn(SerialPortCtrl._serialCliPath, ["open", this._currentPort, "-b", this._currentBaudRate.toString(), "--json"]);

When I run this in a linux console like so:
~/.vscode/extensions/vsciot-vscode.vscode-arduino-0.4.8/out/serial-monitor-cli/main open /dev/ttyUSB0 -b 115200 --json

It works, but crashes when I reset the device:

Exception in thread device_connection:
Traceback (most recent call last):  
  File "threading.py", line 932, in _bootstrap_inner  
  File "threading.py", line 870, in run
  File "dataHandler.py", line 87, in read_from_device
  File "dataHandler.py", line 64, in print_to_console_json
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9e in position 3: invalid start byte

So, at least on my system, this looks like a bug or rather a missing recover feature in serial-monitor-cli.

I tried serial-monitor-cli/main -h which just shows:

usage: main [-h] {list-ports,open} ...

positional arguments:
  {list-ports,open}
    list-ports       List all available serial ports
    open             Open a new serial connection

optional arguments:
  -h, --help         show this help message and exit

[Rembunator]

• I replaced the content of ~/.vscode/extensions/vsciot-vscode.vscode-arduino-0.4.8/out/serial-monitor-cli/ with the python code from the serial-monitor-cli project.
• Changed main.py to main
• Added #!/usr/bin/python to the beginning of this file
• made the file executable
• Modified the print_to_console_json function in dataHandler.py to handle decoding exceptions caused by a broken connection:

def print_to_console_json(data: bytes or str):
    if type(data) is bytes:
        try:
            data = data.decode('utf-8').strip()
        except UnicodeDecodeError:
            data = "< Unable to decode data >"
    if data:
        output = {'payload': data}
        json_output = json.dumps(output)
        encoded_output = json_output.encode('utf-8')
        stdout.buffer.write(encoded_output)
        stdout.buffer.flush()
    elif data is None:
        return

This works for me now. I just get a notice < Unable to decode data > when the device resets.

[Rembunator]

I just realized: When using the Arduino IDE, the serial monitor also has a pause (or silent reconnect), because serial output at the start of the device program is never received, unless a delay is placed in the device program.

[hoekma]

@benmcmorran Any thoughts on getting the serial monitor working for VSCode for mac?

[benmcmorran]

@hoekma I'm on vacation until the end of the year, but I should be able to take a look at this in January. @aleun helped with the initial move to serial-monitor-cli and might also have ideas. Finally, if this worked for you on an older version of the extension, you can try downloading an older release from https://github.com/microsoft/vscode-arduino/releases and see if that works as a workaround until we can fix the extension.

…

On Thu, Dec 9, 2021 at 7:44 PM hoekma ***@***.***> wrote: @benmcmorran <https://github.com/benmcmorran> Any thoughts on getting the serial monitor working for VSCode for mac? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1338 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABVINJHHHD7LLBQNU4NCMDLUQFZTBANCNFSM5DJPMVGA> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[Joneffxn]

I'm trying to get an arduino uno to sending data (preferably in json-format) to my laptop (running ubuntu) to plot the data with a python script.

However I found the issue that when uploading with vscode there will be no output on neither the extension's monitor nor the serial monitor of the arduino IDE.
When uploading the same code with the IDE there will certainly be an output in the IDE's monitor, but not always on the vscode monitor. Sometimes there will be no output at all, other times there will be between one or two to about ten lines of 'old' output (which is from before the restart (which is caused by opening the monitor)).
But then on closing the monitor there will sometimes appear the output that has seemingly been piling up (between the restart and the closing of the monitor). Like - you open the monitor and maybe you get a few old lines - and when you close - maybe the output you were expecting just comes late (If you looked at it after the whole process was done, you couldn't even tell that the output was too late - everything appears in the right order and without any warnings, just too late.) or (most times) it won't come at all.

As it appears to me, the extension has some problem in either compiling or uploading 'Serial.print()'- statements, as well as some (maybe timing-related) problem in actually displaying incoming data, presumably because of the restarts (that come with starting the monitor) causing the monitor to crash (as @Rembunator found out). (So I think it is timing-based because very few times it inexplicably does work, while most times it doesn't.)

I'm currently using:
Ubuntu 20.04.3 LTS,
VSCode:

• Version: 1.63.0
• Commit: 7db1a2b88f7557e0a43fec75b6ba7e50b3e9f77e
• Date: 2021-12-07T05:16:45.441Z
• Electron: 13.5.2
• Chromium: 91.0.4472.164
• Node.js: 14.16.0
• V8: 9.1.269.39-electron.0
• OS: Linux x64 5.4.0-91-generic snap
  Extension-version: v0.4.8

Edit on 12/15/2021:
There actually was no issue with uploading and the 'Serial.print()'-statements, I was just uploading another file, overlooking it was the wrong one.

[Rembunator]

@Joneffxn
I first thought this was happening:
The monitor crashes on a decoding exception when the connection is reset after receiving the first byte of the UTF-8 character.
So the to be decoded string is incomplete half of the time.

But I took a closer look at the data that was being decoded. After every reset, a lot of gibberish (initialization?) comes in first like this:
b"rd\x00l\xdc\x9f|\x00\x8c$\xe0|\x03\x0c\x04\x0c\x84\x0cd\xec\x04c|\x8f\x83\x03\xec\x1b\x93{\x93c\x8c\x04c\xc4\xf3o'\x9fdo'\x9c\xe3\xec\x0c#\x1cx\xe4\x87ds$s$p\xf3o\xe0\x18\x03\x0c\x04\x83\x04d\x04\x84\x9c\x04\x0c\x04c\x04o\xe3|\x03\xe4\x0c\x8e\x87\x04c\x84\xf2ng\xe7\x00l\x84\x87d`\x03\x98\x1b\x1b'o\x04\x87\x0fl\x07\x03os\x8f\xdb\x9bo\x04\x0c{;\x9b`\x03p\xf2n\xe0\x18\x03\x0c\x04s\xc4\xdc\x9c\xdc\x1c\x80\x8c\x03\x0cb\x0co\xe2<\x03\x0c\x8f\x0cc\x8c\xfb'o\xef\x00\x04\xc7l \x03\x98\x13\x1b'o\x04\x87\x0fl\x07\x03or\x8e\xdb\x93o\x04\x0c{\x12d`\x03\x0fr\x9b\x92'\x0c\x04r\x1bl`\x03\xdc\x03\x0c\x04l\x8f8\xe3\xe4\x0c$`\x03\xfc\x83n\xdc\x07l

It seems to always end with a sequence starting with \x03\xfc. I don't know what that is, I'm not that fluent in gibberish.

BTW I tested resetting the device about 30 times in a row and the decoding exception was raised every time.

[benmcmorran]

We shipped v0.4.9 of the extension yesterday. It includes #1412 to throttle the rate at which the extension tries to write to the serial monitor, which helps avoid the situation where no data appears in the serial monitor until the connection is closed. Can you try updating to v0.4.9 and let me know if you're still seeing issues?

[netcrawler00]

Same issue of silent serial monitor with :
Version : 1.63.2 (user setup)
Validation : 899d46d82c4c95423fb7e10e68eba52050e30ba3
Date : 2021-12-15T09:40:02.816Z
Electron : 13.5.2
Chromium: 91.0.4472.164
Node.js : 14.16.0
V8 : 9.1.269.39-electron.0
Operating System : Windows_NT x64 10.0.19042

and extension v0.4.10

[benmcmorran]

Thanks all for reports. This certainly looks like an issue with serial monitor assuming a UTF-8 encoding. Looping in @gcampbell-msft who has been working on other serial monitor changes.

[benmcmorran]

We expect that microsoft/serial-monitor-cli#12 will fix this issue.

[milanbx]

Hi, my VS Code 1.64.1 with Arduino extension v0.4.10 experienced the same behaviour: When clicked to open the serial monitor and reseted teh device (ESP8266 in my case), no output i shown. I have increased the Developer Log Level to 'trace' and ther is en exception related to this extension in Log (Window)Output - see attached log file renderer1.log.

[benmcmorran]

@milanbx based on the stack trace you provided the issue you're seeing isn't related to UTF-8 encoding. I'm moving your report to a new issue.

[benmcmorran]

This issue has been fixed in the latest release of this extension, which is available in the VS Code extension marketplace.

[netcrawler00]

Hello
Thank you for the update.
Unfortunately, I upgraded to extension 0.4.11 and vscode 1.64.2, and the terminal still doesn't show anything.
Can you please help ?

My bad, the wrong baud rate was selected at the bottom of the window in VS Code. Working fine now. Sorry for that.
Good job team, thank you so much

[hoekma]

@benmcmorran Thank you for working on this. I am trying this now on versions 0.4.11 and 0.4.0 with no luck. Can you advise which version you tested this on?

[milanbx]

@netcrawler00, @benmcmorran I have tried this again on Arduino version 1.64.2 and extension 0.4.11 and all works fine, port selection and also serial output. As for the serial output, I have added following delay to my sketch to give the COM some time to start properly:

  Serial.begin(115200);
  delay(700);
  Serial.println("\n-------------------------------");

1000 ms was too much, 600 ms too little to display the first init-like output line.

[netcrawler00]

I edited my last comment, the mistake was on my side. Bug closed for me.
Thank you all!