Lack of Serial Monitor support for control characters is not documented

[HiTechHiTouch]

It appears that sending a value 0x0d to the IDE Serial Monitor does nothing. Back space prints a blob instead of positioning the next character back 1 position. Line feed not only does a line feed, but also adds a carriage return instead of positioning the next character 1 line below and 1 to the right of the previous character.

As with TTYs from the beginning of time, 0x0d should reposition the "next print" location to the first position of the line. The next character output should appear at the left end (the start) of the same (current) line -- no line feed should be done, and any previous character should be overwritten. Back space 0x08 should move the "next print" position one character to the left (respecting the left margin). Line Feed 0x0a should move the "next print" position to the next line, leaving its distance from the margin unchanged.

Many have tried to repeatedly refill just one line on the serial monitor to avoid data scrolling off the screen. They want to rewrite a constantly updated value to the same location of the screen, rather than producing a flood of lines where updated values quickly push headings and other information off the screen. All report that they can not find a sequence of control characters to accomplish the task.

Further, the IDE documentation does not specify what control characters are honored by the serial monitor. If nothing else, please document what will be the result of this little program snipit:

for (uint8_t i = 0; i < 32; i++) { Serial.write(i) }; // output directed to IDE serial monitor.

[per1234]

As a bug report/feature request, this is a duplicate of #3654, which in turn is a partial duplicate of #346.

I suppose the suggestion to document this limitation of the Serial Monitor could be considered a non-duplicate component. This looks like it would be the appropriate page for such documentation if it was to be added:
https://www.arduino.cc/en/Guide/Environment#toc12
I think we need to balance the benefits of complete documentation against documentation that provides only the most commonly needed information in order to remain beginner friendly. The average Arduino user does not use control characters or even know what they are. However, this sort of issue has been reported here at least 6 times now so it's certainly of concern to some. Whether those users would have benefited from documentation is uncertain. Likely a web search on the topic would bring up one of these issue reports.

[HiTechHiTouch]

A web search did not find either of the reports you reference.

My look in github for open issues against "IDE Serial Monitor" found 40+ hits, none of which appeared to cover the problem.

The web search did find lots of people in places like stack exchange and arduino.cc/forums who tried to use CR, and found it did not work. Their goal was generally that given in the submitted issue: update the screen without causing scrolling.

Replies often told "me too" stories of how nothing could be made to to work, and there was no documentation that said what is expected.

Other issues concerned CR, LF, and NL, generally requiring an explanation of the old manual typewriter and mechanical TTys. This implies that suppressing a fuller explanation in the doc (in order to try and save a novice confusion) may actually be adding to their confusion.

Given that C programmers have been inserting control codes in strings since day one, and Arduino programmers have to eventually learn C (even if it's called C++), I don't think it's wise to ignore (or suppress!) control codes either in the documentation or in the IDE Serial Monitor implementation.

C programmers expect control codes to work. The next examples after the iconic "Hello World!" in some texts explains print() vs println(), and then adds control codes.

In other words, "novice" programmers are going to expect them to work, especially when their study texts talk about them.

Making the IDE Serial Monitor behave as the vast majority of C programmers expect doesn't affect a "true novice" who doesn't use control codes. But not having control codes work is a trap waiting for the very large number of programmers who do use.

/rant on
I understand the "novice" issue. Personally, I am constantly plagued by incomplete and ambiguous documentation on the arduino.cc site. I've stopped looking at the official doc and started browsing the source to get my AVR code questions answered.

But, the IDE is still "black box" (unless one pulls from github), and we still lack an exact definition of how the IDE pre-processes a sketch. Separately compiled units and C++ objects are basically "try it and see what's (not) implemented".

For experienced programmers, there's a great lack of info.

There ALSO has to be a way for experienced programmers to easily use Arduino, just like you want it easy for novice programmers.
/rant

[per1234]

My look in github for open issues against "IDE Serial Monitor" found 40+ hits, none of which appeared to cover the problem.

You need to search open and closed issues. If you had done so you would have found this issue previously reported twice in those results. It's unfortunate that GitHub pre-fills the search box with is:open. The reason you wouldn't find the other 3 in that search is because you poisoned it by adding the unnecessary "IDE" keyword. This is the Arduino IDE repository so it goes without saying that any issue about Serial Monitor is about the Arduino IDE.

we still lack an exact definition of how the IDE pre-processes a sketch.

See https://github.com/arduino/Arduino/wiki/Build-Process#pre-processing
If you find that lacking then feel free to improve it or open a separate issue here.

For experienced programmers, there's a great lack of info.

Generally this repository's wiki pages:
https://github.com/arduino/Arduino/wiki
are used for the advanced documentation and the documentation on arduino.cc is written for the average user. That's not to say there isn't room for improvement in much of the documentation and I'm sure suggestions are welcome. You just have to keep in mind the intended audience.

Making the IDE Serial Monitor behave as the vast majority of C programmers expect doesn't affect a "true novice" who doesn't use control codes. But not having control codes work is a trap waiting for the very large number of programmers who do use.

I agree that it would be nice to have support for all the control characters in the Serial Monitor. However, there are other issue reports where that has already been requested. If this issue is valid at all it's as a documentation suggestion so I'm asking that any further discussion here be limited to that topic. As was already mentioned in the other issues, experienced programmers always have the option of using one of the excellent free terminal applications that do support the control characters. Also, it was stated that a pull request to add control character support to the Serial Monitor would be welcome and I'm sure that offer still stands.

[matthijskooijman]

Turning back to documentation: I think a simple remark that control characters are not supported and if you need them, you could use an external editor would be sufficient. Anyone want to suggest an exact wording for that?

[SimonePDA]

Added a note on the Arduino Environment page about Serial Monitor limitations.

Thanks