Issues with Print::print() and partial writes

[cmaglie]

quoting @bperrybap here
http://forum.arduino.cc/index.php?topic=357312.msg2481770#msg2481770

While there was an issue in the i2c library that was triggered by this new Print
class code, there is a still a fundamental issue with new new Print class code.
The issue is that the Print class code is actually composed of two parts.
The low level functions like read()/write() and the higher level output formatting
functions like print(). And both layers support partial/incomplete i/o.
(outputting less than the maximum asked to output)
I believe that if print() is allowed to support partial/incomplete i/o, that we will be
fighting related issues for years.

What seems to have happened is that in 1.6.6. write() was updated to properly
support incomplete i/o and print() is now seeing issues because it doesn't
support incomplete i/o.
(An error is essentially a single type of incomplete i/o)

Yes it was a coding error in a library in this case, but it brings up a larger issue
related to partial/incomplete i/o and the print() functions.

Currently, I don't believe that the write() API is mandated to always swallow all
the data provided unless there is an error.

So the issues becomes, how to deal with print() when write() does
incomplete/partial i/o?

Looking forward, unless the write() API mandates blocking when low level 
buffers are full or updating the Print class print() functions to ensure blocking, 
when write() returns after incomplete i/o, I think we will see more issues in 
the future.

While it is ok for functions like write() to support and potentially even require 
supporting incomplete i/o, it is not appropriate for functions like print() to force
this same requirement back up to its callers.
This is because callers of write() know the full contents of what is being output.
The callers of print() do not know the full contents of what is being output since
print() is performing output formatting.
In other words suppose I do something like:

foo.print(bar);

and the variable bar is an big integer number and is is 1234567

Now suppose that the low level i/o library buffer is full and rather than block
(spin) waiting for enough room to take all the requested output,  decides to 
take only what it can.
Suppose that was only 4 bytes.
So the low level i/o library write() will return 4 and then foo.print() will now 
return 4. What the heck is a sketch or any other potentially caller of print() 
code, supposed to do with that?
It has no way of resuming the output nor does it even know how many total 
output characters there were.

The point is that if the API is going to support incomplete i/o, it MUST have 
a option to enable/disable this mechanism for functions like print() to not 
return until all output characters have been handed off.
(The only exception is a fatal error where the i/o would never complete)
I argued this point (enable/disable incomplete/async i/o) a year or so ago 
when we were looking at and arguing over the buffer available function calls.

There must be a way to allow users of the Print class, specifically the print()
functions to have a mechanism to be assured that when the print() function 
returns that all the characters have been handed off.
This is an absolute requirement since only print() knows what characters 
are to be output when formatting is done. The caller has no idea.

It is unacceptable to require all users of the print() calls to have to support 
incomplete i/o. And even if they do support incomplete i/o, there is no way 
to complete the i/o for output that was formatted by print() like number output.

Suppose somebody writes an i/o library that decides to do very limited 
output buffering and simply returns 0 when the output buffer is full vs 
blocking? This is allowed by the API but a sketch would never be able to 
print numbers using print() calls when the numbers were more digits than 
output buffer could hold, even if the sketch code add in all the complexities 
to support incomplete i/o for print() functions.

I'm strongly suggesting that print() use blocking i/o.
If non blocking print() i/o behavior is supported by print() (which I don't even 
think is possible since print() does output formatting), then there needs to be 
an API to enable incomplete i/o for an object (disabled by default) so that 
sketches can get the existing blocking behavior by default which preserves
backward compatibility and keeps user sketches simple.
But I'd still recommend that print() be kept simple and always be blocking 
since there is no real technical way to support incomplete i/o with print() 
functions. Return on errors, yes, but not incomplete i/o.

With regard to errors vs partial i/o. That brings up another API issue.
The write() API should be able to distinguish between errors and incomplete 
i/o. for example, if the hardware is broken and it can't transmit, how can that
be indicated?
Currently, if you return zero, then that could mean the low level buffers are 
full (incomplete i/o) or it could mean an actual error.
A negative value would be better to indicate an error.

[cmaglie]

Related issues:
arduino/Arduino#3614
arduino/Arduino#3651

/cc @facchinm @matthijskooijman @stickbreaker

[cmaglie]

AFAIR in Arduino ecosystem write has always been blocking, so (at least theoretically) no partial writes are allowed. This means that write should return something less than size only if an error occurs.

I see your point that the current API may be interpreted as allowing partial writes, and probably some action should be taken quickly to discourage this usage.

Maybe we can reinforce this fact by better commenting the function?

[bperrybap]

If write() and all its forms are always blocking and everyone is happy with that, then simply documenting it would solve the issue.

Once write() becomes non blocking things start to get a bit more complicated particularly if there is not a way to configure it to be blocking.

I had always assumed that write() was not required to be blocking and that is why I was concerned.

[bperrybap]

In further thinking and reflecting I think the issue needs a better solution since it can't fully be resolved through documentation - not for the print() functions.
The real issue goes back to the last minute changes that were thrown into the 1.0 release.
Part of those last minute changes was that ALL the Print class functions shifted from void functions to functions that returned a size_t
The real issue here is that it was not properly spec'd to use negative values like -1 to indicate errors.
(Most *nix i/o APIs - which is what read()/write() APIs in all modern OS's are based on, work this way)
Currently, the Print class return value for all the functions is a byte count but there is no way to indicate an error other than for the caller to compare the returned byte count to an expected byte count - Which is pretty goofy and doesn't work for all the Print class functions , the print() functions being the ones with an issue since the caller doesn't know how many characters are supposed to be output.

If the write() functions are not allowed to do partial i/o, what is the value of returning the number of bytes written?
To me the only reason to return a byte count is to support partial i/o, otherwise why not just use a boolean to indicate success/fail?

The issue for print() is that the way the print() code is currently working is that even if write() is not allowed to do partial i/o but must return the number of bytes processed, and print() simply returns the number of characters successfully written, that there is no way to tell that a print() call failed or wasn't fully processed because print() simply passes the amount of successfully written characters. If an error does not occur on the very first character, then the caller has no idea that there was an error since the caller does not know the number of characters that should be output.
For exmaple if I call print() with something like

print(foo);

and foo is an integer and the write() function failed mid way through the output of the characters of the integer, then print() simply returns the number of characters that were written successfully. The caller of print() has no idea that there was an error since print() returned a non zero value but the caller has no idea how many actual characters were supposed to be output.
Example: suppose foo above was 1234567 and during the processing the write() fails outputting the character '4', write would return zero for that character, but print() would return 3

The caller has no way of knowing that all the characters were not written.
print() could know because it can know how many characters should be output but there is no way for print() to tell the caller.

The print() return value is not really a new issue but is something that has been lingering for quite some time.

Here is a nearly fully backward compatible solution that also allows partial writes() in the future:
print() functions return non-zero if all characters went out and zero if there was an issue/error.
The return value is not an output character count but rather a status or even a boolean - The current character count returned is not all the useful.
This would be backward compatible in the sense that a zero return value before and after this change would mean an output error occurred.
Using this true/false or zero/non-zero return status would also allow partial write() i/o in the future.
To support this, all the print() functions would need be updated to return a zero if all the characters were not processed by write() rather to just blindly return the write() return value.
All the print() functions would have to figure out if all characters were successful written and return the proper status.
Sketches could then look at the return value of print() and if zero an i/o error occurred and if non-zero all characters were successfully written by write().
If write() partial i/o is added/allowed in the future, the print() functions could be updated to support/deal with it and no sketches would need to be changed since the print() return status and behavior would be maintained.

I'm curious about other peoples thoughts on partial i/o and then how to indicate errors vs partial i/o and then how to handle return errors from print() functions.

[matthijskooijman]

If the write() functions are not allowed to do partial i/o, what is the value of returning the number of bytes written?

One possible usecase, which I've used in the past, is for alignment, when printing an arbitrary value and you need to pad it with spaces up to a certain amount of characters.

(No time for going over the rest of your post(s), though)

[bperrybap]

matthijskooijman, I'm not following your comment.
I don't understand how this non partial i/o behavior could help with padding/alignment.
Just for clarification, partial i/o is when you tell write() to process a number of bytes and it decides to take less than the total when there is no error condition, but tells you how many it processed.
For example, you tell write() to process 6 bytes and it returns 3 indicating that anything beyond the 3rd byte has not been processed.

When disallowing write() from doing partial i/o,
which requires the write() function to block/wait if there are buffer room issues and eventually return a count which is exactly equal to the number of bytes you told it, how does that help out in any way with respect to padding and alignment?
The caller knows the exact size of what was handed to write() and write() is never going to give you any more information than you already had/knew.

I understand the value of doing partial i/o but as I mentioned above, I don't understand the value of write() being required to return the number of bytes written when partial i/o is not allowed.
i.e. write() always returning the same number of bytes you handed it, unless there is an error.
What value is it to just get back the same value you gave write()?
Normally, write() functions don't work that way as they support partial i/o and use negative values to represent errors.

[matthijskooijman]

I don't understand how this non partial i/o behavior could help with padding/alignment.

Partial writes don't, but returning the number of bytes written helps with padding alignment. You said you didn't know why the number of bytes was returned without supporting partial writes, so I gave a usecase for the return value that did not need partial writes.

Now I realize I that this really only applies to print, since write will (should?) always have a predictable return value (as you also indicated in your last comment). But in your proposal, you also propose to change the meaning of print return values only indicate failure/success, which would break my use case.

Reading your comment, I think I agree - the return value from write doesn't really mean much (other than zero to indicate errors, I guess). If we were designing the Print API, this would probably mean that the return value could be dropped or change to a boolean, but now that we have it, changing it would cause more compatibility issues than it would solve, I guess.

A slightly tuned down version of your proposal could be:

• print() / write() returns the number of bytes written of no error occurred (same as now)
• print() / write() returns 0 when an error occurred, even when some bytes were successfully written

This allows a caller to detect errors, even when it cannot with the current code. On the other hand, this does throw away other information (namely the number of bytes written before an error occurred). I'm inclined to think that this information is less valuable than knowing if an error occurred, but there might be usecases that need the number of bytes written before the error (I can't think of any just now).

[bperrybap]

matthijskooijman, I still don't understand your use case for print() so it is hard to respond to your comments.

I still don't follow how returning partial write information from print() helps with padding and alignment.
When doing padding and alignment the padding and alignment spacing (space characters) is not all on the end. If it was, there would be no need for alignment padding characters.

For example, suppose there is a number and the number should be right justified within a character output area/field.
How is partial writes being supported by the print() function going to help know how many spaces need to be output prior to sending the number to print()?
While the caller of print() may know the field width, the caller of print() has no idea how many digits will be output by the print formatting in print()

Or in another case where you send a string to print() and you want space padding between words to be done to ensure that words don't wrap across a line. If the device returns an "error" or simply a count that is lower than the total output size requested when attempting to write beyond the end of the line and then print() returns the number of characters written on the line, how does that help with the padding to prevent word wrap. The partial word up the wrap point has already been output.

Can you be very specific on what you mean by alignment/spacing and provide an exact example of how print() supporting fractional output helps?

In terms of return values, what I mentioned earlier was that the return value of print() is not really tied to the return value of write(). print() could be smarter and treat it differently since it knows if all the characters were successfully output and could return a status based on that information, and that is what I was proposing.

The real issue is that within the current write() API definition there is no documented way to distinguish between a partial write and an error.
Technically the current write() API does not define how to communicate an error, since it defines the return value as the number of characters written and is silent on how to handle errors.

What you have proposed is the same as what I proposed when partial writes were not allowed and a non-zero return for print() was used vs a boolean.
That was one of the options I proposed earlier depending on how partial writes were handled by write() - in your example they are not allowed.

If partial writes without an error are going to be allowed by write() - then the write() API must be clear on how to support that and clearly indicate the difference between a partial write and error that occurred mid way through the processing.

My biggest concern and focus is return status for print().
print() can be smart enough to return true/false or 0/non-zero regardless of the write() behavior and how/if partial writes are supported.
And for that reason I'd like to decouple their return values from an API perspective.
i.e. I don't think the return value from print() should have to depend on or be tied to the behavior of write() and write() returning zero vs non-zero on partial output or an error.
While it is easier for the print() code if write() mimics the needed print() return status, it is not required.

The reason I want to separate them is that in my view print() is pretty much a simple "fire and forget" interface that may block but yet you might like to know if all your output was successful.

write() on the other hand is different beast and callers of write() may want and need more sophisticated things and so things like support for partial writes and better status information may be necessary.

The real issue is that when this stuff was tossed in back in 1.0 the the Print class API functions didn't include support for errors.
In real systems things like printf() and write() use negative values to indicate an error.

I think print() is easy.
It can return 0 on errors, and either non zero (a byte count) or true on success and be backward compatible. Regardless of the final write() API and behavior print() could and should have a defined behavior that does not need to change including if write() is ever altered or updated in the future.

write() is the potential challenge, particularly if partial writes are supported as that can affect backward compatibility.

[matthijskooijman]

I still don't follow how returning partial write information from print() helps with padding and alignment.

Like I said, partial write info doesn't help here, but returning the number of characters does. Say I wanted to print:

| 123   |
| 1       |

For the number + aligning spaces part, I can now do:

int n = 6 - Serial.print(some_number);
while (n--) Serial.write(' ');

Without the Serial.print return value, I will have no idea how much characters the printed number has taken, so I can not do alignment (other than predicting how much characters there will be, which is cumbersome, duplicates code, and doesn't generalize well, especially when e.g. Printable objects are involved).

Of course, this stuff only helps with left-aligning things, for right-aligning you will need to predict how much characters something is going to take (which probably needs a "print to string" kind of approach, somewhat voiding the need for the print return value). So perhaps this usecase isn't particularly important (but I do think it is valid).

Other than that, I agree with your ideas. Print should be blocking (no meaningful way to support partial writes), but write might support partials. Print can just retry writing repeatedly as well (until an error occurs / nothing is written), to bridge the gap between these two. I do wonder about the performance / code complexity impact. If all print functions need to handle retries, things get complicated. We could add a writeBlocking(), which takes care of that and is called by all print functions.

Perhaps we can even invert this, by keeping the existing write() as blocking, and add a writeNonBlocking() (name might need improvement) that is not blocking. If we add a NonBlockingPrint subclass of Print, that can define a pure virtual writeNonBlocking() (to be implemented by subclasses that support partial writes), and implement a write() function that calls writeNonBlocking() and retries on partial writes. writeNonBlocking() could perhaps also use a negative value for errors to distinguish between errors and partial writes (which I think is needed in "buffer full" situations).

AFAICS, this would add non-blocking support on an opt-in basis, while staying compatible with the existing APIs completely? One potential challenge is that Stream derives from Print, but we can probably figure that one out somehow.

[PaulStoffregen]

Perhaps we can even invert this, by keeping the existing write() as blocking, and add a writeNonBlocking() (name might need improvement) that is not blocking.

Don't we already have availableForWrite() defined in the Arduino API to solve this non-blocking need?

[stickbreaker]

I believe that write() should be single character based. It should return at a minimum 0 or 1, possibly -1 for fatal errors. But, adding -1 handling would require a complete para-dine shift. Just returning 0 and having print() fail back creates the ability to identify the failure. The argument that print()'s conversion to Human Readable Data (HRD) renders the count invalid, and therefore count is useless is an oversight. I agree that using the value of count on HRD is useless. But, when I use count as returned by print(), I always have an expected value to compare to. Most Arduino sketch's exist without any error detection or correction. Returning the Actual number of characters written is valuable. The reason I submitted the fail on write bug, was that I had added RTS/CTS handling to HardwareSerial. This created the possibility of an infinite hang on CTS. I added timeout logic to HardwareSerial.write(). In my upper level code I can either use availableForWrite() or comparing count with an expected value. Without the fail, count became meaningless, and I was not able to recover from a buffer full timeout.
Correcting the behavior of print() to abort on write()==0 removes unexpected effects. if write() returns 0 and print() ignores it and sends the next character the output becomes garbage. If print() fails with the current count at least the sketch has a chance to recover. If print() is going to return a count of bytes sent, that count should be accurate.
The argument between detecting fails and counting output and justifying text is obscuring the actual question:

Should print() abort with a current count of written characters, or should print() write() it's complete buffer before returning?

We need a style sheet. The expected behavior of print() is that it always works, and the all data is accurately transferred. That is not always the case. Serial() always sends the data but the device may not be able to receive/process it fast enough. Overruns Occur.
The Arduino ecosystem is continuously expanding, the net appliances and GPS devices and HID's that use Serial() are creating greater and greater timing issues. I have had to use baud rate speed to improve data transmission reliability. Most of the Serial() devices I work with have RTS/CTS hardware abilities but supporting these controls in the upper level Sketch code creates reliability issues. If the Arduino environment is going to support these complex devices we must enhance the environment to make using them easy and reliable. print()'s reaction to write() failures needs to be consistent and expected. Simplicity should be as viewed from the Sketch. My RTS/CTS implementation uses PinChange interrupts because of the scarcity of interrupt pins. RTS/CTS changes are usually really slow and intermittent. In my implement of RTS/CTS handshaking, the upper level Sketch has only to concern itself with two things.

• TX buffer overflows/timeouts
  • does expected==print() or is availableForWrite() >= expected
  • has some global timeout occured?
• RX buffer
  • service the Serial.read() fast enough such that the device does not give up!

Coding is quite easy:

Serial.begin(  baud, characteristic, ctsPin, rtsPin);
char ch[50];
char chLen=25,i=0;
ch[chLen]='\0';
unsigned long timeout=millis();
//try to send ch[] for up to 10 seconds
while((millis()-timeout)<10000)&&(i<chLen)){
  i+=Serial.print((char*)ch[i]);
  }
// the RTS pin handshaking will pause the device until I can process the data

@matthijskooijman brings up some good points:

Other than that, I agree with your ideas. Print should be blocking (no meaningful way to support partial writes), but write might support partials. Print can just retry writing repeatedly as well (until an error occurs / nothing is written), to bridge the gap between these two. I do wonder about the performance / code complexity impact. If all print functions need to handle retries, things get complicated. We could add a writeBlocking(), which takes care of that and is called by all print functions.

But, I think that print() should always respond to a write()==0 as a fail. The virtual write() function should be the one in control of the retry, block on buffer full, fail out.

Perhaps we can even invert this, by keeping the existing write() as blocking, and add a writeNonBlocking() (name might need improvement) that is not blocking. If we add a NonBlockingPrint subclass of Print, that can define a pure virtual writeNonBlocking() (to be implemented by subclasses that support partial writes), and implement a write() function that calls writeNonBlocking() and retries on partial writes. writeNonBlocking() could perhaps also use a negative value for errors to distinguish between errors and partial writes (which I think is needed in "buffer full" situations).

When an Object uses print() it decides how it implements write() so, write() is custom for the Object. These decision should be left for the custom Object.

print() just needs to respond the same way across all of it's methods.

Chuck.

[bperrybap]

On 01/08/2016 06:38 PM, chuck todd wrote:

I believe that write() should be single character based. It should
return at a minimum 0 or 1, possibly -1 for fatal errors. But, adding
-1 handling would require a complete para-dine shift. Just returning 0
and having print() fail back creates the ability to identify the
failure. The argument that print()'s conversion to Human Readable
Data (HRD) renders the /count/ invalid, and therefore /count/ is
useless is an oversight. I agree that using the value of /count/ on
HRD is useless. But, when I use /count/ as returned by print(), I
always have an expected value to compare to.

Perhaps you may know your output counts, but knowing the expected output
character count is not guaranteed, since it can vary depending on what
print() was told to print.
For example when a number is printed, the caller does not know how many
characters/digits that will end up being sent down through write.
Say the decimal value of a variable was 1234 and there was an issue
after printing the 3rd character and print() returned 2 since only 2
characters were successfully written.
The caller has no idea of knowing if there were only 2 total characters
or whether there were more than 2 and the remaining characters didn't
get output.

Most Arduino sketch's exist without any error detection or
correction. Returning the Actual number of characters written is
valuable. The reason I submitted the fail on write bug, was that I
had added RTS/CTS handling to HardwareSerial. This created the
possibility of an infinite hang on CTS. I added timeout logic to
*HardwareSerial.write(). In my upper level code I can either use
availableForWrite() or comparing /count/ with an expected value.
Without the fail, /count/ became meaningless, and I was not able to
recover from a buffer full timeout.
*

There are other potential conditions that are harder to deal with.
For example, how do you handle the case when using buffered interrupt
driven transmission and the CTS timeout occurs on characters in the TX
buffer that have already been acknowledged by write() ?
i.e you do something like print("Hello World");
This is 11 characters, they all fit in the Transmit buffer, write()
returns 11 but none have actually made out yet.
If CTS is stuck in the incorrect state, they never go out.
If you are going to have a CTS timout and purge characters after an
extended period,
you can't take back the successful acknowledgement of the 11 characters.
The application will never know that there has been an error on those 11
characters and none of them were ever transmitted.

And that is why dealing with these types of low level issues is normally
handled at a much lower layer rather than involving the application with
return values from something like print().
**

Correcting the behavior of print() to abort on write()==0 removes
unexpected effects. if write() returns 0 and print() ignores it
and sends the next character the output becomes garbage.*

The output doesn't necessarily become garbage.
Given your example, all it means is that if print ignores the 0 return
value and print() sends the next character,
the character being sent when write() returned 0 is lost.
This might repeat for several characters so that more than a single
character is lost.

If *print() fails with the current /count/ at least the sketch has a
chance to recover. If print() is going to return a /count/ of bytes
sent, that count should be accurate.
*

There is no way the sketch can really ever recover because the sketch
has no idea what the count should be.
Plus since there can be buffering in the device class library, there is
no way to know which of the count characters made it out and which
didn't but were handed down and buffered but later thrown away due to an
error.
i.e. what is an "accurate" count when buffering is involved?

Just like I pointed out in the example above, if you call print() to
print a number and you get a non zero return value how do you know
if all the characters were actually transmitted or if only part of them
went out? Or how many were buffered but later thrown away before
actually being transmitted?
Even if you ignore issues related to buffering, there would be no way to
know the expected return count value for print(), unless you knew how
many digits were going to printed and the caller doesn't know this.
The sketch could potentially know that there was an issue with the
print() but it can't ever really pickup with the character just after
the error to continue the output.
The best it could do is detect the error, and retry the entire print()
operation.

The argument between detecting fails and counting output and
justifying text is obscuring the actual question:*

/Should *print()* abort with a current count of written
characters, or should *print()* *write()* it's complete buffer
before returning?/

We need a style sheet. The expected behavior of print() is that it
always works, and the all data is accurately transferred. That is not
always the case.

Serial() always sends the data but the device may not be able to
receive/process it fast enough. Overruns Occur.

I do agree that the return behavior of print() should be fully defined.
My understanding was that it was a synchronous interface and would not
return until all characters had been handed to the device class library
or an unrecoverable error occurred.
That seems to be the way it worked in the past before buffering and it
seems to still work this way even now that buffering is done.
And so callers of print() can experience blocking behaviors if they send
more data than the device class library can buffer.
But the caller of print() really doesn't know how many characters were
actually REALLY transmitted. All it knows is that when print() returned,
the device library either successfully took all the characters, or it
didn't.

When you start to talk about receiver overruns, you are starting to talk
about reception of data and not just transmission of data.
As far as the sender who was calling write() goes, all data was
transmitted (or at least handed off to the device class library buffer)
even if the receiver received none of it because of overruns on the
receiver side.

In systems which have real device drivers, there are multiple modes for
transmission that can be configured within the device driver.
I believe that this the issue Arduino is running into.
The original definition of write() was too simplistic and didn't account
for things like errors, flow control or special character character
handling like newlines or configurable modes of operation.
I brought up trying to have configurable modes for read() and write() a
few years back and was shot down even though they were 100% backward
compatible with 1.0 behaviors.

In real device drivers, there are raw modes and cooked modes (which do
special character and new line processing) and there are flow control
modes which handle the flow control
If you look at a function like printf() which is semantically on the
same level as the Arduino print(), it does not deal with such low level
i/o issues.
printf() simply calls the other level 3 library functions which in turn
call the level 2 library functions which in turn call the device drivers.
printf() expects the other functions and layers to handle the i/o and
return an error if they can't.
If the low level has been told to handle flow control, it handles it; if
the low level is not handling flow control and there are receiver overruns,
that is not a printf() issue and in fact printf() will not get an error
or even know about any character losses.
To me that is how it should work.
So my view is that print() and the callers of print() should not be
responsible for handling or ensuring reliable data reception or the
prevention of receiver overrun.
That is the reprehensibility of a lower layer.

Within the existing print() and write() APIs,
the best that can be done is to tell the caller of print() that there
was an issue in getting some of his output data to a lower layer.
There is no way to accurately indicate where that issue really occurred
to allow the application to pick up right at that point, especially when
buffering is involved.

--- bill

[stickbreaker]

@bperrybap , I agree having print() generate HRD reduces the value of count. But having print() ignore write()==0 makes count useless.

I agree that print() does not / should not handle retry, error recovery. print() is a generic data provider for the level 2 / level 1 Object write(). Inside the write() function is the only location where device specific retry/timeout/abort logic should exist.

printf() simply calls the other level 3 library functions which in turn call the level 2 library functions which in turn call the device drivers. printf() expects the other functions and layers to handle the i/o and return an error if they can't. If the low level has been told to handle flow control, it handles it; if the low level is not handling flow control and there are receiver overruns, that is not a printf() issue and in fact printf() will not get an error or even know about any character losses. To me that is how it should work. So my view is that print() and the callers of print() should not be responsible for handling or ensuring reliable data reception or the prevention of receiver overrun. That is the reprehensibility of a lower layer.

print() must be predictable. The only thing print() must do is to honor the abort when write()==0.

To answer you CTS timeout scenario, a call to Serial.availableForWrite() will tell me if the buffer has been emptied, and/or digitalRead(ctsPin) will tell me if the device is busy.

Arduino has been designed with a Best Effort model. It assumes everything worked as expected. I do not think write()==0 abort would break this model. For my modified HardwareSerial(), unless I call my modified Serial.begin(), Serial() functions exactly as before. Calling my modified Serial.begin() initializes the CTS/RTS hardware and changes the behavior. These changes are in the level2 write() and level1 interrupt handlers of the HardwareSerial() object.

I don't know if the UNO has enough guts to handle print()==-1 error propagation. I am having enough problem implementing a network layered RS232 <-> RS485 <-> nRF24L01 <-> RS485 on MEGA2560. I am using the 9bit hardware and packet checksums, so my upperlevel sketch is ignorant of data transmission internals. It just sends and receives packets all of the retries and validations are handled under the 'hood'.

Chuck.

[PaulStoffregen]

Recently there was a lengthy conversation on the Arduino Developers Mailing List regarding RTS/CTS.

After many messages, the consensus was Arduino would (someday) use Serial.attachRts(pin) and Serial.attachCts(pin) to allow hardware flow control. It was also decided these functions would return boolean, indicating success or failure. Boards not supporting hardware flow control, or attempts to use unsupported pins would return false.

I sincerely hope you'll adopt this API, rather than overloading Serial.begin(). Assuming Arduino stays with this already-made decision, following it will mean your code will be compatible with whatever Arduino does in the future.