From 8509cae450c27bc041b991c73e4994a36b7304c7 Mon Sep 17 00:00:00 2001 From: odersky Date: Sat, 17 Dec 2022 19:47:31 +0100 Subject: [PATCH 1/5] Add error handling strawman There is so far no standard way in Scala to address a class of situations that's often called "error handling". In these situations we have two possible outcomes representing success and failure. We would like to process successful results further and propagate failures outward. The aim is to find a "happy path" where we deal with success computations directly, without having to special-treat failing results at every step. There are many different ways in use in Scala to express these situations. - Express success with normal results and failure with exceptions. - Use `Option`, where success is wrapped in `Some` and failure is `None`. - Use `Either`, where success is wrapped in `Right` and failure is wrapped in `Left`. - Use `Try`, where success is wrapped in `Success` and failure (which must be an exception) is wrapped in `Failure`. - Use nulls, where success is a non-null value and failure is `null`. - Use a special `IO` monad like `ZIO`, where errors are built in. Exceptions propagate silently until they are handled in a `catch`. All other failure modes require either ad-hoc code or monadic lifting into for comprehensions to propagate. `nulls` can only be treated with ad-hoc code and they are not very popular in Scala so far. Exceptions should only be used if the failure case is very unlikely (rule of thumb: in less than 0.1% of cases). Even then they are often shunned since they are clunky to use and they currently undermine type safety (the latter point would be addressed by the experimental `saferExceptions` extension). That said, exceptions have their uses. For instance, it's better to abort code that fails in a locally unrecoverable way with an exception instead of aborting the whole program with a panic or `System.exit`. If we admit that not all error handling scenarios should be handled with exceptions, what else should we use? So far the only choices are ad-hoc code or monadic lifting. Both of these techniques suffer from the fact that a failure is propagated out of only a single construct and that major and pointless acrobatics are required to move out failures further. This commit sketches a systematic and uniform solution to this problem. It essentially provides a way to establish "prompts" for result types and a way to return to a prompt with an error value using a method called `_.?`. This is somewhat similar to Rust's `?` postfix operator, which seems to work well. But there are is an important difference: Rust's ? returns from the next-enclosing function or closure whereas our `?` method returns to an enclosing prompt, which can be further outside. This greatly improves the expressiveness of `?`, at the price of a more complex execution semantics. For instance, here is an implementation of a "traverse"-like operation, converting a `List[Option[T]]` into a `Option[List[T]]`. ```scala def traverse[T](xs: List[Option[T]]): Option[List[T]] = optional: xs.map(_.?) ``` Here, `optional` is the prompt for the `Option` type, and `.?` maps `Option[T]` to the underlying type `T`, jumping to the enclosing `optional` prompt in the case a `None` is encountered. You can think of it as a safe version of `Option`'s `get`. This function could not be written like this in Rust since `_.?` is a closure so a `None` element would simply be mapped to itself. Also unlike for Rust, Scala's technique is library-based. Similar prompt/return pairs can be defined for many other types. This commit also defines a type `Result` which is similar to Rust's that supports the pattern. `Result` will hopefully replace `Either`, which is in my opinion an abomination for error handling. All these constructs can be defined in terms of a lower-level construct where one can simply return a result to an enclosing boundary, without any distinction between success and failure. The low-level construct is composed of a `boundary` prompt and a `break` method that returns to it. It can also be used as a convenient replacement for non-local returns. On the JVM, the break operation is in general implemented by throwing a special `Break` exception, similar to how non-local returns were handled before. However, it is foreseen to implement an optimization pass that replaces `Break` exceptions by gotos if they are caught in the same scope without intervening functions. Dotty already has a `LabeledBlock` tree node that can be used for this purpose. This means that local return to prompts are about as fast as in Rust whereas non-local returns in Scala are a bit slower than local returns, whereas in Rust they are impossible. We plan to do benchmarks to quantify the cost more precisely. --- tests/run/errorhandling/Result.scala | 49 +++++++++++++++++++ tests/run/errorhandling/Test.scala | 67 ++++++++++++++++++++++++++ tests/run/errorhandling/break.scala | 21 ++++++++ tests/run/errorhandling/optional.scala | 20 ++++++++ 4 files changed, 157 insertions(+) create mode 100644 tests/run/errorhandling/Result.scala create mode 100644 tests/run/errorhandling/Test.scala create mode 100644 tests/run/errorhandling/break.scala create mode 100644 tests/run/errorhandling/optional.scala diff --git a/tests/run/errorhandling/Result.scala b/tests/run/errorhandling/Result.scala new file mode 100644 index 000000000000..48bb9df1c42b --- /dev/null +++ b/tests/run/errorhandling/Result.scala @@ -0,0 +1,49 @@ +package dotty.util +import boundary.Label + +abstract class Result[+T, +E] +case class Ok[+T](value: T) extends Result[T, Nothing] +case class Err[+E](value: E) extends Result[Nothing, E] + +object Result: + extension [T, E](r: Result[T, E]) + + /** `_.?` propagates Err to current Label */ + transparent inline def ? (using Label[Err[E]]): T = r match + case r: Ok[_] => r.value + case err => break(err.asInstanceOf[Err[E]]) + + /** If this is an `Err`, map its value */ + def mapErr[E1](f: E => E1): Result[T, E1] = r match + case err: Err[_] => Err(f(err.value)) + case ok: Ok[_] => ok + + /** Map Ok values, propagate Errs */ + def map[U](f: T => U): Result[U, E] = r match + case Ok(x) => Ok(f(x)) + case err: Err[_] => err + + /** Flatmap Ok values, propagate Errs */ + def flatMap[U](f: T => Result[U, E]): Result[U, E] = r match + case Ok(x) => f(x) + case err: Err[_] => err + + /** Simlar to `Try`: Convert exceptions raised by `body` to `Err`s. + * In principle, `Try[T]` should be equivalent to `Result[T, Exception]`. + * Note that we do not want to catch and reify all Throwables. + * - severe JVM errors that make continuation impossible should not be reified. + * - control throwables like `boundary.Break` should not be caught. We want + * them to return from a `Result`. + * (Generally, the focus on `Throwable` in Scala libraries is a mistake. + * Use `Exception` instead, as it was meant to in Java.) + */ + def apply[T](body: => T): Result[T, Exception] = + try Ok(body) + catch case ex: Exception => Err(ex) +end Result + +/** A prompt for `_.?`. It establishes a boundary to which `_.?` returns */ +object respond: + transparent inline def apply[T, E](inline body: Label[Err[E]] ?=> T): Result[T, E] = + boundary(Ok(body)) + diff --git a/tests/run/errorhandling/Test.scala b/tests/run/errorhandling/Test.scala new file mode 100644 index 000000000000..de89a0efa443 --- /dev/null +++ b/tests/run/errorhandling/Test.scala @@ -0,0 +1,67 @@ +import dotty.util.* + +/** boundary/break as a replacement for non-local returns */ +def indexOf[T](xs: List[T], elem: T): Int = + boundary: + for (x, i) <- xs.zipWithIndex do + if x == elem then break(i) + -1 + +def breakTest() = + println("breakTest") + assert(indexOf(List(1, 2, 3), 2) == 1) + assert(indexOf(List(1, 2, 3), 0) == -1) + +/** traverse becomes trivial to write */ +def traverse[T](xs: List[Option[T]]): Option[List[T]] = + optional(xs.map(_.?)) + +def optTest() = + println("optTest") + assert(traverse(List(Some(1), Some(2), Some(3))) == Some(List(1, 2, 3))) + assert(traverse(List(Some(1), None, Some(3))) == None) + +/** A check function returning a Result[Unit, _] */ +inline def check[E](p: Boolean, err: E): Result[Unit, E] = + if p then Ok(()) else Err(err) + +/** Another variant of a check function that returns directly to the given + * label in case of error. + */ +inline def check_![E](p: Boolean, err: E)(using l: boundary.Label[Err[E]]): Unit = + if p then () else l.break(Err(err)) + +/** Use `Result` to convert exceptions to `Err` values */ +def parseDouble(s: String): Result[Double, Exception] = + Result(s.toDouble) + +def parseDoubles(ss: List[String]): Result[List[Double], Exception] = + respond: + ss.map(parseDouble(_).?) + +/** Demonstrate combination of `check` and `.?`. */ +def trySqrt(x: Double) = // inferred: Result[Double, String] + respond: + check(x >= 0, s"cannot take sqrt of negative $x").? + math.sqrt(x) + +/** Instead of `check(...).?` one can also use `check_!(...)`. + * Note use of `mapErr` to convert Exception errors to String errors. + */ +def sumRoots(xs: List[String]) = // inferred: Result[Double, String] + respond: + check_!(xs.nonEmpty, "list is empty") // direct jump + val ys = parseDoubles(xs).mapErr(_.toString).? // direct jump + ys.reduce((x, y) => x + trySqrt(y).?) // need exception to propagate `Err` + +def resultTest() = + println("resultTest") + assert(sumRoots(List("1", "4", "9")) == Ok(6)) + assert(sumRoots(List("1", "-2", "4")) == Err(s"cannot take sqrt of negative -2.0")) + assert(sumRoots(List()) == Err("list is empty")) + assert(sumRoots(List("1", "3ab")) == Err("java.lang.NumberFormatException: For input string: \"3ab\"")) + +@main def Test = + breakTest() + optTest() + resultTest() \ No newline at end of file diff --git a/tests/run/errorhandling/break.scala b/tests/run/errorhandling/break.scala new file mode 100644 index 000000000000..5aa6a9dcc963 --- /dev/null +++ b/tests/run/errorhandling/break.scala @@ -0,0 +1,21 @@ +package dotty.util +import scala.util.control.ControlThrowable + +object boundary: + + class Break[T](val label: Label[T], val value: T) extends ControlThrowable + + class Label[T] extends ControlThrowable: + transparent inline def break(value: T): Nothing = throw Break(this, value) + + transparent inline def apply[T <: R, R](inline body: Label[T] ?=> R): R = + val local = Label[T]() + try body(using local) + catch case ex: Break[_] if ex.label eq local => + ex.value.asInstanceOf[T] + +end boundary + +object break: + transparent inline def apply[T](value: T)(using l: boundary.Label[T]): Nothing = + l.break(value) diff --git a/tests/run/errorhandling/optional.scala b/tests/run/errorhandling/optional.scala new file mode 100644 index 000000000000..5dee9cb21996 --- /dev/null +++ b/tests/run/errorhandling/optional.scala @@ -0,0 +1,20 @@ +package dotty.util +import boundary.Label + +/** A mockup of scala.Option */ +abstract class Option[+T] +case class Some[+T](x: T) extends Option[T] +case object None extends Option[Nothing] + +object Option: + /** This extension should be added to the companion object of scala.Option */ + extension [T](r: Option[T]) + transparent inline def ? (using label: Label[None.type]): T = r match + case Some(x) => x + case None => label.break(None) + +/** A prompt for `Option`, which establishes a boundary which `_.?` on `Option` can return */ +object optional: + transparent inline def apply[T](inline body: Label[None.type] ?=> T): Option[T] = + boundary(Some(body)) + From 788af096bf9c20f2f8923f2f277cf0ddb4974289 Mon Sep 17 00:00:00 2001 From: odersky Date: Mon, 26 Dec 2022 19:04:51 +0100 Subject: [PATCH 2/5] Simplify `boundary.apply` --- tests/run/errorhandling/break.scala | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/run/errorhandling/break.scala b/tests/run/errorhandling/break.scala index 5aa6a9dcc963..63778a38434d 100644 --- a/tests/run/errorhandling/break.scala +++ b/tests/run/errorhandling/break.scala @@ -11,8 +11,8 @@ object boundary: transparent inline def apply[T <: R, R](inline body: Label[T] ?=> R): R = val local = Label[T]() try body(using local) - catch case ex: Break[_] if ex.label eq local => - ex.value.asInstanceOf[T] + catch case ex: Break[T] @unchecked if ex.label eq local => + ex.value end boundary From 1067a14db2be6bc0306579c2cada3d64b5d7dcae Mon Sep 17 00:00:00 2001 From: odersky Date: Mon, 26 Dec 2022 19:05:31 +0100 Subject: [PATCH 3/5] Make test Scala.js-compliant --- tests/run/errorhandling/Test.scala | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/tests/run/errorhandling/Test.scala b/tests/run/errorhandling/Test.scala index de89a0efa443..d7c63d1b7001 100644 --- a/tests/run/errorhandling/Test.scala +++ b/tests/run/errorhandling/Test.scala @@ -56,10 +56,12 @@ def sumRoots(xs: List[String]) = // inferred: Result[Double, String] def resultTest() = println("resultTest") + def assertFail(value: Any, s: String) = value match + case Err(msg: String) => assert(msg.contains(s)) assert(sumRoots(List("1", "4", "9")) == Ok(6)) - assert(sumRoots(List("1", "-2", "4")) == Err(s"cannot take sqrt of negative -2.0")) - assert(sumRoots(List()) == Err("list is empty")) - assert(sumRoots(List("1", "3ab")) == Err("java.lang.NumberFormatException: For input string: \"3ab\"")) + assertFail(sumRoots(List("1", "-2", "4")), "cannot take sqrt of negative") + assertFail(sumRoots(List()), "list is empty") + assertFail(sumRoots(List("1", "3ab")), "NumberFormatException") @main def Test = breakTest() From a1a2f59c0450cda90f2e35f0794972d50010c638 Mon Sep 17 00:00:00 2001 From: odersky Date: Wed, 28 Dec 2022 19:38:42 +0100 Subject: [PATCH 4/5] Another test --- tests/run/errorhandling.check | 4 ++++ tests/run/errorhandling/Test.scala | 3 ++- tests/run/errorhandling/break.scala | 5 ++-- tests/run/errorhandling/kostas.scala | 35 ++++++++++++++++++++++++++++ 4 files changed, 44 insertions(+), 3 deletions(-) create mode 100644 tests/run/errorhandling.check create mode 100644 tests/run/errorhandling/kostas.scala diff --git a/tests/run/errorhandling.check b/tests/run/errorhandling.check new file mode 100644 index 000000000000..882ca57f5022 --- /dev/null +++ b/tests/run/errorhandling.check @@ -0,0 +1,4 @@ +breakTest +optTest +resultTest +Person(Kostas,5) diff --git a/tests/run/errorhandling/Test.scala b/tests/run/errorhandling/Test.scala index d7c63d1b7001..112ee9aed4dc 100644 --- a/tests/run/errorhandling/Test.scala +++ b/tests/run/errorhandling/Test.scala @@ -66,4 +66,5 @@ def resultTest() = @main def Test = breakTest() optTest() - resultTest() \ No newline at end of file + resultTest() + parseCsvIgnoreErrors() \ No newline at end of file diff --git a/tests/run/errorhandling/break.scala b/tests/run/errorhandling/break.scala index 63778a38434d..604d5ba888f7 100644 --- a/tests/run/errorhandling/break.scala +++ b/tests/run/errorhandling/break.scala @@ -11,8 +11,9 @@ object boundary: transparent inline def apply[T <: R, R](inline body: Label[T] ?=> R): R = val local = Label[T]() try body(using local) - catch case ex: Break[T] @unchecked if ex.label eq local => - ex.value + catch case ex: Break[T] @unchecked => + if ex.label eq local then ex.value + else throw ex end boundary diff --git a/tests/run/errorhandling/kostas.scala b/tests/run/errorhandling/kostas.scala new file mode 100644 index 000000000000..8caa878ee9d2 --- /dev/null +++ b/tests/run/errorhandling/kostas.scala @@ -0,0 +1,35 @@ +package optionMockup: + import dotty.util.boundary + object optional: + transparent inline def apply[T](inline body: boundary.Label[None.type] ?=> T): Option[T] = + boundary(Some(body)) + + extension [T](r: Option[T]) + transparent inline def ? (using label: boundary.Label[None.type]): T = r match + case Some(x) => x + case None => label.break(None) + +import optionMockup.* + +case class Person(name: String, age: Int) + +object PersonCsvParserIgnoreErrors: + def parse(csv: Seq[String]): Seq[Person] = + for + line <- csv + columns = line.split(",")x + parsed <- parseColumns(columns) + yield + parsed + + private def parseColumns(columns: Seq[String]): Option[Person] = + columns match + case Seq(name, age) => parsePerson(name, age) + case _ => None + + private def parsePerson(name: String, age: String): Option[Person] = + optional: + Person(name, age.toIntOption.?) + +def parseCsvIgnoreErrors() = + println(PersonCsvParserIgnoreErrors.parse(Seq("Kostas,5", "George,invalid", "too,many,columns")).mkString("\n")) \ No newline at end of file From 9f455a670ea2374d966be997923e5eeb4abc1d89 Mon Sep 17 00:00:00 2001 From: odersky Date: Sat, 31 Dec 2022 19:45:50 +0100 Subject: [PATCH 5/5] Add validation to the strawman --- tests/run/errorhandling/Result.scala | 23 ++++++++++++++++++++++- tests/run/errorhandling/Test.scala | 8 ++++++++ tests/run/errorhandling/kostas.scala | 2 +- 3 files changed, 31 insertions(+), 2 deletions(-) diff --git a/tests/run/errorhandling/Result.scala b/tests/run/errorhandling/Result.scala index 48bb9df1c42b..23f42c8100ac 100644 --- a/tests/run/errorhandling/Result.scala +++ b/tests/run/errorhandling/Result.scala @@ -28,6 +28,24 @@ object Result: case Ok(x) => f(x) case err: Err[_] => err + /** Validate both `r` and `other`; return a pair of successes or a list of failures. */ + def * [U](other: Result[U, E]): Result[(T, U), List[E]] = (r, other) match + case (Ok(x), Ok(y)) => Ok((x, y)) + case (Ok(_), Err(e)) => Err(e :: Nil) + case (Err(e), Ok(_)) => Err(e :: Nil) + case (Err(e1), Err(e2)) => Err(e1 :: e2 :: Nil) + + /** Validate both `r` and `other`; return a tuple of successes or a list of failures. + * Unlike with `*`, the right hand side `other` must be a `Result` returning a `Tuple`, + * and the left hand side is added to it. See `Result.empty` for a convenient + * right unit of chains of `*:`s. + */ + def *: [U <: Tuple](other: Result[U, List[E]]): Result[T *: U, List[E]] = (r, other) match + case (Ok(x), Ok(ys)) => Ok(x *: ys) + case (Ok(_), es: Err[?]) => es + case (Err(e), Ok(_)) => Err(e :: Nil) + case (Err(e), Err(es)) => Err(e :: es) + /** Simlar to `Try`: Convert exceptions raised by `body` to `Err`s. * In principle, `Try[T]` should be equivalent to `Result[T, Exception]`. * Note that we do not want to catch and reify all Throwables. @@ -37,9 +55,12 @@ object Result: * (Generally, the focus on `Throwable` in Scala libraries is a mistake. * Use `Exception` instead, as it was meant to in Java.) */ - def apply[T](body: => T): Result[T, Exception] = + inline def apply[T](body: => T): Result[T, Exception] = try Ok(body) catch case ex: Exception => Err(ex) + + /** Right unit for chains of `*:`s. Returns an `Ok` with an `EmotyTuple` value. */ + def empty: Result[EmptyTuple, Nothing] = Ok(EmptyTuple) end Result /** A prompt for `_.?`. It establishes a boundary to which `_.?` returns */ diff --git a/tests/run/errorhandling/Test.scala b/tests/run/errorhandling/Test.scala index 112ee9aed4dc..ef74da885ddb 100644 --- a/tests/run/errorhandling/Test.scala +++ b/tests/run/errorhandling/Test.scala @@ -62,6 +62,14 @@ def resultTest() = assertFail(sumRoots(List("1", "-2", "4")), "cannot take sqrt of negative") assertFail(sumRoots(List()), "list is empty") assertFail(sumRoots(List("1", "3ab")), "NumberFormatException") + val xs = sumRoots(List("1", "-2", "4")) *: sumRoots(List()) *: sumRoots(List("1", "3ab")) *: Result.empty + xs match + case Err(msgs) => assert(msgs.length == 3) + case _ => assert(false) + val ys = sumRoots(List("1", "2", "4")) *: sumRoots(List("1")) *: sumRoots(List("2")) *: Result.empty + ys match + case Ok((a, b, c)) => // ok + case _ => assert(false) @main def Test = breakTest() diff --git a/tests/run/errorhandling/kostas.scala b/tests/run/errorhandling/kostas.scala index 8caa878ee9d2..ec32750f4bc4 100644 --- a/tests/run/errorhandling/kostas.scala +++ b/tests/run/errorhandling/kostas.scala @@ -17,7 +17,7 @@ object PersonCsvParserIgnoreErrors: def parse(csv: Seq[String]): Seq[Person] = for line <- csv - columns = line.split(",")x + columns = line.split(",") parsed <- parseColumns(columns) yield parsed