The Problem with Scala's Error Handling & How Ceylon's Union Types Help

Even though Scala has built-in support for the old-school try & catch blocks, the current idiomatic approach to error handling in the scala world (& many other languages) is to represent “failability” of operations within the language’s strong type system.

Of course, the exact type used to represent failable operations depends on the API we are working with. Familiar examples of these APIs are Future which has failure built into it via its Failure subtype, Stream which represents Error as a possible event type that may be fired at any moment, and scala.util.Try that is very similar to Future except for the fact that it does not imply asynchrony.

Another interesting type for error handling is Either (and a similar type from scalaz). I will discuss this approach later on, as it is very related to the subject of this post. But for now, let’s focus on the other failable types above.

So, like I said, there are quite a few types that can represent the possibility of failure. For the sake of argument in this post, let’s use our own imaginary type called Failable[A] as a representative of all the above. The important thing is that Failable[A] has only a single type parameter (again, don’t think about Either for now, we’ll get back to it) that represents the type of the value that is produced in the success case.

What this means is that the type of failures that can occur is implicitly fixed as Throwable, and this is where the suckiness of our current idiomatic error handling style stems from.

The Suckiness

As a running example, let’s say we have a function like this that we need to call:

def createUser(email: Email, pass: Password): Failable[User] = {...}

This is how we would invoke createUser to implement a REST controller (please don’t mind the REST framework, it is an imaginary hybrid between akka-http and play :~)):

(path("users") & POST).handle {request =>
  val (email, pass) = parseJson(request.body)
  createUser(email, pass) map {_ => Ok} recover {
    case InvalidEmail => BadRequest("invalid_email")
    case EmailTaken => BadRequest("email_taken")
    case InvalidPass => BadRequest("invalid_password")
    // are there any more?!
  }
}

The problem is that after calling createUser above, we have no idea what possible errors it may return. The best case scenario here would be for the createUser function to have been documented to indicate its possible error types. Unfortunately documentation is condemned to always go out of date, never to be trusted.

In the worst case scenario (really the only scenario), on the other hand, we will have to go through the implementation of the function and look for the errors it may return. The problem can get even worse, however, if the implementation of createUser, in turn, invokes other functions that return Failables before returning its result as a flatMap over those Failables. In that case we will have to study the implementations of those functions as well. Effectively, we’ll have to follow a tree of Failable returning functions, rigorously studying the implementation of each.

That’s not ideal! What if we could encode the possible error types of each function as type parameters of Failable? What if we could freely compose various Failables into larger ones with the compiler doing all the heavy lifting of figuring out the error types of the composite Failable? What if we could handle some of the errors in a Failable and return a Failable that only contains the remaining unhandled error types?

Is that too much to ask? It turns out it actually is! At least too much to ask from Scala’s type system!

Handleable vs Fatal Errors

At this point, it is worth taking a moment to distinguish between handleable and non-handleable (i.e. fatal) errors.

Handleable errors, which are not really “error”s (or “exception”s) at all, are normal & expected failure outcomes of an operation. In the createUser example above, for instance, we saw some of the possible failures that we had to handle.

Non-handleable errors, on the other hand, are fatal because they are either logical errors created by the programmer (a good example is divide by zero) or even hardware problems (such as running out of memory or a cpu spontaneously combusting). Obviously there is no point in encoding fatal errors in the result types (or anywhere else in the function signatures) of our operations because we cannot handle them anyway.

Note that the focus of this post is only on handleable errors a.k.a “failure”s. That is, we want to be able to represent them in the return types of our operations.

The Curious Case of Either

As previously mentioned, an interesting return type for failable operations would be Either[A, B] (either scala.util.Either or scalaz.\/).

Basically, Either[A, B] represents a value that can be either an A or a B. In the context of error handling, we can use one of the type parameters (usually A) to represent our error type.

Now, there are two ways to represent our errors in A:

Way #1

Set A to be the lowest common supertype of the possible errors that our operation may return. This lowest common supertype usually turns out to be Throwable it-self, making this approach as useless as Failable.

At this point you may argue that the author of createUser should create an appropriate supertype for all of its possible errors and return Either[ErrorsSuperType, Result]. Even though this is generally good practice, it is not always feasible or convenient. Moreover, as previously mentioned, we are ideally looking for a way to be able to freely compose various Eithers (using monadic flatMaps or applicative means, for instance) to produce our final Either.

Way #2

Set A to nested Eithers. In other words, our return type would look like this:

Either[Error1, Either[Error2, Either[Error3, Result]]]

The main problem with this second approach is that Either is a poor man’s “union type” (about which we will talk shortly) in that it doesn’t really do any real unioning: it just dumbly stitches types together.

For instance, if we were to automatically compose an Either[A, B] with an Either[A, C], then the result would probably end up being an Either[Either[A, B], Either[A, C]] or even an Either[A, Either[B, Either[A, Either[C]]]].

In the above examples, other than the lesser problem that the nesting structure of the Eithers may end up to be completely irregular, the main problem is that type A appears more than once, whereas we really want it to appear once, so that we can handle it once.

The other main problem is in recovery. Specifically, I personally don’t think I could ever come up a generic function recover: (Either, (X -> Y)) -> Either in scala such that in the final Either, type X would be replaced with type Y. In other words, if I have an Either[Either[A, B], Either[A, C]], then it seems impossible in scala to “recover” or “map” using a function, say, f: B -> A, to end-up with an Either[A, C] or even an Either[Either[A, A], Either[A, C]].

Ceylon’s Union Types to the Rescue

Arguably the coolest thing about the Ceylon language are its union types, allowing you to, well, union your types together!

The union of types A and B is written as simply A | B. For instance, you could define the type of an optional integer as Null | Integer. Or you could have a List<Integer | String>.

As for the subject of our discussion, you could define the result type of your favorite failable operation as simply Error1 | Error2 | ... | ErrorN | Result.

And the best thing about all of this is that if you union an A | B with an A | C, the compiler will infer the composition as A | B | C, which is perfect!

flatMap and recover

Let us now define the flatMap & recover operations in Ceylon using its union types. These operations, as we will see, allow us to compose our error types together and also to recover from errors. Note that union types are expressive enough that we won’t really need to define a separate type like Failable at all.

After defining each operation we will also consider how it could look in Scala if it had support for union types.

Composing Failables with flatMap

With the flatMap operator we will be able to compose our “failable”s. That is, we can do the following:

// CEYLON
// Don't be alarmed by the java-like syntax of Ceylon!

// Our handy error types
class E1() {}
class E2() {}
class E3() {}

// Failable function
E1 | E2 | Integer webRequest() {...}

// Compose failable operations using flatMap
// The type of "result" will be: "E1 | E2 | String | E3"
value result = flatMap(webRequest(), (Integer i) =>
    (i > 0 then "All is goood!") else E3()
);

Exactly what we wanted! Of course the syntax of the language it-self is far from perfect, but imagine how awesome this would be if we had union types in scala!

Actually no need to imagine, this is how it would look:

// SCALA
// Failable function
def webRequest: E1 | E2 | Integer = {...}

// Compose failable operations using flatMap
// The type of "result" will be: "E1 | E2 | String | E3"
val result = webRequest flatMap {i: Integer =>
  if(i > 0) "All is goood!" else E3()
}

The implementation of flatMap in Ceylon is pretty simple:

// CEYLON
// Note: the second value param is f: B -> C. The syntax is weird.
A | C flatMap<A, B, C>(A | B x, C(B) f) {
  if(is B x) {
    return f(x);
  } else {
    return x;
  }
}

Remember I said the syntax for using flatMap in ceylon was ugly? Well, in reality you will have to write even uglier code because ceylon can’t automatically infer the types. So:

// CEYLON
// This actually does NOT compile in Ceylon!
// Ceylon cannot automatically infer the types here.
flatMap(webRequest(), (Integer i) =>
    (i > 0 then "All is goood!") else E3()
);

// But this one does!
flatMap<E1 | E2, Integer, String | E3>(webRequest(), (Integer i) =>
    (i > 0 then "All is goood!") else E3()
);

Recovering from Errors using recover

The recover operator allows us to recover from some of the errors of a failable operation. The result of the recovery will be a failable with fewer (or no) error types. For instance:

// CEYLON
// Failable function
E1 | E2 | Integer webRequest() {...}

// Recover from both errors. Type of result is simply "Integer".
value recovered =
    recover(recover(webRequest(),
                   (E2 e2) => 404),
           (E1 e1) => 200);

And here is the implementation of recover:

// CEYLON
// Implementation 1 (from scratch)
A recover<A, B>(A | B x, A(B) f) {
  if(is A x) {
    return x;
  } else {
    return f(x);
  }
}

// Implementation 2 (as a special case of flatMap)
A recover<A, B>(A | B x, A(B) f) {
  return flatMap<A, B, A>(x, f);
}

Again, please note that Ceylon can’t really infer the types when invoking recover, so we will actually have to manually specify them to help the compiler:

// CEYLON
// This actually does NOT compile in Ceylon!
// Ceylon cannot automatically infer the types here.
value recovered =
    recover(recover(webRequest(),
                   (E2 e2) => 404),
           (E1 e1) => 200);

// This one does compile!
value recovered = recover<Integer, E1>(
    recover<E1 | Integer, E2>(webRequest(), (E2 e2) => 0),
    (E1 e1) => 200);

In an ideal future scala with unions, this would probably look like below:

// SCALA
// Usage Style 1
// The compiler should infer the type of recovered as "Integer".
val recovered = webRequest recover {
  case E1 => 200
  case E2 => 404
}

// Usage Style 2
// The compiler should infer the type of recovered as "Integer | E1".
val recovered = webRequest recover {
  case E2 => 404
}

// Usage Style 3
// We specify the type of the result.
// The compiler should not allow us to ommit either case in recovery.
val recovered: Integer = webRequest recover {
  case E1 => 200
  case E2 => 404
}