Exception inside async {} is logged by uncaught exception handler

[kevinherron]

I can't find anything in the docs to explain why the following is or isn't a bug:

fun main(args: Array<String>) {
    val job = SupervisorJob()
    val scope = CoroutineScope(job + Dispatchers.Default)

    val deferred = GlobalScope.async(Dispatchers.Default, start = CoroutineStart.LAZY) {
        delay(5000)
        throw Exception("boom")
    }

    scope.launch {
        try {
            deferred.await()
        } catch (e: Exception) {
            println("error awaiting: $e")
        }
    }

    runBlocking {
        job.cancelAndJoin()

        try {
            deferred.await()
        } catch (e: Exception) {
            println("2nd await() caught $e")
        }
    }


    Thread.sleep(10000)
}

Results in:

error awaiting: kotlinx.coroutines.JobCancellationException: Job was cancelled; job=SupervisorJobImpl{Cancelling}@57ead6a
.
. (5 seconds later)
.
Exception in thread "DefaultDispatcher-worker-1" java.lang.Exception: boom
	at com.inductiveautomation.ignition.gateway.opcua.AsyncKt$main$deferred$1.invokeSuspend(async.kt:12)
	at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:32)
	at kotlinx.coroutines.DispatchedTask$DefaultImpls.run(Dispatched.kt:235)
	at kotlinx.coroutines.AbstractContinuation.run(AbstractContinuation.kt:19)
	at kotlinx.coroutines.scheduling.Task.run(Tasks.kt:94)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:586)
	at kotlinx.coroutines.scheduling.CoroutineScheduler.access$runSafely(CoroutineScheduler.kt:60)
	at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:732)
2nd await() caught java.lang.Exception: boom

The uncaught exception handler prints the exception after the 5 second delay expires whether or not the second call to await() is made.

Without the cancellation it behaves as expected:

error awaiting: java.lang.Exception: boom
2nd await() caught java.lang.Exception: boom

[kevinherron]

Hmm, seems similar to #875

[elizarov]

It happens so by design. Deferred is a multi-use future. Cancelling Defered.await() canceles this particular await, but does not cancel the underlying job of the Deferred itself (you have to explicitly invoke Deferred.cancel() if you need to).

So when await() is cancelled, the underlyng job is still working and when it crashes it gets handled by default exception handler, because it works in GlobalScope and has not custom CoroutineExceptionHandler installed.

You can read more about this paritucular design decision in this blog post: https://medium.com/@elizarov/futures-cancellation-and-coroutines-b5ce9c3ede3a

[kevinherron]

@elizarov why does an external coroutine being cancelled cause the crash inside the async block to be handled by the exception handler? In any other case it's not logged and only delivered to any future await() calls on the Deferred instance.

[kevinherron]

@elizarov that example prints nothing to my console...

[elizarov]

I see... That is indeed the question... let me dig deeper.

[elizarov]

I've minimized it to this:

fun main(args: Array<String>) {
    val deferred = GlobalScope.async {
        throw Exception("boom")
    }
    val job = GlobalScope.launch {
        try {
            deferred.await()
        } catch (e: Exception) {
            println("error awaiting: $e")
        }
    }
    job.cancel()
    Thread.sleep(3000)
}

[kevinherron]

Your new example does not exhibit the behavior (on my machine), but if I adjust the timing a bit, it does:

fun main(args: Array<String>) {
    val deferred = GlobalScope.async {
        delay(1000)
        throw Exception("boom")
    }
    val job = GlobalScope.launch(Dispatchers.Unconfined) {
        try {
            deferred.await()
        } catch (e: Exception) {
            println("error awaiting: $e")
        }
    }
    job.cancel()
    Thread.sleep(3000)
}

[elizarov]

Ok. Now I'm getting to it. So, essentially it is the same problem as explained in #830, so switching to PR #896 branch also fixes this problem. However, this particular problem that you've reported has deeper roots and it consistently reproduces even if cancel/fail do not race with each other. This deeper problem problem stems from the fact that Deferred.await() implementation does not remove its listeners on cancellation. The exception that you see goes away with #896, but this deeper problem stays, leading to potential resource exhaustion. I'll write a separate test & fix for it.

[kevinherron]

Thank you for looking into it.

[elizarov]

Thank you for reporting! As you can see, we still don't have all corner cases covered by test, so all those "something strange going on" observations are very important.