Narrowing down cause of ChildCancelledException with no stack trace

[stappon]

I’m working on an Android feature using coroutines 1.4.2 that involves many complex flows using combine, flatMapLatest, stateIn, etc.

I'm getting reports of production crashes that are missing stack traces:

kotlinx.coroutines.flow.internal.ChildCancelledException unknown method:0
Child of the scoped flow was cancelled

This sounds similar to several other issues caused by Channel.offer throwing (#974, #1433, #2104). However, I already have these error-handling wrappers around every offer and sendBlocking call in my application code:

fun <E> SendChannel<E>.offerSafe(element: E): Boolean {
    return runCatching { offer(element) }.getOrDefault(false)
}

fun <E> SendChannel<E>.sendBlockingSafe(element: E) = try {
    sendBlocking(element)
} catch (e: ClosedSendChannelException) {
    null
}

I’ve been unable to reproduce this locally, so without a stack trace, I am unsure how to proceed. I know this report is way too vague to be actionable for you either (sorry!), but am hoping you may have tips for narrowing it down more.

Questions:

1. I am already calling System.setProperty(DEBUG_PROPERTY_NAME, DEBUG_PROPERTY_VALUE_ON) fairly early in application startup.

   • Would you expect this to solve the missing stack trace issue?
   • Is there a way to confirm whether or not debug mode has been successfully activated? I think I read somewhere that it only works if the system property is set before any coroutine-related code has been invoked. I think I am setting it early enough (I do see coroutine names in my debugger after adding it), but I can't rule out that the timing is different in production.

2. What is the best way to catch these ChildCancelledExceptions exceptions manually while preserving as much info as possible about their source? If they are indeed coming from an offer-after-cancellation somewhere, would you expect them to be catchable with Flow.catch downstream, even though normal cancellation exceptions are not?

3. I’ve noticed unguarded offer calls in a couple of pieces of third-party code I’m using. Do these look capable of causing this sort of crash when used inside flatMapLatest etc?

   • Android Room createFlow function
   • Android lifecycle LiveData.asFlow function

Thank you!

[qwwdfsad]

Would you expect this to solve the missing stack trace issue?

Yes. This property indeed should be set prior to any coroutines initialization.
Please beware of the default coroutines ProGuard config that you may want to disable: https://github.com/Kotlin/kotlinx.coroutines/blob/master/ui/kotlinx-coroutines-android/resources/META-INF/com.android.tools/r8-from-1.6.0/coroutines.pro#L22

Is there a way to confirm whether or not debug mode has been successfully activated

Unfortunately, no public API. Filed #2551. As a hacky way, you can try using launch {}.toString().contains("coroutine#")

Do these look capable of causing this sort of crash when used inside flatMapLatest etc?

Hard to tell without actually checking all the code and tests, but indeed these calls look pretty suspicious. We are likely to deprecate offer in 1.5 and provide a proper replacement.

What is the best way to catch these ChildCancelledExceptions exceptions manually while preserving as much info as possible about their source?

The best way is to enable a debug mode. The exception is thrown from flatMapLatest when the previous flow has been cancelled, and it is propagated to the upstream. So you may want to double-check these places.

would you expect them to be catchable with Flow.catch downstream, even though normal cancellation exceptions are not?

No, because ChildCancelledExceptions is a cancellation exception, just with a specific type and message

[qwwdfsad]

Also, could you please elaborate on the following:

• Are you using these operators along with callbackFlow or any integration module (rx2, rx3, javafx)?
• Who exactly is crashing an application? Is it coroutines exception handler, or some framework-specific exception handler?

[stappon]

We were able to reproduce this in a debug build and get a call stack! The crash was indeed from using a callbackFlow inside flatMapLatest. It turns out that our sendBlockingSafe wrapper does not actually work anymore, because it only catches ClosedSendChannelException - same situation as #1454. We’ve seen no new crashes since updating it to this:

fun <E> SendChannel<E>.sendBlockingSafe(element: E) =
    runCatching { sendBlocking(element) }.getOrDefault(Unit)

So, the crash itself is expected behavior. However, I’m still working on getting debug mode enabled in our release builds:

As a hacky way, you can try using launch {}.toString().contains("coroutine#")

That snippet always returns false for me, but it got me on the right track. I modified it like this:

val coroutineName = CoroutineName("foo")
val coroutineContext = Dispatchers.Default + coroutineName
val isDebugEnabled = coroutineContext.launch {}.toString().contains(coroutineName.name)

This works in debug builds - returns true if the debug mode system property is set and false otherwise. However, it always returns false in my release builds.

Please beware of the default coroutines ProGuard config that you may want to disable

This may be the culprit. Is that config applied to all Proguard-using Android coroutines projects by default? Should adding this to my proguard.cfg file be enough to override it?

-assumenosideeffects class kotlinx.coroutines.DebugKt {
    boolean getDEBUG() return true;
    boolean getRECOVER_STACK_TRACES() return true;
}

I tried making a release build with that modified config, but isDebugEnabled was still false. Unclear whether that’s because I didn’t disable Proguard properly, or because the debug-mode-detection hack just doesn’t work in release builds in general.

[qwwdfsad]

Glad you've found the root cause!

Should adding this to my proguard.cfg file be enough to override it?

Unfortunately, I'm not well-experienced in a proguard, so it's worth asking it in an Android-specific community.

Closing as "resolved", please stay tuned for #974!