Merge pull request #967 from Kotlin/stacktrace-recovery-improvements

Stacktrace recovery improvements and documentation

Author: qwwdfsad

COMPATIBILITY.md

@@ -7,9 +7,6 @@
 Library support for Kotlin coroutines with [multiplatform](#multiplatform) support.
 This is a companion version for Kotlin `1.3.20` release.
 
-**NOTE**: `0.30.2` was the last release with Kotlin 1.2 and experimental coroutines.
-See [COMPATIBILITY.md](COMPATIBILITY.md) for details of migration onto the stable Kotlin 1.3 coroutines.
-
 ```kotlin
 GlobalScope.launch {
     delay(1000)
@@ -57,6 +54,8 @@ GlobalScope.launch {
   * [Guide to kotlinx.coroutines by example](docs/coroutines-guide.md) (**read it first**)
   * [Guide to UI programming with coroutines](ui/coroutines-guide-ui.md)
   * [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md)
+  * [Debugging capabilities in kotlinx.coroutines](docs/debugging.md)
+* [Compatibility policy and experimental annotations](docs/compatibility.md)
 * [Change log for kotlinx.coroutines](CHANGES.md)
 * [Coroutines design document (KEEP)](https://github.com/Kotlin/KEEP/blob/master/proposals/coroutines.md)
 * [Full kotlinx.coroutines API reference](http://kotlin.github.io/kotlinx.coroutines)

README.md

@@ -135,6 +135,10 @@ public final class kotlinx/coroutines/CompletionHandlerException : java/lang/Run
 	public fun <init> (Ljava/lang/String;Ljava/lang/Throwable;)V
 }
 
+public abstract interface class kotlinx/coroutines/CopyableThrowable {
+	public abstract fun createCopy ()Ljava/lang/Throwable;
+}
+
 public final class kotlinx/coroutines/CoroutineContextKt {
 	public static final fun newCoroutineContext (Lkotlinx/coroutines/CoroutineScope;Lkotlin/coroutines/CoroutineContext;)Lkotlin/coroutines/CoroutineContext;
 }

binary-compatibility-validator/reference-public-api/kotlinx-coroutines-core.txt

@@ -0,0 +1,115 @@
+<!--- 
+/*
+ * Copyright 2016-2019 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
+ */
+---> 
+
+<!--- TOC -->
+
+* [Compatibility](#compatibility)
+* [Public API types](#public-api-types)
+  * [Experimental API](#experimental-api)
+  * [Obsolete API](#obsolete-api)
+  * [Internal API](#internal-api)
+  * [Stable API](#stable-api)
+  * [Deprecation cycle](#deprecation-cycle)
+* [Using annotated API](#using-annotated-api)
+  * [Programmatically](#programmatically)
+  * [Gradle](#gradle)
+  * [Maven](#maven)
+
+<!--- END_TOC -->
+
+## Compatibility
+This document describes the compatibility policy of `kotlinx.coroutines` library since version 1.0.0 and semantics of compatibility-specific annotations.
+
+
+## Public API types
+`kotlinx.coroutines` public API comes in five flavours: stable, experimental, obsolete, internal and deprecated. 
+All public API except stable is marked with the corresponding annotation.
+
+### Experimental API
+Experimental API is marked with [@ExperimentalCoroutinesApi][ExperimentalCoroutinesApi] annotation.
+API is marked experimental when its design has potential open questions which may eventually lead to 
+either semantics changes of the API or its deprecation.
+
+By default, most of the new API is marked as experimental and becomes stable in one of the next major releases if no new issues arise.
+Otherwise, either semantics is fixed without changes in ABI or API goes through deprecation cycle. 
+
+When using experimental API may be dangerous:
+* You are writing a library which depends on `kotlinx.coroutines` and want to use experimental coroutines API in a stable library API.
+It may lead to undesired consequences when end users of your library update their `kotlinx.coroutines` version where experimental API
+has slightly different semantics.
+* You want to build core infrastructure of the application around experimental API. 
+
+### Obsolete API
+Obsolete API is marked with [@ObsoleteCoroutinesApi][ObsoleteCoroutinesApi] annotation.
+Obsolete API is similar to experimental, but already known to have serious design flaws and its potential replacement, 
+but replacement is not yet implemented.
+
+The semantics of this API won't be changed, but it will go through a deprecation cycle as soon as the replacement is ready.
+
+### Internal API
+Internal API is marked with [@InternalCoroutinesApi][InternalCoroutinesApi] or is part of `kotlinx.coroutines.internal` package.
+This API has no guarantees on its stability, can and will be changed and/or removed in the future releases. 
+If you can't avoid using internal API, please report it to [issue tracker](https://github.com/Kotlin/kotlinx.coroutines/issues/new).
+
+### Stable API
+Stable API is guaranteed to preserve its ABI and documented semantics. If at some point unfixable design flaws will be discovered, 
+this API will go through a deprecation cycle and remain binary compatible as long as possible.
+
+### Deprecation cycle
+When some API is deprecated, it goes through multiple stages and there is at least one major release between stages.
+* Feature is deprecated with compilation warning. Most of the time, proper replacement 
+(and corresponding `replaceWith` declaration) is provided to automatically migrate deprecated usages with a help of IntelliJ IDEA.
+* Deprecation level is increased to `error` or `hidden`. It is no longer possible to compile new code against deprecated API, 
+  though it is still present in the ABI.
+* API is completely removed. While we give our best efforts not to do so and have no plans of removing any API, we still are leaving 
+this option in case of unforeseen problems such as security holes.
+
+## Using annotated API
+All API annotations are [kotlin.Experimental](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-experimental/index.html).
+It is done in order to produce compilation warning about using experimental or obsolete API.
+Warnings can be disabled either programmatically for a specific call site or globally for the whole module.
+
+### Programmatically
+For a specific call-site, warning can be disabled by using [UseExperimental](https://kotlinlang.org/api/latest/jvm/stdlib/kotlin/-use-experimental/index.html) annotation:
+```kotlin
+@UseExperimental(ExperimentalCoroutinesApi::class) // Disables warning about experimental coroutines API 
+fun experimentalApiUsage() {
+    someKotlinxCoroutinesExperimentalMethod()
+}
+``` 
+
+### Gradle
+For the Gradle project, a warning can be disabled by passing a compiler flag in your `build.gradle` file:
+
+```groovy
+tasks.withType(org.jetbrains.kotlin.gradle.tasks.AbstractKotlinCompile).all {
+    kotlinOptions.freeCompilerArgs += ["-Xuse-experimental=kotlinx.coroutines.ExperimentalCoroutinesApi"]
+}
+
+```
+
+### Maven
+For the Maven project, a warning can be disabled by passing a compiler flag in your `pom.xml` file:
+```xml
+<plugin>
+    <artifactId>kotlin-maven-plugin</artifactId>
+    <groupId>org.jetbrains.kotlin</groupId>
+    ... your configuration ...
+    <configuration>
+        <args>
+            <arg>-Xuse-experimental=kotlinx.coroutines.ExperimentalCoroutinesApi</arg>
+        </args>
+    </configuration>
+</plugin>
+```
+
+
+<!--- MODULE kotlinx-coroutines-core -->
+<!--- INDEX kotlinx.coroutines -->
+[ExperimentalCoroutinesApi]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-experimental-coroutines-api/index.html
+[ObsoleteCoroutinesApi]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-obsolete-coroutines-api/index.html
+[InternalCoroutinesApi]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-internal-coroutines-api/index.html
+<!--- END -->

docs/compatibility.md

@@ -0,0 +1,96 @@
+**Table of contents**
+
+<!--- TOC -->
+
+* [Debugging coroutines](#debugging-coroutines)
+* [Debug mode](#debug-mode)
+* [Stacktrace recovery](#stacktrace-recovery)
+  * [Stacktrace recovery machinery](#stacktrace-recovery-machinery)
+* [Debug agent](#debug-agent)
+  * [Debug agent and Android](#debug-agent-and-android)
+
+<!--- END_TOC -->
+
+
+## Debugging coroutines
+Debugging asynchronous programs is challenging, because multiple concurrent coroutines are typically working at the same time.
+To help with that, `kotlinx.coroutines` comes with additional features for debugging: debug mode, stacktrace recovery 
+and debug agent.
+
+## Debug mode
+
+The first debugging feature of `kotlinx.coroutines` is debug mode.
+It can be enabled either by setting system property [DEBUG_PROPERTY_NAME] or by running Java with enabled assertions (`-ea` flag).
+The latter is helpful to have debug mode enabled by default in unit tests.
+
+Debug mode attaches a unique [name][CoroutineName] to every launched coroutine.
+Coroutine name can be seen in a regular Java debugger, 
+in a string representation of the coroutine or in the thread name executing named coroutine. 
+Overhead of this feature is negligible and it can be safely turned on by default to simplify logging and diagnostic.
+
+## Stacktrace recovery
+
+Stacktrace recovery is another useful feature of debug mode. It is enabled by default in the debug mode, 
+but can be separately disabled by setting `kotlinx.coroutines.stacktrace.recovery` system property to `false`.
+
+Stacktrace recovery tries to stitch asynchronous exception stacktrace with a stacktrace of the receiver by copying it, providing
+not only information where an exception was thrown, but also where it was asynchronously rethrown or caught.
+
+It is easy to demonstrate with actual stacktraces of the same program that awaits asynchronous operation in `main` function 
+(runnable code is [here](../kotlinx-coroutines-debug/test/RecoveryExample.kt)):
+
+| Without recovery | With recovery |
+| - | - |
+| ![before](images/before.png "before") | ![after](images/after.png "after") |
+
+The only downside of this approach is losing referential transparency of the exception. 
+
+### Stacktrace recovery machinery
+
+This section explains the inner mechanism of stacktrace recovery and can be skipped.
+
+When an exception is rethrown between coroutines (e.g. through `withContext` or `Deferred.await` boundary), stacktrace recovery
+machinery tries to create a copy of the original exception (with the original exception as the cause), then rewrite stacktrace
+of the copy with coroutine-related stack frames (using [Throwable.setStackTrace](https://docs.oracle.com/javase/9/docs/api/java/lang/Throwable.html#setStackTrace-java.lang.StackTraceElement:A-)) 
+and then throws the resulting exception instead of the original one.
+
+Exception copy logic is straightforward:
+  1) If the exception class implements [CopyableThrowable], [CopyableThrowable.createCopy] is used.
+  2) If the exception class has class-specific fields not inherited from Throwable, the exception is not copied.
+  3) Otherwise, one of the public exception's constructor is invoked reflectively with an optional `initCause` call.  
+
+## Debug agent
+
+[kotlinx-coroutines-debug](../kotlinx-coroutines-debug) module provides one of the most powerful debug capabilities in `kotlinx.coroutines`.
+
+This is a separate module with a JVM agent that keeps track of all alive coroutines, introspects and dumps them similar to thread dump command,
+additionally enhancing stacktraces with information where coroutine was created.
+
+The full tutorial of how to use debug agent can be found in the corresponding [readme](../kotlinx-coroutines-debug/README.md).
+
+### Debug agent and Android
+
+Unfortunately, Android runtime does not support Instrument API necessary for `kotlinx-coroutines-debug` to function, triggering `java.lang.NoClassDefFoundError: Failed resolution of: Ljava/lang/management/ManagementFactory;`.
+
+Nevertheless, it will be possible to support debug agent on Android as soon as [GradleAspectJ-Android](https://github.com/Archinamon/android-gradle-aspectj)  will support androin-gradle 3.3 
+
+<!---
+Make an exception googlable
+java.lang.NoClassDefFoundError: Failed resolution of: Ljava/lang/management/ManagementFactory;
+        at kotlinx.coroutines.repackaged.net.bytebuddy.agent.ByteBuddyAgent$ProcessProvider$ForCurrentVm$ForLegacyVm.resolve(ByteBuddyAgent.java:1055)
+        at kotlinx.coroutines.repackaged.net.bytebuddy.agent.ByteBuddyAgent$ProcessProvider$ForCurrentVm.resolve(ByteBuddyAgent.java:1038)
+        at kotlinx.coroutines.repackaged.net.bytebuddy.agent.ByteBuddyAgent.install(ByteBuddyAgent.java:374)
+        at kotlinx.coroutines.repackaged.net.bytebuddy.agent.ByteBuddyAgent.install(ByteBuddyAgent.java:342)
+        at kotlinx.coroutines.repackaged.net.bytebuddy.agent.ByteBuddyAgent.install(ByteBuddyAgent.java:328)
+        at kotlinx.coroutines.debug.internal.DebugProbesImpl.install(DebugProbesImpl.kt:39)
+        at kotlinx.coroutines.debug.DebugProbes.install(DebugProbes.kt:49)
+-->
+
+<!--- MODULE kotlinx-coroutines-core -->
+<!--- INDEX kotlinx.coroutines -->
+[DEBUG_PROPERTY_NAME]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-d-e-b-u-g_-p-r-o-p-e-r-t-y_-n-a-m-e.html
+[CoroutineName]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-coroutine-name/index.html
+[CopyableThrowable]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-copyable-throwable/index.html
+[CopyableThrowable.createCopy]: https://kotlin.github.io/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-copyable-throwable/create-copy.html
+<!--- MODULE kotlinx-coroutines-debug -->
+<!--- END -->

docs/debugging.md

@@ -140,7 +140,7 @@ class TaskTest : TestBase() {
         } catch (e: Exception) {
             assertTrue(e is TestException)
             assertEquals("something went wrong", e.message)
-            assertSame(e, deferred.getCompletionExceptionOrNull())
+            assertSame(e.cause, deferred.getCompletionExceptionOrNull()) // debug mode stack augmentation
         }
     }
 

docs/images/after.png

@@ -30,8 +30,6 @@ public annotation class ObsoleteCoroutinesApi
  * Marks declarations that are **internal** in coroutines API, which means that should not be used outside of
  * `kotlinx.coroutines`, because their signatures and semantics will be changing between release without any
  * warnings and without providing any migration aids.
- *
- * @suppress **This an internal API and should not be used from general code.**
  */
 @Retention(value = AnnotationRetention.BINARY)
 @Experimental(level = Experimental.Level.ERROR)

docs/images/before.png

@@ -4,8 +4,8 @@
 
 package kotlinx.coroutines
 
-import kotlin.coroutines.Continuation
 import kotlinx.coroutines.internal.*
+import kotlin.coroutines.*
 
 /**
  * Name of the property that controls coroutine debugging. See [newCoroutineContext][CoroutineScope.newCoroutineContext].
@@ -26,6 +26,32 @@ public const val DEBUG_PROPERTY_NAME = "kotlinx.coroutines.debug"
  */
 internal const val STACKTRACE_RECOVERY_PROPERTY_NAME = "kotlinx.coroutines.stacktrace.recovery"
 
+/**
+ * Throwable which can be cloned during stacktrace recovery in a class-specific way.
+ * For additional information about stacktrace recovery see [STACKTRACE_RECOVERY_PROPERTY_NAME]
+ *
+ * Example of usage:
+ * ```
+ * class BadResponseCodeException(val responseCode: Int) : Exception(), CopyableThrowable<BadResponseCodeException> {
+ *
+ *  override fun createCopy(): BadResponseCodeException {
+ *    val result = BadResponseCodeException(responseCode)
+ *    result.initCause(this)
+ *    return result
+ *  }
+ * ```
+ */
+@ExperimentalCoroutinesApi
+public interface CopyableThrowable<T> where T : Throwable, T : CopyableThrowable<T> {
+
+    /**
+     * Creates a copy of the current instance.
+     * For better debuggability, it is recommended to use original exception as [cause][Throwable.cause] of the resulting one.
+     * Stacktrace of copied exception will be overwritten by stacktrace recovery machinery by [Throwable.setStackTrace] call.
+     */
+    public fun createCopy(): T
+}
+
 /**
  * Automatic debug configuration value for [DEBUG_PROPERTY_NAME]. See [newCoroutineContext][CoroutineScope.newCoroutineContext].
  */