Merge branch 'develop' into new-channels-select

Author: ndkoval

CHANGES.md

@@ -1,5 +1,72 @@
 # Change log for kotlinx.coroutines
 
+## Version 1.6.0
+
+Note that this is a full changelog relative to 1.5.2 version. Changelog relative to 1.6.0-RC3 can be found in the end.
+
+### kotlinx-coroutines-test rework
+
+* `kotlinx-coroutines-test` became a multiplatform library usable from K/JVM, K/JS, and K/N.
+* Its API was completely reworked to address long-standing issues with consistency, structured concurrency and correctness (#1203, #1609, #2379, #1749, #1204, #1390, #1222, #1395, #1881, #1910, #1772, #1626, #1742, #2082, #2102, #2405, #2462
+  ).
+* The old API is deprecated for removal, but the new API is based on the similar concepts ([README](kotlinx-coroutines-test/README.md)), and the migration path is designed to be graceful: [migration guide](kotlinx-coroutines-test/MIGRATION.md).
+
+### Dispatchers
+
+* Introduced `CoroutineDispatcher.limitedParallelism` that allows obtaining a view of the original dispatcher with limited parallelism (#2919).
+* `Dispatchers.IO.limitedParallelism` usages ignore the bound on the parallelism level of `Dispatchers.IO` itself to avoid starvation (#2943).
+* Introduced new `Dispatchers.shutdown` method for containerized environments (#2558).
+* `newSingleThreadContext` and `newFixedThreadPoolContext` are promoted to delicate API (#2919).
+
+### Breaking changes
+
+* When racing with cancellation, the `future` builder no longer reports unhandled exceptions into the global `CoroutineExceptionHandler`. Thanks @vadimsemenov! (#2774, #2791).
+* `Mutex.onLock` is deprecated for removal (#2794).
+* `Dispatchers.Main` is now used as the default source of time for `delay` and `withTimeout` when present(#2972).
+    * To opt-out from this behaviour, `kotlinx.coroutines.main.delay` system property can be set to `false`.
+* Java target of coroutines build is now 8 instead of 6 (#1589).
+
+### Bug fixes and improvements
+
+* Kotlin is updated to 1.6.0.
+* Kotlin/Native [new memory model](https://blog.jetbrains.com/kotlin/2021/08/try-the-new-kotlin-native-memory-manager-development-preview/) is now supported in regular builds of coroutines conditionally depending on whether `kotlin.native.binary.memoryModel` is enabled (#2914).
+* Introduced `CopyableThreadContextElement` for mutable context elements shared among multiple coroutines. Thanks @yorickhenning! (#2893).
+* `transformWhile`, `awaitClose`, `ProducerScope`, `merge`, `runningFold`, `runingReduce`, and `scan` are promoted to stable API (#2971).
+* `SharedFlow.subscriptionCount` no longer conflates incoming updates and gives all subscribers a chance to observe a short-lived subscription (#2488, #2863, #2871).
+* `Flow` exception transparency mechanism is improved to be more exception-friendly (#3017, #2860).
+* Cancellation from `flat*` operators that leverage multiple coroutines is no longer propagated upstream (#2964).
+* `SharedFlow.collect` now returns `Nothing` (#2789, #2502).
+* `DisposableHandle` is now `fun interface`, and corresponding inline extension is removed (#2790).
+* `FlowCollector` is now `fun interface`, and corresponding inline extension is removed (#3047).
+* Deprecation level of all previously deprecated signatures is raised (#3024).
+* The version file is shipped with each JAR as a resource (#2941).
+* Unhandled exceptions on K/N are passed to the standard library function `processUnhandledException` (#2981).
+* A direct executor is used for `Task` callbacks in `kotlinx-coroutines-play-services` (#2990).
+* Metadata of coroutines artifacts leverages Gradle platform to have all versions of dependencies aligned (#2865).
+* Default `CoroutineExceptionHandler` is loaded eagerly and does not invoke `ServiceLoader` on its exception-handling path (#2552).
+* Fixed the R8 rules for `ServiceLoader` optimization (#2880).
+* Fixed BlockHound integration false-positives (#2894, #2866, #2937).
+* Fixed the exception handler being invoked several times on Android, thanks to @1zaman (#3056).
+* `SendChannel.trySendBlocking` is now available on Kotlin/Native (#3064).
+* The exception recovery mechanism now uses `ClassValue` when available (#2997).
+* JNA is updated to 5.9.0 to support Apple M1 (#3001).
+* Obsolete method on internal `Delay` interface is deprecated (#2979).
+* Support of deprecated `CommonPool` is removed.
+* `@ExperimentalTime` is no longer needed for methods that use `Duration` (#3041).
+* JDK 1.6 is no longer required for building the project (#3043).
+* New version of Dokka is used, fixing the memory leak when building the coroutines and providing brand new reference visuals (https://kotlin.github.io/kotlinx.coroutines/) (#3051, #3054).
+
+### Changelog relative to version 1.6.0-RC3
+
+* Restored MPP binary compatibility on K/JS and K/N (#3104).
+* Fixed Dispatchers.Main not being fully initialized on Android and Swing (#3101).
+
+## Version 1.6.0-RC3
+
+* Fixed the error in 1.6.0-RC2 because of which `Flow.collect` couldn't be called due to the `@InternalCoroutinesApi` annotation (#3082)
+* Fixed some R8 warnings introduced in 1.6.0-RC (#3090)
+* `TestCoroutineScheduler` now provides a `TimeSource` with its virtual time via the `timeSource` property. Thanks @hfhbd! (#3087)
+
 ## Version 1.6.0-RC2
 
 * `@ExperimentalTime` is no longer needed for methods that use `Duration` (#3041).
@@ -23,7 +90,7 @@
 
 ### Dispatchers
 
-* * Introduced `CoroutineDispatcher.limitedParallelism` that allows obtaining a view of the original dispatcher with limited parallelism (#2919).
+* Introduced `CoroutineDispatcher.limitedParallelism` that allows obtaining a view of the original dispatcher with limited parallelism (#2919).
 * `Dispatchers.IO.limitedParallelism` usages ignore the bound on the parallelism level of `Dispatchers.IO` itself to avoid starvation (#2943).
 * Introduced new `Dispatchers.shutdown` method for containerized environments (#2558).
 * `newSingleThreadContext` and `newFixedThreadPoolContext` are promoted to delicate API (#2919).

README.md

@@ -2,7 +2,7 @@
 
 [![official JetBrains project](https://jb.gg/badges/official.svg)](https://confluence.jetbrains.com/display/ALL/JetBrains+on+GitHub)
 [![GitHub license](https://img.shields.io/badge/license-Apache%20License%202.0-blue.svg?style=flat)](https://www.apache.org/licenses/LICENSE-2.0)
-[![Download](https://img.shields.io/maven-central/v/org.jetbrains.kotlinx/kotlinx-coroutines-core/1.6.0-RC2)](https://search.maven.org/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core/1.6.0-RC2/pom)
+[![Download](https://img.shields.io/maven-central/v/org.jetbrains.kotlinx/kotlinx-coroutines-core/1.6.0)](https://search.maven.org/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core/1.6.0/pom)
 [![Kotlin](https://img.shields.io/badge/kotlin-1.6.0-blue.svg?logo=kotlin)](http://kotlinlang.org)
 [![Slack channel](https://img.shields.io/badge/chat-slack-green.svg?logo=slack)](https://kotlinlang.slack.com/messages/coroutines/)
 
@@ -83,7 +83,7 @@ Add dependencies (you can also add other modules that you need):
 <dependency>
     <groupId>org.jetbrains.kotlinx</groupId>
     <artifactId>kotlinx-coroutines-core</artifactId>
-    <version>1.6.0-RC2</version>
+    <version>1.6.0</version>
 </dependency>
 ```
 
@@ -101,7 +101,7 @@ Add dependencies (you can also add other modules that you need):
 
 ```groovy
 dependencies {
-    implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.0-RC2'
+    implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.0'
 }
 ```
 
@@ -127,7 +127,7 @@ Add dependencies (you can also add other modules that you need):
 
 ```groovy
 dependencies {
-    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.0-RC2")
+    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.0")
 }
 ```
 
@@ -147,7 +147,7 @@ Add [`kotlinx-coroutines-android`](ui/kotlinx-coroutines-android)
 module as a dependency when using `kotlinx.coroutines` on Android:
 
 ```groovy
-implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.6.0-RC2'
+implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.6.0'
 ```
 
 This gives you access to the Android [Dispatchers.Main]
@@ -180,7 +180,7 @@ In common code that should get compiled for different platforms, you can add a d
 ```groovy
 commonMain {
     dependencies {
-        implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.0-RC2")
+        implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.0")
     }
 }
 ```
@@ -192,7 +192,7 @@ Platform-specific dependencies are recommended to be used only for non-multiplat
 #### JS
 
 Kotlin/JS version of `kotlinx.coroutines` is published as 
-[`kotlinx-coroutines-core-js`](https://search.maven.org/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core-js/1.6.0-RC2/jar)
+[`kotlinx-coroutines-core-js`](https://search.maven.org/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core-js/1.6.0/jar)
 (follow the link to get the dependency declaration snippet) and as [`kotlinx-coroutines-core`](https://www.npmjs.com/package/kotlinx-coroutines-core) NPM package. 
 
 #### Native

build.gradle

@@ -58,6 +58,7 @@ buildscript {
         classpath "com.moowork.gradle:gradle-node-plugin:$gradle_node_version"
         classpath "org.jetbrains.kotlinx:binary-compatibility-validator:$binary_compatibility_validator_version"
         classpath "ru.vyarus:gradle-animalsniffer-plugin:1.5.3" // Android API check
+        classpath "org.jetbrains.kotlinx:kover:$kover_version"
 
         // JMH plugins
         classpath "com.github.jengelman.gradle.plugins:shadow:5.1.0"
@@ -105,7 +106,8 @@ allprojects {
 }
 
 apply plugin: "binary-compatibility-validator"
-apply plugin: 'base'
+apply plugin: "base"
+apply plugin: "kover-conventions"
 
 apiValidation {
     ignoredProjects += unpublished + ["kotlinx-coroutines-bom"]
@@ -303,7 +305,7 @@ def publishTasks = getTasksByName("publish", true) + getTasksByName("publishNpm"
 
 task deploy(dependsOn: publishTasks)
 
-apply plugin: 'animalsniffer-convention'
+apply plugin: "animalsniffer-conventions"
 
 clean.dependsOn gradle.includedBuilds.collect { it.task(':clean') }
 

buildSrc/build.gradle.kts

@@ -60,4 +60,5 @@ dependencies {
         exclude(group = "org.jetbrains.kotlin", module = "kotlin-stdlib")
     }
     implementation("ru.vyarus:gradle-animalsniffer-plugin:1.5.3") // Android API check
+    implementation("org.jetbrains.kotlinx:kover:${version("kover")}")
 }

buildSrc/src/main/kotlin/animalsniffer-convention.gradle.kts renamed to buildSrc/src/main/kotlin/animalsniffer-conventions.gradle.kts

@@ -0,0 +1,54 @@
+import kotlinx.kover.api.*
+import kotlinx.kover.tasks.*
+
+/*
+ * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
+ */
+apply(plugin = "kover")
+
+val notCovered = sourceless + internal + unpublished
+
+val expectedCoverage = mutableMapOf(
+    // These have lower coverage in general, it can be eventually fixed
+    "kotlinx-coroutines-swing" to 70,
+    "kotlinx-coroutines-android" to 50,
+    "kotlinx-coroutines-javafx" to 39, // JavaFx is not tested on TC because its graphic subsystem cannot be initialized in headless mode
+
+    // TODO figure it out, these probably should be fixed
+    "kotlinx-coroutines-debug" to 84,
+    "kotlinx-coroutines-reactive" to 65,
+    "kotlinx-coroutines-reactor" to 65,
+    "kotlinx-coroutines-rx2" to 78,
+    "kotlinx-coroutines-slf4j" to 81
+)
+
+extensions.configure<KoverExtension> {
+    disabledProjects = notCovered
+    /*
+     * Is explicitly enabled on TC in a separate build step.
+     * Examples:
+     * ./gradlew :p:check -- doesn't verify coverage
+     * ./gradlew :p:check -Pkover.enabled=true -- verifies coverage
+     * ./gradlew :p:koverReport -Pkover.enabled=true -- generates report
+     */
+    isDisabled = !(properties["kover.enabled"]?.toString()?.toBoolean() ?: false)
+}
+
+subprojects {
+    val projectName = name
+    if (projectName in notCovered) return@subprojects
+    tasks.withType<KoverVerificationTask> {
+        rule {
+            bound {
+                /*
+                 * 85 is our baseline that we aim to raise to 90+.
+                 * Missing coverage is typically due to bugs in the agent
+                 * (e.g. signatures deprecated with an error are counted),
+                 * sometimes it's various diagnostic `toString` or `catch` for OOMs/VerificationErrors,
+                 * but some places are definitely worth visiting.
+                 */
+                minValue = expectedCoverage[projectName] ?: 85 // COVERED_LINES_PERCENTAGE
+            }
+        }
+    }
+}

buildSrc/src/main/kotlin/kover-conventions.gradle.kts

@@ -3,7 +3,7 @@
 #
 
 # Kotlin
-version=1.6.0-RC2-SNAPSHOT
+version=1.6.0-SNAPSHOT
 group=org.jetbrains.kotlinx
 kotlin_version=1.6.0
 
@@ -22,7 +22,8 @@ rxjava2_version=2.2.8
 rxjava3_version=3.0.2
 javafx_version=11.0.2
 javafx_plugin_version=0.0.8
-binary_compatibility_validator_version=0.8.0-RC
+binary_compatibility_validator_version=0.8.0
+kover_version=0.5.0-RC2
 blockhound_version=1.0.2.RELEASE
 jna_version=5.9.0
 

gradle.properties

@@ -62,6 +62,6 @@ Integration with Guava [ListenableFuture](https://github.com/google/guava/wiki/L
 
 <!--- INDEX com.google.common.util.concurrent -->
 
-[com.google.common.util.concurrent.ListenableFuture]: https://kotlin.github.io/kotlinx.coroutines/https://google.github.io/guava/releases/28.0-jre/api/docs/com/google/common/util/concurrent/ListenableFuture.html
+[com.google.common.util.concurrent.ListenableFuture]: https://kotlin.github.io/kotlinx.coroutines/https://google.github.io/guava/releases/31.0.1-jre/api/docs/com/google/common/util/concurrent/ListenableFuture.html
 
 <!--- END -->

integration/kotlinx-coroutines-guava/README.md

@@ -2,12 +2,17 @@
  * Copyright 2016-2021 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
  */
 
-val guavaVersion = "28.0-jre"
+val guavaVersion = "31.0.1-jre"
 
 dependencies {
     compile("com.google.guava:guava:$guavaVersion")
 }
 
+java {
+    targetCompatibility = JavaVersion.VERSION_1_8
+    sourceCompatibility = JavaVersion.VERSION_1_8
+}
+
 externalDocumentationLink(
     url = "https://google.github.io/guava/releases/$guavaVersion/api/docs/"
 )

integration/kotlinx-coroutines-guava/build.gradle.kts

@@ -135,10 +135,8 @@ public fun <T> ListenableFuture<T>.asDeferred(): Deferred<T> {
     // Finally, if this isn't done yet, attach a Listener that will complete the Deferred.
     val deferred = CompletableDeferred<T>()
     Futures.addCallback(this, object : FutureCallback<T> {
-        override fun onSuccess(result: T?) {
-            // Here we work with flexible types, so we unchecked cast to trick the type system
-            @Suppress("UNCHECKED_CAST")
-            runCatching { deferred.complete(result as T) }
+        override fun onSuccess(result: T) {
+            runCatching { deferred.complete(result) }
                 .onFailure { handleCoroutineException(EmptyCoroutineContext, it) }
         }
 
@@ -351,7 +349,7 @@ private class JobListenableFuture<T>(private val jobToCancel: Job): ListenableFu
      *
      * To preserve Coroutine's [CancellationException], this future points to either `T` or [Cancelled].
      */
-    private val auxFuture = SettableFuture.create<Any>()
+    private val auxFuture = SettableFuture.create<Any?>()
 
     /**
      * `true` if [auxFuture.get][ListenableFuture.get] throws [ExecutionException].
@@ -436,7 +434,7 @@ private class JobListenableFuture<T>(private val jobToCancel: Job): ListenableFu
     }
 
     /** See [get()]. */
-    private fun getInternal(result: Any): T = if (result is Cancelled) {
+    private fun getInternal(result: Any?): T = if (result is Cancelled) {
         throw CancellationException().initCause(result.exception)
     } else {
         // We know that `auxFuture` can contain either `T` or `Cancelled`.

integration/kotlinx-coroutines-guava/src/ListenableFuture.kt

@@ -3,10 +3,10 @@
  */
 
 dependencies {
-    compile("org.slf4j:slf4j-api:1.7.25")
-    testCompile("io.github.microutils:kotlin-logging:1.5.4")
-    testRuntime("ch.qos.logback:logback-classic:1.2.3")
-    testRuntime("ch.qos.logback:logback-core:1.2.3")
+    implementation("org.slf4j:slf4j-api:1.7.32")
+    testImplementation("io.github.microutils:kotlin-logging:2.1.0")
+    testRuntimeOnly("ch.qos.logback:logback-classic:1.2.7")
+    testRuntimeOnly("ch.qos.logback:logback-core:1.2.7")
 }
 
 externalDocumentationLink(

integration/kotlinx-coroutines-slf4j/build.gradle.kts

@@ -43,6 +43,7 @@ public class MDCContext(
     /**
      * The value of [MDC] context map.
      */
+    @Suppress("MemberVisibilityCanBePrivate")
     public val contextMap: MDCContextMap = MDC.getCopyOfContextMap()
 ) : ThreadContextElement<MDCContextMap>, AbstractCoroutineContextElement(Key) {
     /**

integration/kotlinx-coroutines-slf4j/src/MDCContext.kt

@@ -918,6 +918,7 @@ public final class kotlinx/coroutines/flow/FlowKt {
 	public static final fun catch (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow;
 	public static final fun channelFlow (Lkotlin/jvm/functions/Function2;)Lkotlinx/coroutines/flow/Flow;
 	public static final fun collect (Lkotlinx/coroutines/flow/Flow;Lkotlin/coroutines/Continuation;)Ljava/lang/Object;
+	public static final synthetic fun collect (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object;
 	public static final fun collectIndexed (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;Lkotlin/coroutines/Continuation;)Ljava/lang/Object;
 	public static final fun collectLatest (Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function2;Lkotlin/coroutines/Continuation;)Ljava/lang/Object;
 	public static final fun combine (Lkotlinx/coroutines/flow/Flow;Lkotlinx/coroutines/flow/Flow;Lkotlin/jvm/functions/Function3;)Lkotlinx/coroutines/flow/Flow;

kotlinx-coroutines-core/api/kotlinx-coroutines-core.api

@@ -185,7 +185,6 @@ kotlin.sourceSets {
     jvmTest.dependencies {
         api "org.jetbrains.kotlinx:lincheck:$lincheck_version"
         api "org.jetbrains.kotlinx:kotlinx-knit-test:$knit_version"
-        api "com.esotericsoftware:kryo:4.0.0"
         implementation project(":android-unit-tests")
     }
 }
@@ -257,7 +256,7 @@ static void configureJvmForLincheck(task) {
     task.minHeapSize = '1g'
     task.maxHeapSize = '4g' // we may need more space for building an interleaving tree in the model checking mode
     task.jvmArgs = ['--add-opens', 'java.base/jdk.internal.misc=ALL-UNNAMED',   // required for transformation
-                     '--add-exports', 'java.base/jdk.internal.util=ALL-UNNAMED'] // in the model checking mode
+                    '--add-exports', 'java.base/jdk.internal.util=ALL-UNNAMED'] // in the model checking mode
     task.systemProperty 'kotlinx.coroutines.semaphore.segmentSize', '2'
     task.systemProperty 'kotlinx.coroutines.semaphore.maxSpinCycles', '1' // better for the model checking mode
 }
@@ -266,6 +265,28 @@ static void configureJvmForLincheck(task) {
 task moreTest(dependsOn: [jvmStressTest, jvmLincheckTest])
 check.dependsOn moreTest
 
+tasks.jvmLincheckTest {
+    kover {
+        enabled = false // Always disabled, lincheck doesn't really support coverage
+    }
+}
+
+def commonKoverExcludes =
+        ["kotlinx.coroutines.test.*", // Deprecated package for removal
+         "kotlinx.coroutines.debug.*", // Tested by debug module
+         "kotlinx.coroutines.channels.ChannelsKt__DeprecatedKt.*", // Deprecated
+         "kotlinx.coroutines.scheduling.LimitingDispatcher", // Deprecated
+         "kotlinx.coroutines.scheduling.ExperimentalCoroutineDispatcher" // Deprecated
+        ]
+
+tasks.koverHtmlReport {
+    excludes = commonKoverExcludes
+}
+
+tasks.koverVerify {
+    excludes = commonKoverExcludes
+}
+
 task testsJar(type: Jar, dependsOn: jvmTestClasses) {
     classifier = 'tests'
     from compileTestKotlinJvm.destinationDir

kotlinx-coroutines-core/build.gradle

@@ -88,7 +88,8 @@ public abstract class CoroutineDispatcher :
      * // At most 1 thread will be doing IO
      * private val fileWriterDispatcher = backgroundDispatcher.limitedParallelism(1)
      * ```
-     * is 6. Yet at most 4 coroutines can be executed simultaneously as each view limits only its own parallelism.
+     * Note how in this example the application has an executor with 4 threads, but the total sum of all limits
+     * is 6. Still, at most 4 coroutines can be executed simultaneously as each view limits only its own parallelism.
      *
      * Note that this example was structured in such a way that it illustrates the parallelism guarantees.
      * In practice, it is usually better to use [Dispatchers.IO] or [Dispatchers.Default] instead of creating a