Merge pull request #2 from Kotlin/develop

Sync develop with parent project

Author: circusmagnus

CHANGES.md

@@ -1,5 +1,90 @@
 # Change log for kotlinx.coroutines
 
+## Version 1.3.5
+
+* `firstOrNull` operator. Contributed by @bradynpoulsen.
+* `java.time` adapters for Flow operators. Contributed by @fvasco.
+* `kotlin.time.Duration` support (#1402). Contributed by @fvasco. 
+* Memory leak with a mix of reusable and non-reusable continuations is fixed (#1855).
+* `DebugProbes` are ready for production installation: its performance is increased, the flag to disable creation stacktraces to reduce the footprint is introduced (#1379, #1372).
+* Stacktrace recovery workaround for Android 6.0 and earlier bug (#1866).
+* New integration module: `kotlinx-coroutines-jdk9` with adapters for `java.util.concurrent.Flow`.
+* `BroadcastChannel.close` properly starts lazy coroutine (#1713).
+* `kotlinx-coroutines-bom` is published without Gradle metadata.
+* Make calls to service loader in reactor integrations optimizable by R8 (#1817).
+
+## Version 1.3.4
+
+### Flow
+
+* Detect missing `awaitClose` calls in `callbackFlow` to make it less error-prone when used with callbacks (#1762, #1770). This change makes `callbackFlow` **different** from `channelFlow`.
+* `ReceiveChannel.asFlow` extension is introduced (#1490).
+* Enforce exception transparency invariant in `flow` builder (#1657).
+* Proper `Dispatcher` support in `Flow` reactive integrations (#1765).
+* Batch `Subscription.request` calls in `Flow` reactive integration (#766).
+* `ObservableValue.asFlow` added to JavaFx integration module (#1695).
+* `ObservableSource.asFlow` added to RxJava2 integration module (#1768).
+
+### Other changes
+
+* `kotlinx-coroutines-core` is optimized for R8, making it much smaller for Android usages (75 KB for `1.3.4` release).
+* Performance of `Dispatchers.Default` is improved (#1704, #1706).
+* Kotlin is updated to 1.3.70.
+* `CoroutineDispatcher` and `ExecutorCoroutineDispatcher` experimental coroutine context keys are introduced (#1805).
+* Performance of various `Channel` operations is improved (#1565).
+
+## Version 1.3.3
+
+### Flow
+* `Flow.take` performance is significantly improved (#1538).
+* `Flow.merge` operator (#1491).
+* Reactive Flow adapters are promoted to stable API (#1549).
+* Reusable cancellable continuations were introduced that improved the performance of various flow operators and iteration over channels (#1534).
+* Fixed interaction of multiple flows with `take` operator (#1610).
+* Throw `NoSuchElementException` instead of `UnsupportedOperationException` for empty `Flow` in `reduce` operator (#1659).
+* `onCompletion` now rethrows downstream exceptions on emit attempt (#1654).
+* Allow non-emitting `withContext` from `flow` builder (#1616).
+
+### Debugging
+
+* `DebugProbes.dumpCoroutines` is optimized to be able to print the 6-digit number of coroutines (#1535).
+* Properly capture unstarted lazy coroutines in debugger (#1544).
+* Capture coroutines launched from within a test constructor with `CoroutinesTimeout` test rule (#1542).
+* Stacktraces of `Job`-related coroutine machinery are shortened and prettified (#1574).
+* Stacktrace recovery unification that should provide a consistent experience recover of stacktrace (#1597).
+* Stacktrace recovery for `withTimeout` is supported (#1625).
+* Do not recover exception with a single `String` parameter constructor that is not a `message` (#1631).
+
+### Other features
+
+* `Dispatchers.Default` and `Dispatchers.IO` rework: CPU consumption is significantly lower, predictable idle threads termination (#840, #1046, #1286).
+* Avoid `ServiceLoader` for loading `Dispatchers.Main` (#1572, #1557, #878, #1606).
+* Consistently handle undeliverable exceptions in RxJava and Reactor integrations (#252, #1614).
+* `yield` support in immediate dispatchers (#1474).
+* `CompletableDeferred.completeWith(result: Result<T>)` is introduced.
+* Added support for tvOS and watchOS-based Native targets (#1596).
+
+### Bug fixes and improvements
+
+* Kotlin version is updated to 1.3.61.
+* `CoroutineDispatcher.isDispatchNeeded` is promoted to stable API (#1014).
+* Livelock and stackoverflows in mutual `select` expressions are fixed (#1411, #504).
+* Properly handle `null` values in `ListenableFuture` integration (#1510).
+* Making ReceiveChannel.cancel linearizability-friendly.
+* Linearizability of Channel.close in a complex contended cases (#1419).
+* ArrayChannel.isBufferEmpty atomicity is fixed (#1526).
+* Various documentation improvements.
+* Reduced bytecode size of `kotlinx-coroutines-core`, reduced size of minified `dex` when using basic functionality of `kotlinx-coroutines`.
+
+## Version 1.3.2
+
+This is a maintenance release that does not include any new features or bug fixes.
+
+* Reactive integrations for `Flow` are promoted to stable API.
+* Obsolete reactive API is deprecated.
+* Deprecation level for API deprecated in 1.3.0 is increased.
+* Various documentation improvements.
+
 ## Version 1.3.1
 
 This is a minor update with various fixes:

README.md

@@ -2,10 +2,10 @@
 
 [![official JetBrains project](https://jb.gg/badges/official.svg)](https://confluence.jetbrains.com/display/ALL/JetBrains+on+GitHub)
 [![GitHub license](https://img.shields.io/badge/license-Apache%20License%202.0-blue.svg?style=flat)](https://www.apache.org/licenses/LICENSE-2.0)
-[![Download](https://api.bintray.com/packages/kotlin/kotlinx/kotlinx.coroutines/images/download.svg?version=1.3.1) ](https://bintray.com/kotlin/kotlinx/kotlinx.coroutines/1.3.1)
+[![Download](https://api.bintray.com/packages/kotlin/kotlinx/kotlinx.coroutines/images/download.svg?version=1.3.5) ](https://bintray.com/kotlin/kotlinx/kotlinx.coroutines/1.3.5)
 
 Library support for Kotlin coroutines with [multiplatform](#multiplatform) support.
-This is a companion version for Kotlin `1.3.50` release.
+This is a companion version for Kotlin `1.3.70` release.
 
 ```kotlin
 suspend fun main() = coroutineScope {
@@ -45,8 +45,9 @@ suspend fun main() = coroutineScope {
   * [CoroutinesTimeout] test rule to automatically dump coroutines on test timeout.
 * [reactive](reactive/README.md) — modules that provide builders and iteration support for various reactive streams libraries:
   * Reactive Streams ([Publisher.collect], [Publisher.awaitSingle], [publish], etc), 
-    RxJava 2.x ([rxFlowable], [rxSingle], etc), and 
-    Project Reactor ([flux], [mono], etc). 
+  * Flow (JDK 9) (the same interface as for Reactive Streams),
+  * RxJava 2.x ([rxFlowable], [rxSingle], etc), and
+  * Project Reactor ([flux], [mono], etc).
 * [ui](ui/README.md) — modules that provide coroutine dispatchers for various single-threaded UI libraries:
   * Android, JavaFX, and Swing.
 * [integration](integration/README.md) — modules that provide integration with various asynchronous callback- and future-based libraries:
@@ -62,7 +63,6 @@ suspend fun main() = coroutineScope {
 * Guides and manuals: 
   * [Guide to kotlinx.coroutines by example](https://kotlinlang.org/docs/reference/coroutines/coroutines-guide.html) (**read it first**)
   * [Guide to UI programming with coroutines](ui/coroutines-guide-ui.md)
-  * [Guide to reactive streams with coroutines](reactive/coroutines-guide-reactive.md)
   * [Debugging capabilities in kotlinx.coroutines](docs/debugging.md)
 * [Compatibility policy and experimental annotations](docs/compatibility.md)
 * [Change log for kotlinx.coroutines](CHANGES.md)
@@ -83,15 +83,15 @@ Add dependencies (you can also add other modules that you need):
 <dependency>
     <groupId>org.jetbrains.kotlinx</groupId>
     <artifactId>kotlinx-coroutines-core</artifactId>
-    <version>1.3.1</version>
+    <version>1.3.5</version>
 </dependency>
 ```
 
 And make sure that you use the latest Kotlin version:
 
 ```xml
 <properties>
-    <kotlin.version>1.3.50</kotlin.version>
+    <kotlin.version>1.3.70</kotlin.version>
 </properties>
 ```
 
@@ -101,15 +101,15 @@ Add dependencies (you can also add other modules that you need):
 
 ```groovy
 dependencies {
-    implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.1'
+    implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.5'
 }
 ```
 
 And make sure that you use the latest Kotlin version:
 
 ```groovy
 buildscript {
-    ext.kotlin_version = '1.3.50'
+    ext.kotlin_version = '1.3.70'
 }
 ```
 
@@ -127,15 +127,15 @@ Add dependencies (you can also add other modules that you need):
 
 ```groovy
 dependencies {
-    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.1")
+    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.3.5")
 }
 ```
 
 And make sure that you use the latest Kotlin version:
 
 ```groovy
 plugins {
-    kotlin("jvm") version "1.3.50"
+    kotlin("jvm") version "1.3.70"
 }
 ```
 
@@ -146,7 +146,7 @@ Make sure that you have either `jcenter()` or `mavenCentral()` in the list of re
 Core modules of `kotlinx.coroutines` are also available for 
 [Kotlin/JS](#js) and [Kotlin/Native](#native).
 In common code that should get compiled for different platforms, add dependency to  
-[`kotlinx-coroutines-core-common`](https://search.maven.org/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core-common/1.3.1/jar)
+[`kotlinx-coroutines-core-common`](https://search.maven.org/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core-common/1.3.5/jar)
 (follow the link to get the dependency declaration snippet).
 
 ### Android
@@ -155,7 +155,7 @@ Add [`kotlinx-coroutines-android`](ui/kotlinx-coroutines-android)
 module as dependency when using `kotlinx.coroutines` on Android:
 
 ```groovy
-implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.3.1'
+implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-android:1.3.5'
 ```
 
 This gives you access to Android [Dispatchers.Main]
@@ -165,24 +165,21 @@ threads are handled by Android runtime.
 
 #### R8 and ProGuard
 
-For R8 no actions required, it will take obfuscation rules from the jar.
-
-For Proguard  you need to add options from [coroutines.pro](kotlinx-coroutines-core/jvm/resources/META-INF/proguard/coroutines.pro) to your rules manually.
- 
-R8 is a replacement for ProGuard in Android ecosystem, it is enabled by default since Android gradle plugin 3.4.0 (3.3.0-beta also had it enabled).
+R8 and ProGuard rules are bundled into the [`kotlinx-coroutines-android`](ui/kotlinx-coroutines-android) module.
+For more details see ["Optimization" section for Android](ui/kotlinx-coroutines-android/README.md#optimization). 
 
 ### JS
 
 [Kotlin/JS](https://kotlinlang.org/docs/reference/js-overview.html) version of `kotlinx.coroutines` is published as 
-[`kotlinx-coroutines-core-js`](https://search.maven.org/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core-js/1.3.1/jar)
+[`kotlinx-coroutines-core-js`](https://search.maven.org/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core-js/1.3.5/jar)
 (follow the link to get the dependency declaration snippet).
 
 You can also use [`kotlinx-coroutines-core`](https://www.npmjs.com/package/kotlinx-coroutines-core) package via NPM. 
 
 ### Native
 
 [Kotlin/Native](https://kotlinlang.org/docs/reference/native-overview.html) version of `kotlinx.coroutines` is published as 
-[`kotlinx-coroutines-core-native`](https://search.maven.org/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core-native/1.3.1/jar)
+[`kotlinx-coroutines-core-native`](https://search.maven.org/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core-native/1.3.5/jar)
 (follow the link to get the dependency declaration snippet).
 
 Only single-threaded code (JS-style) on Kotlin/Native is currently supported. 
@@ -204,8 +201,9 @@ to Gradle (in Preferences -> Build, Execution, Deployment -> Build Tools -> Grad
 
 ### Requirements
 
-* JDK >= 1.8 referred to by the `JAVA_HOME` environment variable.
+* JDK >= 11 referred to by the `JAVA_HOME` environment variable.
 * JDK 1.6 referred to by the `JDK_16` environment variable. It is okay to have `JDK_16` pointing to `JAVA_HOME` for external contributions.
+* JDK 1.8 referred to by the `JDK_18` environment variable. Only used by nightly stress-tests. It is okay to have `JDK_16` pointing to `JAVA_HOME` for external contributions.
 
 ## Contributions and releases
 
@@ -218,6 +216,12 @@ The `develop` branch is pushed to `master` during release.
 
 * Full release procedure checklist is [here](RELEASE.md).
 * Steps for contributing new integration modules are explained [here](integration/README.md#Contributing).
+* Use [Knit](https://github.com/Kotlin/kotlinx-knit/blob/master/README.md) for updates to documentation:
+  * In project root directory run `./gradlew knit`.
+  * Commit updated documents and examples together with other changes.
+* Use [Binary Compatibility Validator](https://github.com/Kotlin/binary-compatibility-validator/blob/master/README.md) for updates to public API:
+  * In project root directory run `./gradlew apiDump`. 
+  * Commit updated API index together with other changes.
 
 <!--- MODULE kotlinx-coroutines-core -->
 <!--- INDEX kotlinx.coroutines -->

RELEASE.md

@@ -53,7 +53,8 @@ To release new `<version>` of `kotlinx-coroutines`:
    * Create a release named `<version>`. 
    * Cut & paste lines from [`CHANGES.md`](CHANGES.md) into description.    
 
-3. Build and publish documentation for web-site: <br>
+3. Build and publish documentation for web-site
+   (make sure you have [Docker](https://www.docker.com/) installed first): <br>
    `site/deploy.sh <version> push`
 
 4. In [Bintray](https://bintray.com/kotlin/kotlinx/kotlinx.coroutines) admin interface:

benchmarks/build.gradle

@@ -1,5 +1,5 @@
 /*
- * Copyright 2016-2018 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
+ * Copyright 2016-2020 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
  */
 sourceCompatibility = 1.8
 targetCompatibility = 1.8
@@ -74,4 +74,6 @@ dependencies {
     compile "org.openjdk.jmh:jmh-core:1.21"
     compile 'com.typesafe.akka:akka-actor_2.12:2.5.0'
     compile project(':kotlinx-coroutines-core')
+    // add jmh dependency on main
+    jmh sourceSets.main.runtimeClasspath
 }

benchmarks/scripts/generate_plots_flow_flatten_merge.py

@@ -0,0 +1,75 @@
+# To run this script run the command 'python3 scripts/generate_plots_flow_flatten_merge.py' in the /benchmarks folder
+
+
+import pandas as pd
+import sys
+import locale
+import matplotlib.pyplot as plt
+from matplotlib.ticker import FormatStrFormatter
+
+input_file = "build/reports/jmh/results.csv"
+output_file = "out/flow-flatten-merge.svg"
+# Please change the value of this variable according to the FlowFlattenMergeBenchmarkKt.ELEMENTS
+elements = 100000
+benchmark_name = "benchmarks.flow.FlowFlattenMergeBenchmark.flattenMerge"
+csv_columns = ["Benchmark", "Score", "Unit", "Param: concurrency", "Param: flowsNumberStrategy"]
+rename_columns = {"Benchmark": "benchmark", "Score" : "score", "Unit" : "unit",
+                  "Param: concurrency" : "concurrency", "Param: flowsNumberStrategy" : "flows"}
+
+markers = ['.', 'v', '^', '1', '2', '8', 'p', 'P', 'x', 'D', 'd', 's']
+colours = ['red', 'gold', 'sienna', 'olivedrab', 'lightseagreen', 'navy', 'blue', 'm', 'crimson', 'yellow', 'orangered', 'slateblue', 'aqua', 'black', 'silver']
+
+def next_colour():
+    i = 0
+    while True:
+        yield colours[i % len(colours)]
+        i += 1
+
+def next_marker():
+    i = 0
+    while True:
+        yield markers[i % len(markers)]
+        i += 1
+
+def draw(data, plt):
+    plt.xscale('log', basex=2)
+    plt.gca().xaxis.set_major_formatter(FormatStrFormatter('%0.f'))
+    plt.grid(linewidth='0.5', color='lightgray')
+    if data.unit.unique()[0] != "ops/s":
+        print("Unexpected time unit: " + data.unit.unique()[0])
+        sys.exit(1)
+    plt.ylabel("elements / ms")
+    plt.xlabel('concurrency')
+    plt.xticks(data.concurrency.unique())
+
+    colour_gen = next_colour()
+    marker_gen = next_marker()
+    for flows in data.flows.unique():
+        gen_colour = next(colour_gen)
+        gen_marker = next(marker_gen)
+        res = data[(data.flows == flows)]
+#         plt.plot(res.concurrency, res.score*elements/1000, label="flows={}".format(flows), color=gen_colour, marker=gen_marker)
+        plt.errorbar(x=res.concurrency, y=res.score*elements/1000, yerr=res.score_error*elements/1000, solid_capstyle='projecting',
+                     label="flows={}".format(flows), capsize=4, color=gen_colour, linewidth=2.2)
+
+langlocale = locale.getdefaultlocale()[0]
+locale.setlocale(locale.LC_ALL, langlocale)
+dp = locale.localeconv()['decimal_point']
+if dp == ",":
+    csv_columns.append("Score Error (99,9%)")
+    rename_columns["Score Error (99,9%)"] = "score_error"
+elif dp == ".":
+    csv_columns.append("Score Error (99.9%)")
+    rename_columns["Score Error (99.9%)"] = "score_error"
+else:
+    print("Unexpected locale delimeter: " + dp)
+    sys.exit(1)
+data = pd.read_csv(input_file, sep=",", decimal=dp)
+data = data[csv_columns].rename(columns=rename_columns)
+data = data[(data.benchmark == benchmark_name)]
+plt.rcParams.update({'font.size': 15})
+plt.figure(figsize=(12.5, 10))
+draw(data, plt)
+plt.legend(loc='upper center', borderpad=0, bbox_to_anchor=(0.5, 1.3), ncol=2, frameon=False, borderaxespad=2, prop={'size': 15})
+plt.tight_layout(pad=12, w_pad=2, h_pad=1)
+plt.savefig(output_file, bbox_inches='tight')

benchmarks/src/jmh/java/benchmarks/flow/scrabble/RxJava2PlaysScrabble.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2016-2019 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
+ * Copyright 2016-2020 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
  */
 
 package benchmarks.flow.scrabble;
@@ -160,4 +160,4 @@ public List<Entry<Integer, List<String>>> play() throws Exception {
                     .blockingGet() ;
         return finalList2 ;
     }
-}
+}

benchmarks/src/jmh/java/benchmarks/flow/scrabble/RxJava2PlaysScrabbleOpt.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2016-2019 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
+ * Copyright 2016-2020 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
  */
 
 package benchmarks.flow.scrabble;
@@ -171,4 +171,4 @@ public List<Entry<Integer, List<String>>> play() throws Exception {
 
         return finalList2 ;
     }
-}
+}

benchmarks/src/jmh/java/benchmarks/flow/scrabble/optimizations/FlowableCharSequence.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2016-2019 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
+ * Copyright 2016-2020 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
  */
 
 package benchmarks.flow.scrabble.optimizations;

benchmarks/src/jmh/java/benchmarks/flow/scrabble/optimizations/FlowableSplit.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2016-2019 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
+ * Copyright 2016-2020 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
  */
 
 package benchmarks.flow.scrabble.optimizations;

benchmarks/src/jmh/java/benchmarks/flow/scrabble/optimizations/StringFlowable.java

@@ -1,5 +1,5 @@
 /*
- * Copyright 2016-2019 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
+ * Copyright 2016-2020 JetBrains s.r.o. Use of this source code is governed by the Apache 2.0 license.
  */
 
 package benchmarks.flow.scrabble.optimizations;