[Question] Is there thread switching when moving from Default dispatcher to IO dispatcher using withContext?

[haphamdev]

According to the documents of Dispatchers.IO, there will be no thread switching when using withContext to switch from Default dispatcher to IO dispatcher because IO dispatcher share the thread pool with Default dispatcher.
However, when I run my test code, the result doesn't seem to be like that.

Here is the code:

fun main(args: Array<String>) = runBlocking {
    repeat(4) {
        launch(Dispatchers.Default) {
            println("Start $it in ${Thread.currentThread().name}")
            withContext(Dispatchers.IO) {
                println("Continue $it in ${Thread.currentThread().name}")
                delay(10)
                println("Resume $it in ${Thread.currentThread().name}")
            }
        }
    }
}

And this is the result:

Start 0 in DefaultDispatcher-worker-1
Start 1 in DefaultDispatcher-worker-2
Start 2 in DefaultDispatcher-worker-3
Start 3 in DefaultDispatcher-worker-4
Continue 0 in DefaultDispatcher-worker-6
Continue 1 in DefaultDispatcher-worker-5
Continue 2 in DefaultDispatcher-worker-2
Continue 3 in DefaultDispatcher-worker-1
Resume 0 in DefaultDispatcher-worker-5
Resume 1 in DefaultDispatcher-worker-2
Resume 2 in DefaultDispatcher-worker-11
Resume 3 in DefaultDispatcher-worker-6

As you can see, every coroutine start in one thread and continue in another thread.

Could anyone explain why?
Or is there anything wrong with my code?

[qwwdfsad]

Hi, the documentations stays the following:

typically execution continues in the same thread.

It is rather an explanation of implementation details than any sort of guarantee.
To understand why the execution does or does not switches between threads it's better to study the actual implementation. TL;DR we provide best-effort guarantee which is timing and load sensitive

[psteiger]

@qwwdfsad, hi!

I think this is an opportunity to improve the docs. This is the complete sentence:

so using withContext(Dispatchers.IO) { ... } when already running on the Default dispatcher does not lead to an actual switching to another thread — typically execution continues in the same thread.

Even though the text after "—" uses the typically qualifier, the text before is adamant that the dispatcher does not lead to an actual switching to another thread.

In my opinion, a better phrasing would be:

so using withContext(Dispatchers.IO) { ... } when already running on the Default dispatcher typically does not lead to an actual switching to another thread — execution typically continues in the same thread.

Maybe some additional clarification of when the user shall expect context switching or not would also be nice.

[haphamdev]

@qwwdfsad One more question for my specific case.
If I have an operation that does a simple database query, should I switch from Default to IO dispatcher? My concern is whether the performance of the app will be improved, switching to Default to IO will spare Default thread pool for other request but thread switching is also costly.

[qwwdfsad]

@psteiger thanks for the suggestion, I've reworded the documentation!

@fanliver yes, in general it is a good practice to do all I/O on a pool separate from a regular threads doing CPU-related work. Though I do not have any strong recommendations here -- it's your application, you know its business and performance characteristics better and the final decision should be driven by the actual application-specific knowledge.

[broo2s]

@qwwdfsad
It is understood that we don't have a guarantee that the thread won't switch. But I'm really surprised that it switches in practice in the example above, because I think there are no technical limitations that make the switch necessary. If it switches even in such simple cases then it seems this optimization is pretty ineffective.

Do I understand correctly that switching from Default to IO prioritizes tasks in the queue over the current coroutine, so it executes without dispatching only if there are no tasks in the queue? Is there any reason why it doesn't work similarly to for example Dispatchers.Main.immediate or coroutineScope(), so it generally do not dispatch if possible?

edit:
Actually, even withContext(someDispatcher) does not dispatch if we use the same dispatcher. So it seems coroutines generally prefer to not dispatch if only possible. Which is reasonable, because it is faster. This Default -> IO is surprising to me, because it is one of very few cases where it dispatches even if it doesn't have to. I think it is inconsistent with how the rest of the framework works (but I know, there are no guarantees) and it affects the performance at least a little.

[qwwdfsad]

I think there are no technical limitations that make the switch necessary
Is there any reason why it doesn't work similarly to for example Dispatchers.Main.immediate or coroutineScope

There are, because coroutine-specific dispatching mechanism and dispatcher/executor/scheduler-specific context switching and task distribution (between threads) are not coupled in any way and do not know about each other.
The immediate/scope mechanism leverages dispatching knowledge, while specific Executor implementation just cannot do it.

But I'm really surprised that it switches in practice in the example above, because I think there are no technical limitations that make the switch necessary. If it switches even in such simple cases then it seems this optimization is pretty ineffective.

That's exactly the reason I do not want to write any additional information in the documentation :)

The scheduler is optimized for applications in steady state, not unit tests that are being executed in Java interpreter, where the cost of stack spilling (what coroutine code actually does prior to getting a chance to execute withContext) is much bigger than the cost of thread wakeup which just "steals" IO task.

To illustrate that, see the following, modified example:

fun Int.printIf(msg: String) {
    if (this >= 10_000) { // Show only warmed-up iterations
        println(msg)
    }
}

repeat(10_010) { 
    val outerJitIt = it
    withContext(coroutineContext) { // just to avoid join call and still have structured output
        repeat(4) {
            launch(Dispatchers.Default) {
                outerJitIt.printIf("Start $it in ${Thread.currentThread().id}")
                withContext(Dispatchers.IO) {
                    outerJitIt.printIf("Continue $it in ${Thread.currentThread().id}")
                    yield()
                    outerJitIt.printIf("Resume $it in ${Thread.currentThread().id}")
                }
            }
        }
        outerJitIt.printIf("")
    }
}

with that, we now have much better and much more consistent output:

[broo2s]

@qwwdfsad
Thank you for detailed explanation!

There are, because coroutine-specific dispatching mechanism and dispatcher/executor/scheduler-specific context switching and task distribution (between threads) are not coupled in any way and do not know about each other.

Ah, ok, by seeing that we can use dispatchers/continuation interceptors to hook into the process of both creating continuations and resuming them, I somehow suspected that dispatchers have quite a lot of control and understanding over the whole process. I was thinking about something similar to (just an example):

object DefaultIoScheduler {
    fun isDispatchNeeded(context: CoroutineContext): Boolean {
        return callerDispatcher != Dispatchers.IO && callerDispatcher != Dispatchers.Default
    }
}

Where callerDispatcher is the dispatcher used by the code that called withContext(). Technically speaking, it would be the dispatcher that was used when continuation was resumed (I believe?). After reading some code now I see dispatchers don't know who resumed the continuation, so they don't know if they have to dispatch or not. As a matter of fact, continuation can be resumed by anyone, it doesn't have to be a coroutine, so it is at all not clear who is the caller.

Dispatchers.Main.immediate is a very special case, because there is only one thread and we can easily check for it.

Is my understanding more or less correct?

To illustrate that, see the following, modified example:

Ohh, wow, so the answer to the whole discussion is basically that OPs and mine expectations were correct, but we verified results in a wrong way :-) That's great explanation!

[qwwdfsad]

return callerDispatcher != Dispatchers.IO && callerDispatcher != Dispatchers.Default

It will affect a lot of code in a pretty unforeseeable way.

For example the following code

// Current context: IO or Default
repeat(10) {
    launch(Dispatchers.IO) { Thread.sleep(1000) }
}

will now become sequential and will take 10 seconds instead of 1.

Dispatching and context switching are fundamentally different -- dispatching controls whether something should be dispatched (executed not in place) at all, while context switching is an optimization that does not force anything, but rather nudges implementation to do more efficient choices when it's possible: if the dispatching thread became dormant, it is hinted to take recently scheduled by this very thread task. For better understanding, one can read http://gee.cs.oswego.edu/dl/papers/fj.pdf as an example of a good design for similar (but not the same though) goals.

The fact, that this is an optimization that guarantees liveness (e.g. that the code in the former example will be executed in 1 sec nevertheless) makes it extremely hard and timing-sensitive.

I think following example may shed more light on the difference between dispatching choice and context switching optimization:

// We are in Default or IO

launch(Dispatchers.IO) { ... }
delay(10_000) // example 1
Thread.sleep(10_000) // example 2

In scenario "1", the launching thread immediately suspends in its delay and becomes dormant. The optimization is able to detect it dynamically and pick up scheduled launch on the same thread even if there are other scheduled coroutines.

In scenario "2", the optimization sees that the thread is still "busy" (blocked) and thus dynamically allocated/hands the task to another thread in IO dispatcher.

Dispatching cannot distinguish these two situations statically at the moment of launch dispatching -- it either can pick safe scenario ("dispatch") or unsafe ("execute in place"), potentially transforming multithreaded program into a single-threaded or deadlocked one