Coroutines high CPU consuming v1.3.3

[rumatoest]

Hello my issue is related to #840
I'm using java 11, Kotlin 1.3.60 Coroutines 1.3.3 Ktor 1.3.1

I've struggle with the same CPU consuming issue as in #840 .
Setting system property kotlinx.coroutines.scheduler to off fixed CPU issue.

Looks like patch related to #840 was published before 1.3.3 release, thus it was not fix this issue completely.

[qwwdfsad]

Thanks for the report.
Could you please provide a reproducer or a sample of usage?
Because when evaluating changes in the scheduler, I've used "Ktor+websockets on FJP" as a baseline against "Ktor+websockets on CoroutineScheduler" and they seem to be on par

[rumatoest]

Hello.
After 24 hours I've realized that kotlinx.coroutines.scheduler gave no help.

I can not share my code because it is not in public domain.
But I could describe flow here

I use com.github.shyiko.skedule to schedule my tasks

fun schedule(cron: String, action: suspend () -> Unit) {
    val start = ZonedDateTime.now().truncatedTo(ChronoUnit.DAYS)
    val schTriggers = Schedule.parse(cron).iterate(start)

    logger.info { "Scheduler starting: $cron" }
    GlobalScope.launch {
        for (next in schTriggers) {
            val now = ZonedDateTime.now();
            if (next.isBefore(now)) {
                continue
            }
            val delayms = ChronoUnit.MILLIS.between(now, next)
            logger.debug("Delay ${delayms / 1000} sec for $cron ")
            delay(delayms)
            runBlocking {
                try {
                    logger.info { "Trigger scheduler: $cron" }
                    action()
                } catch (ex: Exception) {
                    logger.error("Exception in scheduler ${cron}. ${ex.message}", ex)
                }
            }
        }
    }
}

Each tasks use ktor client to fetch some data from different rest endpoints
I configure ktor client in each *.kt file like this

private val objMapper = jacksonObjectMapper().apply {
    propertyNamingStrategy = PropertyNamingStrategy.SNAKE_CASE
}

private val client = HttpClient(Apache) {
    install(JsonFeature) {
                serializer = JacksonSerializer()
    }
}

As a result after unpredictable time after application launch it start consume 100% of CPU

Probably, but not sure. It takes much more time till 100% CPU if I use Thread.sleep() loop inside main thread than coroutines delay runBlocking { while (true) { delay(1000) }}

I was able to catch 100% CPU in profiler

And here is thread dump
threaddump-1582888687739.tdump.zip

[IRus]

Probably related ktorio/ktor#1018

[rumatoest]

Well. Looks like it was not coroutines issue. It is mostly like Apache client for Ktor issue. When I'd change ktor client implementation I was not have any 100% CPU spikes.

But there was one interesting thing - some king of correlation between coroutine delay vs thread.sleep loop in main thread which somehow affects CPU spike appearance time.

[rrva]

@rumatoest Can you confirm if this was related to ktorio/ktor#1018 ? Was HTTPS involved for example, and did you try the fix of upgrading apache client listed there?

[rumatoest]

Well it is definitely was about ktor client implementation. It was related with https because I used it for https based API calls. I've never tried a fix or upgrade because I do not need to right now. Everything work with CIO client without any issues for me.