Skip to content

Refactor mpp/native build, introduce "concurrent" source set, test launcher #2074

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Oct 12, 2020

Conversation

elizarov
Copy link
Contributor

@elizarov elizarov commented Jun 1, 2020

Refactor mpp/native build, introduce "concurrent" source set, test launcher

New source sets:

  • "concurrent" source set is shared between "jvm" and "native"
  • "native" source set is subdivided into "nativeDarwin" (Apple) and "nativeOther" (Linux, etc)

Native tests are launched in two variants:

  • A default "test" task runs tests with memory leak checker from "mainNoExit" entry point.
  • A special "backgroundTest" task runs tests in a background worker from "mainBackground" entry point.

Other build improvement:

  • Modernize old-style IDEA-active hacks to kts helper.
  • Extract versions of JS test runner dependencies.
  • Remove redundant google repo reference from android tests.

@elizarov elizarov requested a review from qwwdfsad June 1, 2020 13:28
@elizarov elizarov force-pushed the mpp-concurrent-build branch 2 times, most recently from c1b3d70 to b10740b Compare June 2, 2020 10:30
@elizarov
Copy link
Contributor Author

elizarov commented Jun 2, 2020

Fixed import to IDEA, so that it now resolves properly in IDEA.

fun mainBackground(args: Array<String>) {
val worker = Worker.start(name = "main-background")
worker.execute(TransferMode.SAFE, { args.freeze() }) {
val result = testLauncherEntryPoint(it)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

testLauncherEntryPoint is K/N internal API to run tests, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes

// Test for memory leaks using a special entry point that does not exit but returns from main
binaries.getTest("DEBUG").freeCompilerArgs += ["-e", "kotlinx.coroutines.mainNoExit"]
// Configure a separate test where code runs in background
test("background", [org.jetbrains.kotlin.gradle.plugin.mpp.NativeBuildType.DEBUG]) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why it is not possible to use -trw flag instead of hand-rolled implementation?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not have CFRunLoopRun in the main thread, thus attempts to dispatch tasks to the main thread from -trw will hang.

@elizarov elizarov requested a review from qwwdfsad June 3, 2020 07:32
@elizarov
Copy link
Contributor Author

elizarov commented Jun 3, 2020

I've also added additional workarounds to improve IDE experience under 1.4. It is not ideal under 1.4 (platform declaration like CFRunLoopRun are not resolved), but it is better, because stdlib resolves properly.

def implementationConfiguration = configurations[hostMainCompilation.defaultSourceSet.implementationMetadataConfigurationName]
// Now find the libraries:
// Kotlin 1.3.x: Find all platform libs
// Kotlin 1.4.x: Finds platform libs & stdlib, but platform declarations are still not resolved due to IDE bugs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have the links to these bugs to allow future reviews to check if they are resolved? If so, it'd be nice to add extra comment lines for them IMO.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They will be fixed before 1.4 release and affect only HMPP projects anyway.

Copy link
Collaborator

@qwwdfsad qwwdfsad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, will merge after Kotlin 1.4

…uncher

New source sets:
* "concurrent" source set is shared between "jvm" and "native"
* "native" source set is subdivided into "nativeDarwin" (Apple) and "nativeOther" (Linux, etc)

Native tests are launched in two variants:
* A default "test" task runs tests with memory leak checker from "mainNoExit" entry point.
* A special "backgroundTest" task runs tests in a background worker from "mainBackground" entry point.

Other build improvement:
* Modernize old-style IDEA-active hacks to kts helper.
* Extract versions of JS test runner dependencies.
* Remove redundant google repo reference from android tests.
@elizarov elizarov force-pushed the mpp-concurrent-build branch 2 times, most recently from fc1845c to af4a9d1 Compare September 10, 2020 16:13
@elizarov
Copy link
Contributor Author

I've dropped hacks for 1.3.x and enabled HPMM. Now need to figure out why tests with workers don't pass under K/N.

@elizarov elizarov merged commit 738f5a2 into develop Oct 12, 2020
@elizarov elizarov deleted the mpp-concurrent-build branch October 12, 2020 16:03
recheej pushed a commit to recheej/kotlinx.coroutines that referenced this pull request Dec 28, 2020
…uncher (Kotlin#2074)

New source sets:
* "concurrent" source set is shared between "jvm" and "native"
* "native" source set is subdivided into "nativeDarwin" (Apple) and "nativeOther" (Linux, etc)

Native tests are launched in two variants:
* A default "test" task runs tests with memory leak checker from "mainNoExit" entry point.
* A special "backgroundTest" task runs tests in a background worker from "mainBackground" entry point.

Other build improvement:
* Modernize old-style IDEA-active hacks to kts helper.
* Extract versions of JS test runner dependencies.
* Remove redundant google repo reference from android tests.
recheej pushed a commit to recheej/kotlinx.coroutines that referenced this pull request Dec 28, 2020
…uncher (Kotlin#2074)

New source sets:
* "concurrent" source set is shared between "jvm" and "native"
* "native" source set is subdivided into "nativeDarwin" (Apple) and "nativeOther" (Linux, etc)

Native tests are launched in two variants:
* A default "test" task runs tests with memory leak checker from "mainNoExit" entry point.
* A special "backgroundTest" task runs tests in a background worker from "mainBackground" entry point.

Other build improvement:
* Modernize old-style IDEA-active hacks to kts helper.
* Extract versions of JS test runner dependencies.
* Remove redundant google repo reference from android tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants