Skip to content

Support JS IR target #151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jun 27, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions core/build.gradle.kts
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,16 @@ kotlin {
}
}

js(IR) {
nodejs {
testTask {
useMocha {
timeout = "30s"
}
}
}
}

configureNativePlatforms()
sourceSets {
val commonMain by getting
Expand Down
2 changes: 2 additions & 0 deletions core/common/src/files/Paths.kt
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ import kotlinx.io.Source
/**
* A wrapper around a string representing a file path allowing to read from and write to a
* corresponding file using [Sink] and [Source].
*
* **This API is unstable and subject to change.**
*/
public expect class Path

Expand Down
22 changes: 18 additions & 4 deletions core/common/test/files/SmokeFileTest.kt
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,7 @@

package kotlinx.io.files

import kotlinx.io.createTempFile
import kotlinx.io.deleteFile
import kotlinx.io.readUtf8Line
import kotlinx.io.writeUtf8
import kotlinx.io.*
import kotlin.test.AfterTest
import kotlin.test.BeforeTest
import kotlin.test.Test
Expand All @@ -31,6 +28,7 @@ class SmokeFileTest {
@Test
fun testBasicFile() {
val path = Path(tempFile!!)

path.sink().use {
it.writeUtf8("example")
}
Expand All @@ -39,4 +37,20 @@ class SmokeFileTest {
assertEquals("example", it.readUtf8Line())
}
}

@OptIn(ExperimentalStdlibApi::class)
@Test
fun testReadWriteMultipleSegments() {
val path = Path(tempFile!!)

val data = ByteArray((Segment.SIZE * 2.5).toInt()) { it.toByte() }

path.sink().use {
it.write(data)
}

path.source().use {
assertArrayEquals(data, it.readByteArray())
}
}
}
19 changes: 19 additions & 0 deletions core/js/src/-PlatfromJs.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
/*
* Copyright 2010-2023 JetBrains s.r.o. and Kotlin Programming Language contributors.
* Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE.txt file.
*/

package kotlinx.io

import kotlinx.io.internal.commonAsUtf8ToByteArray

internal actual fun String.asUtf8ToByteArray(): ByteArray = commonAsUtf8ToByteArray()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: can't we just use String.encodeToByteArray(): ByteArray on all platforms?

Copy link
Collaborator Author

@fzhinkin fzhinkin Jun 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It (encodeToByteArray) behaves differently on JVM and Native when encoding invalid code points (and I'll revisit its usage in ByteString's).


public actual open class IOException actual constructor(
message: String?,
cause: Throwable?
) : Exception(message, cause) {
public actual constructor(message: String?) : this(message, null)
}

public actual open class EOFException actual constructor(message: String?) : IOException(message)
15 changes: 15 additions & 0 deletions core/js/src/RawSink.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
/*
* Copyright 2010-2023 JetBrains s.r.o. and Kotlin Programming Language contributors.
* Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE.txt file.
*/

package kotlinx.io

@OptIn(ExperimentalStdlibApi::class)
public actual interface RawSink : AutoCloseableAlias {
public actual fun write(source: Buffer, byteCount: Long)

public actual fun flush()

actual override fun close()
}
17 changes: 17 additions & 0 deletions core/js/src/SegmentPool.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
/*
* Copyright 2010-2023 JetBrains s.r.o. and Kotlin Programming Language contributors.
* Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE.txt file.
*/

package kotlinx.io

internal actual object SegmentPool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: will JS/Native have pooling some day? If so may be it's better to create an issue for this. Or this will be implemented in scope of #135/different segment types?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will be considered after #135 (I expect to see the performance improvement on Native w/ enabled pooling, but I didn't measure it yet).

actual val MAX_SIZE: Int = 0

actual val byteCount: Int = 0

actual fun take(): Segment = Segment()

actual fun recycle(segment: Segment) {
}
}
112 changes: 112 additions & 0 deletions core/js/src/files/PathsJs.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
/*
* Copyright 2010-2023 JetBrains s.r.o. and Kotlin Programming Language contributors.
* Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE.txt file.
*/

package kotlinx.io.files

import kotlinx.io.*

private val fs: dynamic
get(): dynamic {
return try {
js("require('fs')")
} catch (t: Throwable) {
null
}
}

private val buffer: dynamic
get(): dynamic {
return try {
js("require('buffer')")
} catch (t: Throwable) {
null
}
}

public actual class Path internal constructor(private val path: String,
@Suppress("UNUSED_PARAMETER") any: Any?) {
override fun toString(): String = path
}

public actual fun Path(path: String): Path {
return Path(path, null)
}

public actual fun Path.source(): Source {
check(fs !== null) { "Module 'fs' was not found" }
return FileSource(this).buffered()
}

public actual fun Path.sink(): Sink {
check(fs !== null) { "Module 'fs' was not found" }
check(buffer !== null) { "Module 'buffer' was not found" }
return FileSink(this).buffered()
}

private class FileSource(private val path: Path) : RawSource {
private var buffer: dynamic = null
private var closed = false
private var offset = 0

@OptIn(ExperimentalUnsignedTypes::class)
override fun readAtMostTo(sink: Buffer, byteCount: Long): Long {
check(!closed) { "Source is closed." }
if (byteCount == 0L) {
return 0
}
if (buffer === null) {
buffer = fs.readFileSync(path.toString(), null)
}
val len: Int = buffer.length as Int
if (offset >= len) {
return -1L
}
val bytesToRead = minOf(byteCount, (len - offset))
for (i in 0 until bytesToRead) {
sink.writeByte(buffer.readInt8(offset++) as Byte)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Buffer::copy expects Buffer or UInt8Array as the destination, and Segment's backing array (Segment::data) is an instance of Int8Array.

I think the issue could be addressed later, after #135

}

return bytesToRead
}

override fun close() {
closed = true
}
}

private class FileSink(private val path: Path) : RawSink {
private var closed = false
private var append = false

override fun write(source: Buffer, byteCount: Long) {
check(!closed) { "Sink is closed." }
if (byteCount == 0L) {
return
}

var remainingBytes = minOf(byteCount, source.size)
while (remainingBytes > 0) {
val head = source.head!!
val segmentBytes = head.limit - head.pos
val buf = buffer.Buffer.allocUnsafe(segmentBytes)
buf.fill(head.data, head.pos, segmentBytes)
if (append) {
fs.appendFileSync(path.toString(), buf)
} else {
fs.writeFileSync(path.toString(), buf)
append = true
}

source.skip(segmentBytes.toLong())
remainingBytes -= segmentBytes
}
}

override fun flush() = Unit

override fun close() {
closed = true
}
}
56 changes: 56 additions & 0 deletions core/js/test/utils.kt
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
/*
* Copyright 2010-2023 JetBrains s.r.o. and Kotlin Programming Language contributors.
* Use of this source code is governed by the Apache 2.0 license that can be found in the LICENSE.txt file.
*/

package kotlinx.io

import kotlin.random.Random
import kotlin.random.nextULong

private val os: dynamic
get(): dynamic {
return try {
js("require('os')")
} catch (t: Throwable) {
null
}
}

private val fs: dynamic
get(): dynamic {
return try {
js("require('fs')")
} catch (t: Throwable) {
null
}
}

private val nodePath: dynamic
get(): dynamic {
return try {
js("require('path')")
} catch (t: Throwable) {
null
}
}

actual fun createTempFile(): String {
while (true) {
val tmpdir = os.tmpdir()
val filename = Random.nextULong().toString()
val fullpath = "$tmpdir${nodePath.sep}$filename.txt"

if (fs.existsSync(fullpath) as Boolean) {
continue
}
return fullpath
}
}

actual fun deleteFile(path: String) {
if (!fs.existsSync(path) as Boolean) {
throw IOException("File does not exist: $path")
}
fs.rmSync(path)
}
4 changes: 2 additions & 2 deletions core/native/src/files/PathsNative.kt
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,13 @@ public actual class Path internal constructor(internal val path: String,
public actual fun Path(path: String): Path = Path(path, null)

public actual fun Path.source(): Source {
val openFile: CPointer<FILE> = fopen(path, "r")
val openFile: CPointer<FILE> = fopen(path, "rb")
?: throw IOException("Failed to open $path with ${strerror(errno)?.toKString()}")
return FileSource(openFile).buffered()
}

public actual fun Path.sink(): Sink {
val openFile: CPointer<FILE> = fopen(path, "w")
val openFile: CPointer<FILE> = fopen(path, "wb")
?: throw IOException("Failed to open $path with ${strerror(errno)?.toKString()}")
return FileSink(openFile).buffered()
}
Expand Down
Loading