TastyInspector.inspectTastyFilesInJar only inspects top-level tastys

[jodersky]

Compiler version

3.0.0

Minimized code

See this example project https://github.com/jodersky/tastyjar.

The gist of it is that given a jar file, TastyInspector.inspectTastyFilesInJar will only ever read .tasty files that are top-level. For instance, assume a jar with content:

a.tasty
pkg/b.tasty

then only a.tasty will be inspected, and pkg/b.tasty is silently ignored.

Expectation

All tasty files in a jar should be inspected.

Notes

This issue is probably the same underlying cause for #11696

[jodersky]

cc @nicolasstucki

[jodersky]

FYI, in case anyone else comes across this before it's fixed, here's the ugly workaround that we use at my organization:

import java.util.zip
import scala.collection.mutable

// extract all .tasty jars in a jar to a directory
def extractTastys(destDir: os.Path, jar: os.Path): Seq[os.Path] = {
  val paths = collection.mutable.ListBuffer.empty[os.Path]
  val zin = new java.util.zip.ZipInputStream(os.read.inputStream(jar))
  try {
    var entry: zip.ZipEntry = null
    while {
      entry = zin.getNextEntry()
      entry != null
    } do {
      if (entry.getName.endsWith(".tasty")) {
        val path = destDir / os.RelPath(entry.getName)
        val out = os.write.over.outputStream(path, createFolders = true)
        try {
          val buffer = new Array[Byte](8192)
          var count = 0
          while {
            count = zin.read(buffer, 0, buffer.length)
            count != -1 // as per javadoc, -1 is returned when the zip input stream reaches the end of an entry
          } do {
            out.write(buffer, 0, count)
          }
        } finally {
          out.close()
        }
        paths += path
      }
    }
  } finally {
    zin.close()
  }
  paths.result()
}

// WORKAROUND: the tasty inspector's built-in jar reader does not read deep
// tasty files in jars, so we need to work around it with temporary file
// extraction.
//
// See: https://github.com/lampepfl/dotty/issues/11696
def inspectAllJars(jars: Seq[os.Path], inspector: ins.Inspector) = {
  val tmp = os.temp.dir()
  try {
    val paths = collection.mutable.ListBuffer.empty[os.Path]
    for (path <- jars) {
      paths ++= extractTastys(tmp, path)
    }

    ins.TastyInspector.inspectAllTastyFiles(
      tastyFiles = paths.toList.map(_.toString),
      jars = Nil,
      dependenciesClasspath = jars.map(_.toString).toList
    )(inspector)
  } finally {
    os.remove.all(tmp)
  }
}

It's not ideal, but maybe it can help someone out at a pinch

[romanowski]

In scaladoc we are using OldTastyInspector.inspectFilesInContext to process jars without any problems so it may be another workaround.