NoClassDefFoundError thrown in dotc.transform.Splicer$Interpretter.getMethod #13516
Labels
Milestone
Comments
Kordyjan
added
stat:needs info
stat:needs minimization
|
Is a code that gives you this crash a publicly available one? Having at least the way to reproduce the error would greatly improve the chances of this getting fixed. |
|
@DavidGoodenough could you try it on |
|
Will do, I keep missing compiler updates as dependencyUpdates does not report them with scala-3 (yes I have put in feature request for that) |
|
We released that one last week. I believe there I believe that there is a bug with the dependency updates that is being looked at or fixed. |
|
No, 3.0.2 does not fix the problem. Is it worth trying 3.1.0-RC1? Or should I continue trying to minimize the problem to give you something to test against? |
|
Then we will need a self-contained example (ideally minimized). |
|
OK, I have a set of 4 .scala files (and its build.sbt) which I can upload or email somewhere. Where would you like it sent? I have also identified the problem. It is in code that I am actually not going to be using any more (it is from an old version) I have left it in place so that the compiler crash can be squashed. The problem is in the the file Persistable.scala, in the serialize and deserialize methods, which take Persistance objects as arguments or generic types. This introduces a dependency loop, but nothing in the error messages with any dotty up to and including 3.0.2 is helpful in diagnosing that as the problem. |
|
You can paste the 4 files here if you want. |
|
build.sbt: lazy val root = project.in(file("."))
.settings(
// Name of the project
name := "Persist",
organization := "uk.co.dga",
// Project version
version := "1.2-SNAPSHOT",
// Version of Scala used by the project
// scalaVersion := "2.13.5"
scalaVersion := "3.0.2",
scalacOptions --= Seq("-explain-types","-explain"),
scalacOptions ++= Seq("-no-indent"),
// Add dependency on ScalaFX library
libraryDependencies += "com.lihaoyi" % "scalatags_2.13" % "0.9.4",
libraryDependencies += "io.jvm.uuid" % "scala-uuid_2.13" % "0.3.1",
libraryDependencies += "uk.co.dga" %% "formfx" % "2.0-SNAPSHOT",
libraryDependencies += "org.postgresql" % "postgresql" % "42.2.23",
libraryDependencies += "org.apache.bcel" % "bcel" % "6.5.0",
libraryDependencies += "com.h2database" % "h2-mvstore" % "1.4.200",
// Temp while is falls over if present with NullPointerException
Compile / packageDoc / publishArtifact := false,
resolvers += Resolver.sonatypeRepo("snapshots"),
// Fork a new JVM for 'run' and 'test:run', to avoid JavaFX double initialization problems
fork := true,
assembly / assemblyMergeStrategy := {
case PathList("module-info.class") => MergeStrategy.discard
case PathList("module-info.java") => MergeStrategy.discard
case x =>
val oldStrategy = (assembly / assemblyMergeStrategy).value
oldStrategy(x)
},
assembly / assemblyJarName := "persist.jar",
assembly / mainClass := Some("uk.co.dga.persist.Loader")
)
Next four files all in src/main/scala/uk/co/dga/persist Persistable.scala: package uk.co.dga.persist
import io.jvm.uuid._
import uk.co.dga.formfx.FormApp
import scala.util.Try
import java.lang.reflect.Modifier
import scala.annotation.unused
import scala.language.implicitConversions
/**
* This is the base class for all persistable classes, or rather for the root objects
* that are persistable and which are references by Reference objects or lists of
* Reference objects.
* This class must be explicitly given a SerialVersionUID
* as DatabaseHeader (which extends this) is loaded from the JAR not the database
*/
class Persistable private[persist] (private[persist] val uuid:UUID) extends Versioned {
private[persist] var lastUpdated:Long = 0L // Mark this persistable as never saved, once saved it will have a real time here
override def serialize:String = {
Seq(
serial(lastUpdated)
).mkString(s""""${getClass.getCanonicalName}" : {""",",\n","}")
}
override def restore(rhs:NodeRhs) = {
lastUpdated = rhs.get[Long]("lastUpdated")
}
}
object Persistable {
val persistCls = (new Persistable(UUID(0L,0L))).getClass
}
Persistance.scala:- package uk.co.dga.persist
import java.io.ByteArrayInputStream
import java.io.ByteArrayOutputStream
import java.io.ObjectOutputStream
import java.io.ObjectInputStream
import java.io.ObjectStreamClass
import scala.quoted.Expr
import scala.quoted.Quotes
private[persist] object Persistance {
inline def nameOf(inline expr:Any):String = ${nameOfImpl('expr)}
private def nameOfImpl(expr:Expr[Any])(using q:Quotes):Expr[String] = {
import q.reflect.*
import q.*
var name = s"""${expr.show.split('.').last}"""
Expr(name)
}
inline def tagOf(inline expr:Any):String = {
s""""${nameOf(expr)}:expr.getClass.getCanonicalName""""
}
def serialize(any:Persistable):Array[Byte] = {
???
}
def deserialize[T<:Persistable](in:Array[Byte],loader:ClassLoader):T = {
???
}
}PersistException.scala:- package uk.co.dga.persist
case class PersistException(msg:String) extends Exception { }Versionable.scala package uk.co.dga.persist
import java.nio.charset.StandardCharsets
import java.text.ParseException
import io.jvm.uuid._
import scala.BufferedIterator
import scala.util.chaining.*
import scala.annotation.tailrec
import scala.language.implicitConversions
import scala.collection.mutable.HashMap
import scala.collection.mutable.Buffer
import java.time.LocalDate
import scala.quoted.*
import scala.language.implicitConversions
import scala.annotation.compileTimeOnly
import Persistance.{nameOf,tagOf}
import scalatags.generic.TypedTag
import scala.util.Try
/**
* All Persistable class and any class that changed should extend this class. You
* should override the upgrade method to modify the stored data ready to be loaded
* into the new instances coping for any changes that are made. The version can be
* checked using the savedVersion field. If the version has not changed or is compatible
* simply return old from upgrade
*/
class Versioned {
def version = 1
private var savedVersion = version
inline def serial[T](inline t:T):String = {
inline t match {
case v:Versioned => s""""${tagOf(t)}" : ${compiletime.summonInline[Conversion[T,String]].apply(v)}"""
case bv:Buffer[Versioned] => s""""${tagOf(t)}" : ${compiletime.summonInline[Conversion[T,String]].apply(_)}.mkString("[ { "," },\n{ ","} ]")}"""
case s:String => s""""${tagOf(t)}" : $s"""
case os:Option[String] => os match {
case Some(s) => s""""${tagOf(t)}" : $s"""
case None => "" // do nothing for optional nones
}
case i:Int => s"""${tagOf(t)} : ${i.toString}"""
case l:Long => s"""${tagOf(t)} : ${l.toString}"""
case u:UUID => s"""${tagOf(t)} : ${u.toString}"""
case ld:LocalDate => s"""${tagOf(t)} : ${ld.toString}"""
case as:Array[String] => s"""${tagOf(t)} : ${as.mkString("[ \"","\",\n\"","\"")}"""
}
}
inline def serial[T](inline ot:Option[T]):String = {
ot match {
case Some(t) => serial(t)
case None => ""
}
}
def serialize:String = {
Seq(
serial(savedVersion)
).mkString(s""""${getClass.getCanonicalName}" : {""",",\n","}")
}
def restore(rhs:NodeRhs) = {
savedVersion = rhs.get[Int]("savedVersion")
}
}
/**
* Each object derived from Versioned and that is instantiated in its own right
* MUST have an apply method, taking a single argument, a NodeRhs. If the class
* has internal state then the restore method should be called, passing the NodeRhs
* as its arguement. Any arguments to the class constructor can be pulled from the
* NodeRhs using the get method, passing the devvclared type of the argument as the generic
* argument to get.
*/
object Versioned {
def apply(rhs:NodeRhs):Versioned = {
new Versioned {
restore(rhs)
}
}
def upgrade(old:Rhs):Rhs = {
old
}
}
class Lhs(val lhs:String) {
val (name,desc,num) = parse(lhs)
def parse(lhs:String) = {
val a = lhs.split(":")
if(a.size != 2) throw PersistException(s"Unable to parse LHS $lhs")
val n = a(0)
val a2 = a(1).split("#")
a2.size match {
case 1 => (n,a2(0),0)
case 2 => (n,a2(0),a2(1))
case _ => throw PersistException(s"Unable to parse LHS $lhs")
}
}
}
trait Rhs { }
class StringRhs(val lhs:Lhs,val value:String) extends Rhs {
lazy val obj = Try(
lhs.desc match {
case "scala.Int" => value.toInt
case "scala.Short" => value.toShort
case "scala.Long" => value.toLong
case "scala.Double" => value.toDouble
case "scala.Float" => value.toFloat
case "scala.math.BigDecimal" => BigDecimal(value)
case "scala.math.BigInt" => BigInt(value)
case "java.util.UUID" => UUID.fromString(value)
case "java.time.LocalDate" => LocalDate.parse(value)
case u => throw PersistException(s"Invalid type ${lhs.desc} for ${lhs.name} with $value")
}).getOrElse(throw PersistException(s"Invalid value $value for ${lhs.name}"))
def get[T](name:String):T = {
obj.asInstanceOf[T]
}
}
class NodeRhs(val lhs:Lhs,val value:Map[String,Rhs]) {
lazy val obj = (Try{
val cls = getClass.getClassLoader.loadClass(lhs.desc)
val companion = cls.getField("MODULE$").get(cls)
val applyMethod = cls.getMethod("apply",getClass)
applyMethod.invoke(cls,this)
}).getOrElse(throw PersistException(s"Invalid value $value for ${lhs.name}"))
def get[T](name:String):T = obj.asInstanceOf[T]
}
class SeqRhs(val lhs:Lhs,var value:Seq[Rhs]) {
// We are not storing tuples, so we know the elements are all of the same (base) type(T).
transparent inline def get[T](name:String)(using t:Type[T])(using Quotes):Any ={
???
}
}
object Versionable {
type NodeField = VersionNode | String | Null
}
extension(bi:scala.collection.BufferedIterator[Char]) {
@tailrec def foldUntil[A](initial:A)(proc:(A,Char)=>A)(check:(A,Char)=>Boolean):A = {
if(!bi.hasNext) initial
else {
val next = bi.next
if(!check(initial,next)) initial
else foldUntil(proc(initial,next))(proc)(check)
}
}
}
/**
* This class represents one level of the saved VersionTree.
*
* The top level object must be a Persistable, and thus contain a uuid, but that is
* not enforced here but rather when persistance happens. The object does not have a
* name as it is not retrieved by name, and it is the only place where the deserializing
* code does not know the receiving class. So the name in that case is the fully qualified
* name of the class of the object.
*
* The saved version has the name on the left followed by its ID "field"#n and on the right
* it is either a reference to an already stored object (#n), a value (a String we know how to
* store, such as a String, Numeric, dates and times etc) "value", a class which is a list of fields
* enclosed in {field} or an array of fields [value] where value is a primitive in "" or a class in {},
* separated by \n.
*
* @param V the expected type of this object
* @param name the variable name
* @param saved the external form of the value, but decoded
* @param value the contructed value of this object
*/
class VersionNode(val name:String,val values:Map[String,Versionable.NodeField]) {
def optString(name:String):Option[String] = values.get(name) match {
case Some(s) => Some(s.toString)
case None => None
}
}
object VersionNode {
val refMap = collection.mutable.HashMap[Int,Rhs]()
import VersionablePersistance._
def loadRhs(lhs:Lhs,bi:BufferedIterator[Char]):Rhs = {
nextLead(bi) match {
case '[' => loadArray(lhs,bi)
case '{' => loadNode(lhs,bi)
case '"' => StringRhs(lhs,getString(bi)).asInstanceOf[Rhs]
case '#' => refMap.get(getRef(bi).toInt).getOrElse(throw new PersistException(s"No referrence value at ${bi.toString}"))
case unknown => throw PersistException(s"""Unknown lead character $unknown, expecting [{" or # before ${bi.toString}""")
}
}
def loadArray(lhs:Lhs,bi:BufferedIterator[Char]):Rhs = { // Could be array of objects or strings or nodes or arrays
SeqRhs(lhs,bi.foldUntil(Seq[Rhs]()){(s,c)=> s :+ loadRhs(lhs,bi)}{(_,c)=>c match {
case ',' => false
case ']' => true
case unknown => throw PersistException(s"Expected ']' or ',' found '$unknown' before ${bi.toString}")
}}).asInstanceOf[Rhs]
}
/**
* Read in the next object, which should be a sequence of name : rhs pairs ending in a }
*/
def loadNode(lhs:Lhs,bi:BufferedIterator[Char]):Rhs = {
NodeRhs(lhs,bi.foldUntil(Map[String,Rhs]()){(s,c)=>
s + loadField(bi)
}{(_,c)=>c match {
case ',' => false
case '}' => true
case unknown => throw PersistException(s"Expected '}' or ',' found '$unknown' before ${bi.toString}")
}}).asInstanceOf[Rhs]
}
def loadField(bi:BufferedIterator[Char]):(String,Rhs) = {
val v = getString(bi)
nextLead(bi) match {
case ':' =>
(v , loadRhs(Lhs(v),bi))
case unknown => throw PersistException(s"Unexpected character $unknown expecting ':' rest ${bi.toString}")
}
}
}
/**
* A ResolvedVersionTree is like a VersionTree but includes the resolved value of the object.
*/
class ResolvedVersionNode[V](name:String,value:V,saved:Versionable.NodeField*) {
}
object VersionablePersistance {
def getString(iterator:BufferedIterator[Char]):String = {
val sb = StringBuilder()
var more = true
while(more) {
getChar(iterator) match {
case '\\' => sb.append(getChar(iterator)) // escapes
case '"' => more = false // end of string
case c => sb.append(c) // more token, append
}
}
sb.toString
}
def nextLead(iterator:BufferedIterator[Char]):Char = {
// step over leading whitespace
while(iterator.head.isWhitespace) { getChar(iterator) } // throw away leading strings
getChar(iterator)
}
def getRef(iterator:BufferedIterator[Char]):Int = {
var more = true
var result = 0
while(more) {
if(iterator.head.isDigit) result = (result * 10) + iterator.next - '0'
}
result
}
// If we hit the end before we expect then scream, we can not recover.
def getChar(iterator:BufferedIterator[Char]):Char = {
if(iterator.hasNext) iterator.next
throw PersistException("Premature end of string")
}
}
|
|
Minimized to: class Versioned:
def serialize: String = Persistance.nameOf(0)import scala.quoted.*
object Persistance:
inline def nameOf(inline e: Any): String = ${ nameOfImpl('e) }
private def nameOfImpl(e: Expr[Any])(using Quotes): Expr[String] = Expr("")
def foo(p: Versioned): Unit = {} |
|
That minimization is really odd. I got rid of the crash by removing the serialize and deserialize methods in Persistance.scala, so all the macros were still there and it compiled cleanly. And the serialize and deserialize methods do introduce a dependency loop (so I know they are wrong), but that should give a message saying there is a loop (and hopefully itemising it) rather than crashing. If the macro is causing an extra problem then perhaps we have two bugs. |
|
The minimized code shows that there is cyclic dependency when expanding the macro.
|
|
The same cycle exists in the original code due to inheritance. class Persistable private[persist] (private[persist] val uuid:UUID) extends Versioned |
nicolasstucki
added a commit
to dotty-staging/dotty
that referenced
this issue
Sep 14, 2021
|
True, but as serialize and deserialize do not reference nameOf, why does simply removing these two method definitions also solve the problem? I am not suggesting that the nameOf problem does not exist, and quite possibly the fix to one dependency problem might fix the other, but as I understand it macros and type resolution happen at different times in the compiler, so there might be two problems, or do both use the same resolution mechanism? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Compiler version
3.0.1
Minimized code
I am unsure where the error is, but I am using both macros and extensions in the relevant set of source files. If the compiler could tell me what it was trying to do when it fell over, rather than falling over, I might be able to narrow down the problem.
It would seem that at:-
dotty.tools.dotc.transform.Splicer$Interpreter.getMethod(Splicer.scala:414)
At that point in the code only the NoSuchMethodException is checked, not the NoClassDefFoundError. Either NoClassDefFound should have been caught earlier and this code never invoked, or an extra check is needed here.
(Note the current head does not quite match the line number, but I am using 3.0.1 rather than head).
Output (click arrow to expand)
The text was updated successfully, but these errors were encountered: