Skip to content

ClassTags: New phase which synthesises class tags. #563

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
May 15, 2015
1 change: 1 addition & 0 deletions src/dotty/tools/dotc/Compiler.scala
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ class Compiler {
new InterceptedMethods,
new Literalize,
new Getters,
new ClassTags,
new ElimByName,
new ResolveSuper),
List(new Erasure),
Expand Down
2 changes: 2 additions & 0 deletions src/dotty/tools/dotc/core/StdNames.scala
Original file line number Diff line number Diff line change
Expand Up @@ -149,8 +149,10 @@ object StdNames {
final val Seq: N = "Seq"
final val Symbol: N = "Symbol"
final val ClassTag: N = "ClassTag"
final val classTag: N = "classTag"
final val WeakTypeTag: N = "WeakTypeTag"
final val TypeTag : N = "TypeTag"
final val typeTag: N = "typeTag"
final val Expr: N = "Expr"
final val String: N = "String"
final val Annotation: N = "Annotation"
Expand Down
4 changes: 2 additions & 2 deletions src/dotty/tools/dotc/core/SymDenotations.scala
Original file line number Diff line number Diff line change
Expand Up @@ -1346,7 +1346,7 @@ object SymDenotations {
// than this is a scope that will eventually become decls of this symbol.
// And this should only happen if this is first time the scope of symbol
// is computed, ie symbol yet has no future.
assert(this.nextInRun == this)
assert(this.nextInRun.validFor.code <= this.validFor.code)
scope
case _ => unforcedDecls.openForMutations
}
Expand All @@ -1367,7 +1367,7 @@ object SymDenotations {

/** Enter a symbol in given `scope` without potentially replacing the old copy. */
def enterNoReplace(sym: Symbol, scope: MutableScope)(implicit ctx: Context): Unit = {
require((sym.denot.flagsUNSAFE is Private) || !(this is Frozen))
require((sym.denot.flagsUNSAFE is Private) || !(this is Frozen) || (scope ne this.unforcedDecls))
scope.enter(sym)

if (myMemberFingerPrint != FingerPrint.unknown)
Expand Down
65 changes: 65 additions & 0 deletions src/dotty/tools/dotc/transform/ClassTags.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
package dotty.tools.dotc
package transform

import core._
import TreeTransforms._
import Contexts.Context
import Flags._
import SymUtils._
import Symbols._
import SymDenotations._
import Types._
import Decorators._
import DenotTransformers._
import StdNames._
import NameOps._
import ast.Trees._
import dotty.tools.dotc.ast.tpd
import dotty.tools.dotc.core.Constants.Constant
import util.Positions._
import Names._
import collection.mutable

/** This phase replaces calls to DottyPredef.classTag by code that synthesizes appropriate ClassTag
*/
class ClassTags extends MiniPhaseTransform with IdentityDenotTransformer { thisTransform =>
import ast.tpd._

private var classTagCache: Symbol = null
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not make classTag, typeTag lazy vals? Seems overkill to use side-effects here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They depend on context to be initialized. Which context would those lazy vals use?

private var typeTagCache: Symbol = null
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I usually use myClassTag/myTypeTag for private variables, includes variables serving as a cache.

private var scala2ClassTagModule: Symbol = null


override def prepareForUnit(tree: tpd.Tree)(implicit ctx: Context): TreeTransform = {
val predefClass = defn.DottyPredefModule.moduleClass.asClass
classTagCache = ctx.requiredMethod(predefClass, nme.classTag)
typeTagCache = ctx.requiredMethod(predefClass, nme.typeTag)
scala2ClassTagModule = ctx.requiredModule("scala.reflect.ClassTag")
this
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you intend for Context to be eligible for garbage collection after a compilation run in a resident setting? Will caches like these prevent that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The phases themselves are only stored in context. https://github.com/lampepfl/dotty/blob/master/src/dotty/tools/dotc/core/Phases.scala

They are reallocated per-run. Currently, this will not stop GC. If it would, we can cleanup in transformUnit.

}

override def phaseName: String = "classTags"

override def transformTypeApply(tree: tpd.TypeApply)(implicit ctx: Context, info: TransformerInfo): tpd.Tree =
if (tree.fun.symbol eq classTagCache) {
val tp = tree.args.head.tpe
val defn = ctx.definitions
val (elemType, ndims) = tp match {
case defn.MultiArrayType(elem, ndims) => (elem, ndims)
case _ => (tp, 0)
}

val claz = tp.classSymbol
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will this do the right thing for:

scala> reflect.classTag[Array[Int]]
res2: scala.reflect.ClassTag[Array[Int]] = Array[int]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to get NoSymbol back here? classSymbol seems to return that for some OrTypes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for Array example, It's indeed not handled correctly.
classSymbol for OrTypes is bugged, see comment https://github.com/lampepfl/dotty/blob/master/src/dotty/tools/dotc/core/Types.scala#L308. It should not return NoSymbol.
Though this code will not work for some AndTypes.

It seems that classSymbol.orElse(classSymbols.head) should do it correctly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On Thu, May 14, 2015 at 9:07 AM, Dmitry Petrashko [email protected]
wrote:

In src/dotty/tools/dotc/transform/ClassTags.scala
#563 (comment):

  • override def prepareForUnit(tree: tpd.Tree)(implicit ctx: Context): TreeTransform = {
  • val predefClass = defn.DottyPredefModule.moduleClass.asClass
  • classTagCache = ctx.requiredMethod(predefClass, names.classTag)
  • typeTagCache = ctx.requiredMethod(predefClass, names.typeTag)
  • scala2ClassTagModule = ctx.requiredModule("scala.reflect.ClassTag")
  • this
  • }
  • override def phaseName: String = "classTags"
  • override def transformTypeApply(tree: tpd.TypeApply)(implicit ctx: Context, info: TransformerInfo): tpd.Tree =
  • if (tree.fun.symbol eq classTagCache) {
  •  val tp = tree.args.head.tpe
    
  •  val claz = tp.classSymbol
    

Thanks for Array example, It's indeed not handled correctly.
classSymbol for OrTypes is bugged, see comment
https://github.com/lampepfl/dotty/blob/master/src/dotty/tools/dotc/core/Types.scala#L308.
It should not return NoSymbol.
Though this code will not work for some AndTypes.

It seems that classSymbol.orElse(classSymbols.head) should do it
correctly.

That will return a symbol. but how do you know it's the right one? We are
missing a spec here. I think the most robust would be to take the
classSymbol of the erased type. That has a simple spec and it is likely to
be the one we will want in the end.


Reply to this email directly or view it on GitHub
https://github.com/lampepfl/dotty/pull/563/files#r30301852.

Martin Odersky
EPFL

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. Thanks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In Scala, give a value class Meter, classTag[Meter] and classOf[Meter] return Meter.class, not the underlying type.

So I think using Scala erasure might be a little bit too simple. You could instead use the java erasure, which, after my fix for @smarter's recent bug report, does not unwrap value classes.

Here's a few test cases:

object Test { 
  type T = String
  type U

  def test = {

    val a /* : Class[T]                  */ = classOf[T]                        // [Ljava/lang/String;
    val b /* : ClassTag[T]               */ = reflect.classTag[T]               // ClassTag(classOf[java.lang.String])

//  val c /* :                           */ = classOf[T with U]                 // error: class type required but Test.T with Test.U found
    val d /* : ClassTag[T with U]        */ = reflect.classTag[T with U]        // ClassTag(classOf[java.lang.String])

    val e /* : Class[Array[T with U]]    */ = classOf[Array[T with U]]          // [Ljava/lang/String;
    val f /* : ClassTag[Array[T with U]] */ = reflect.classTag[Array[T with U]] // ClassTag(arrayClass(classOf[java.lang.String]))

    val g /* : Class[Meter]              */ = classOf[Meter]                    // [LMeter;
    val h /* : ClassTag[Meter]           */ = reflect.classTag[Meter]           // ClassTag(classOf[Meter])
  }
}

class Meter(val i: Int) extends AnyVal

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

val elemClaz = elemType.classSymbol
assert(!claz.isPrimitiveValueClass) // should be inserted by typer
val elemTag = if (defn.ScalaValueClasses.contains(elemClaz) || elemClaz == defn.NothingClass || elemClaz == defn.NullClass)
ref(defn.DottyPredefModule).select(s"${elemClaz.name}ClassTag".toTermName)
else if (ValueClasses.isDerivedValueClass(elemClaz)) ref(claz.companionModule)
else if (elemClaz eq defn.AnyClass) ref(scala2ClassTagModule).select(nme.Any)
else {
val erazedTp = TypeErasure.erasure(elemType).classSymbol.typeRef
ref(scala2ClassTagModule).select(nme.apply).appliedToType(erazedTp).appliedTo(Literal(Constant(erazedTp)))
}
(1 to ndims).foldLeft(elemTag)((arr, level) => Select(arr, nme.wrap).ensureApplied).ensureConforms(tree.tpe)
} else tree
}
51 changes: 36 additions & 15 deletions src/dotty/tools/dotc/transform/ExtensionMethods.scala
Original file line number Diff line number Diff line change
Expand Up @@ -46,35 +46,56 @@ class ExtensionMethods extends MiniPhaseTransform with DenotTransformer with Ful
override def runsAfterGroupsOf = Set(classOf[FirstTransform]) // need companion objects to exist

override def transform(ref: SingleDenotation)(implicit ctx: Context): SingleDenotation = ref match {
case ref: ClassDenotation if ref is ModuleClass =>
ref.linkedClass match {
case origClass: ClassSymbol if isDerivedValueClass(origClass) =>
val cinfo = ref.classInfo
case moduleClassSym: ClassDenotation if moduleClassSym is ModuleClass =>
moduleClassSym.linkedClass match {
case valueClass: ClassSymbol if isDerivedValueClass(valueClass) =>
val cinfo = moduleClassSym.classInfo
val decls1 = cinfo.decls.cloneScope
val moduleSym = moduleClassSym.symbol.asClass

var newSuperClass: Type = null

ctx.atPhase(thisTransformer.next) { implicit ctx =>
// In Scala 2, extension methods are added before pickling so we should
// not generate them again.
if (!(origClass is Scala2x)) ctx.atPhase(thisTransformer) { implicit ctx =>
for (decl <- origClass.classInfo.decls) {
if (!(valueClass is Scala2x)) ctx.atPhase(thisTransformer) { implicit ctx =>
for (decl <- valueClass.classInfo.decls) {
if (isMethodWithExtension(decl))
decls1.enter(createExtensionMethod(decl, ref.symbol))
decls1.enter(createExtensionMethod(decl, moduleClassSym.symbol))
}
}

val sym = ref.symbol
val underlying = erasure(underlyingOfValueClass(origClass))
val evt = ErasedValueType(origClass, underlying)
val u2evtSym = ctx.newSymbol(sym, nme.U2EVT, Synthetic | Method,
val underlying = erasure(underlyingOfValueClass(valueClass))
val evt = ErasedValueType(valueClass, underlying)
val u2evtSym = ctx.newSymbol(moduleSym, nme.U2EVT, Synthetic | Method,
MethodType(List(nme.x_0), List(underlying), evt))
val evt2uSym = ctx.newSymbol(sym, nme.EVT2U, Synthetic | Method,
val evt2uSym = ctx.newSymbol(moduleSym, nme.EVT2U, Synthetic | Method,
MethodType(List(nme.x_0), List(evt), underlying))

val defn = ctx.definitions

val underlyingCls = underlying.classSymbol
val underlyingClsName =
if (defn.ScalaNumericValueClasses.contains(underlyingCls) ||
underlyingCls == defn.BooleanClass) underlyingCls.name else nme.Object


val syp = ctx.requiredClass(s"dotty.runtime.vc.VC${underlyingClsName}Companion").asClass

newSuperClass = tpd.ref(syp).select(nme.CONSTRUCTOR).appliedToType(valueClass.typeRef).tpe.resultType

decls1.enter(u2evtSym)
decls1.enter(evt2uSym)
}
if (decls1.isEmpty) ref
else ref.copySymDenotation(info = cinfo.derivedClassInfo(decls = decls1))

// add a VCXXXCompanion superclass

moduleClassSym.copySymDenotation(info =
cinfo.derivedClassInfo(
classParents = ctx.normalizeToClassRefs(List(newSuperClass), moduleSym, decls1),
decls = decls1))
case _ =>
ref
moduleClassSym
}
case ref: SymDenotation
if isMethodWithExtension(ref) && ref.hasAnnotation(defn.TailrecAnnotationClass) =>
Expand Down