Skip to content

Add/unpickling #394

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 118 commits into from
Mar 18, 2015
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
118 commits
Select commit Hold shift + click to select a range
3f5d15d
Fix desugaring of refined types with "&"-parent.
odersky Feb 12, 2015
61cb51a
Disallow refinements of types or methods that do not appear in parent.
odersky Feb 12, 2015
340dc52
First prototype of pickler.
odersky Feb 8, 2015
0c755d2
Allow up to 64 phases.
odersky Feb 10, 2015
df404e5
Make bestFit work for partially filled arrays
odersky Feb 10, 2015
41922c1
Treat <root> as static
odersky Feb 10, 2015
1c5f3b7
Add TASTY readers and printers for TASTy info.
odersky Feb 10, 2015
f9eaa3e
Avoid pickling for Java-parsed compilation units.
odersky Feb 12, 2015
4f80478
Eliminate raw numbers from Tasy format
odersky Feb 12, 2015
1026672
Changes to Tasty format:
odersky Feb 12, 2015
c28bf8d
Turn on full compression of Tasty tree buffers.
odersky Feb 12, 2015
ecb7582
Turn on pickling.
odersky Feb 12, 2015
5ae8b12
Make pickling output printer-dependent.
odersky Feb 12, 2015
0ed27cc
Fix serialization of Bind-defined types in type patterns.
odersky Feb 12, 2015
85e9922
Make Definitions#rootPackage a root.
odersky Feb 12, 2015
b1cded3
Change scheme of translating array creations new Array(...)
odersky Feb 12, 2015
32892db
Tightening of orphans tests.
odersky Feb 13, 2015
93747cd
Stop type inference from creating oprphans.
odersky Feb 13, 2015
c078ad3
Tweaks in comments
odersky Feb 13, 2015
2031d75
Weaks to format
odersky Feb 13, 2015
9262d47
Add auxiliary constructor for TastyPrinter
odersky Feb 14, 2015
5b63106
Make some tree fields lazy
odersky Feb 14, 2015
f922a46
Moved part of computation of types of DefDefs from Namer to TypeOps
odersky Feb 19, 2015
ed986b5
Add INSUPERCALL flag to pickle format.
odersky Feb 19, 2015
30f08b0
Changes to pickling annotations
odersky Feb 19, 2015
82f1ac6
Polishings on TastyReader
odersky Feb 19, 2015
64beb11
Change order of fields in PickleFormat
odersky Feb 20, 2015
1daa94f
Move more functionality from Namer proper to context ops.
odersky Feb 21, 2015
89c00f6
Add signed ints in Tasty format.
odersky Feb 23, 2015
b32244b
Add UUID to Tasty
odersky Feb 23, 2015
4115eda
First version of Unpickler
odersky Feb 23, 2015
8fea2ef
Record pickled trees in a hashmap
odersky Feb 23, 2015
f16b12c
First version of position pickler.
odersky Feb 23, 2015
499aeaf
Add hooks for unpickling positions to trees.
odersky Feb 23, 2015
92976de
Refactor traversals to be in Edge instead of PositionPickler.
odersky Feb 23, 2015
8800adb
Fix implementation of readEnd
odersky Feb 23, 2015
aebc626
First version of unpickler for Positions
odersky Feb 23, 2015
5d09a0c
Have pkg intsead of static external references
odersky Feb 24, 2015
adac8f7
Save pickled bytes in compilation unit
odersky Feb 24, 2015
757f3d5
Fixed bugs related to Unpickling.
odersky Feb 26, 2015
afeb331
Various fixes to unpickling
odersky Feb 25, 2015
1b301e9
New scheme for recording positions
odersky Feb 25, 2015
bfa5e46
Halfway to yet another scheme for handling positions
odersky Feb 26, 2015
76bf36d
Handle ParsedTry nodes in RefinedPrinter
odersky Feb 26, 2015
21f042e
Avoid capturing context in lazy trees
odersky Feb 26, 2015
252ed17
Bugfix: Avoid importing constructors
odersky Feb 26, 2015
af65672
Bugfix: Take account of asSeenFrom in matchingDenotation
odersky Feb 26, 2015
9069614
Companion objects of abstract case classes are not functions.
odersky Feb 26, 2015
66e9c7e
Finished new position unpickling code.
odersky Feb 26, 2015
c669d08
Added pickling part of new scheme.
odersky Feb 26, 2015
471881d
Ensure that start position is <= end position in Parser
odersky Feb 27, 2015
43a03c3
Added testing hooks for unpickler
odersky Feb 27, 2015
e837105
Tweaks to printing
odersky Feb 27, 2015
a0c2d6c
Set NoInits also for non-trait classes
odersky Feb 27, 2015
60e520c
Pickle trait flag and compute PureInterface and NoInits in unpickler.
odersky Feb 27, 2015
62df23e
Align PickleFormat with doc spec
odersky Feb 27, 2015
42333ce
Don't suppress ambiguous implicit notes.
odersky Feb 28, 2015
dd5be32
Fix implicit problem in RefinedPrinter.
odersky Feb 28, 2015
a543ed1
Bugfixes in unpickling
odersky Mar 1, 2015
f2cc1a1
Run unpickler tests in next run.
odersky Mar 1, 2015
eabef58
Tweaks to printing
odersky Mar 1, 2015
2009df2
Pass the correct context down in tree accumulators.
odersky Mar 1, 2015
9adaf25
Print templates again in sugared form.
odersky Mar 2, 2015
7bd12db
Avoid escaping pattern bound variables
odersky Mar 2, 2015
2b4236e
Fix problems in avoid
odersky Mar 2, 2015
fb72eaf
Fix variance of type prefixes in TypeMap and TypeAccumulator
odersky Mar 2, 2015
8f3e327
Avoiding dependent method types in closures
odersky Mar 2, 2015
6801850
More careful determination of MethodType#isDependent
odersky Mar 2, 2015
6dc3d62
Avoid dependent methods being closures.
odersky Mar 2, 2015
6220d5c
Better tracking of unhygienic closure types
odersky Mar 3, 2015
b0d7380
Better tracking of unhygienic closure types
odersky Mar 3, 2015
6022078
Keeping track of unpickling definition order in Pickler.
odersky Mar 3, 2015
fe48a0c
Change handling of roots when unpickling
odersky Mar 4, 2015
de11f33
Tweaks to printing
odersky Mar 4, 2015
9d71cb6
Avoid annotations being dropped by stripTypeVar
odersky Mar 4, 2015
e926f31
Various fixes to PickleFormat, pickler and unpickler
odersky Mar 4, 2015
96fbd7b
Allow several units to be pickle-tested at once.
odersky Mar 4, 2015
a4b2a67
Changes to Tasty format
odersky Mar 4, 2015
7fd242f
More fixes to pickling
odersky Mar 5, 2015
3215639
Compute PureInterface flag after pickling.
odersky Mar 5, 2015
cf79474
Fix TastyReader#readLongInt
odersky Mar 5, 2015
3823f29
Fix tricky problem with re-establishing denotations of selftypes
odersky Mar 5, 2015
35d1160
More tests
odersky Mar 5, 2015
4c80f89
Tweak to printing Typevars
odersky Mar 5, 2015
c6bba9b
More fixes to unpickling
odersky Mar 5, 2015
073449e
Systematic treatement of homogenized views in printing
odersky Mar 6, 2015
aa56083
Revert of interpolation decision when generating APPLIEDTYPEs
odersky Mar 6, 2015
8d370d4
Deal gracefully with out-of-scope references wehn unpickling
odersky Mar 6, 2015
8e57bc6
Harmonize parameterless constructors between unpickling and namer
odersky Mar 6, 2015
655555e
Avoid treating setters as class parameters when pickling
odersky Mar 6, 2015
a4e1ba7
Don't print implicit for classes in homogenized view
odersky Mar 6, 2015
52c586f
Refinements to pickling testing framework
odersky Mar 6, 2015
2863af2
Avoid print-types being on by default in testing
odersky Mar 7, 2015
ac47d1c
Re-org of pickling framework to enter before read
odersky Mar 7, 2015
1cd7d00
Add pickling tests
odersky Mar 7, 2015
3bcab09
Fix to pickling literals
odersky Mar 8, 2015
a7a8563
Show compilation unit which failed pickling in tests.
odersky Mar 8, 2015
f1ffa4c
Rename inheritedName -> shadowedName
odersky Mar 8, 2015
5a43cb1
Pickle shadowed names
odersky Mar 8, 2015
3d987df
Fix unpickling parameter aliases
odersky Mar 8, 2015
78ef6bb
Harmonize treatment of simplified between typer and unpickler
odersky Mar 8, 2015
d572ec0
Pickle signatures using qualified names.
odersky Mar 8, 2015
53a1230
Drop choice of separator in expanded name.
odersky Mar 8, 2015
bb5762a
Further harmonization between typing and unpickling
odersky Mar 8, 2015
31a90b4
Homogenize package ids and literals
odersky Mar 8, 2015
26babc3
Homogenize printing of super calls.
odersky Mar 8, 2015
90b05b4
Two more tweaks to make pickling invariant under printing
odersky Mar 8, 2015
19bfc0c
Remove Throw from pickling
odersky Mar 8, 2015
5962a93
Pickling test reorg
odersky Mar 8, 2015
83e7b23
Simplifications to TASTy format
odersky Mar 9, 2015
8c6339b
Add support of Shadowed names to TastyPrinter
odersky Mar 10, 2015
79e2228
Change to PickleFormat
odersky Mar 10, 2015
cb86676
Drop environment from pickled closures.
odersky Mar 10, 2015
cc65c3a
Simplifications to pickled flags
odersky Mar 11, 2015
f7f10c8
Added missing patch to DefaultInit drop.
odersky Mar 11, 2015
1e0856b
Tweaks to pickle format
odersky Mar 12, 2015
8a99155
Bringing back fix to TreeMap.transform(ValDef) that was lost during r…
DarkDimius Mar 18, 2015
0afbd6c
Fixing conflicts between #361 and #394
DarkDimius Mar 18, 2015
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/dotty/tools/backend/jvm/DottyBackendInterface.scala
Original file line number Diff line number Diff line change
Expand Up @@ -295,7 +295,7 @@ class DottyBackendInterface()(implicit ctx: Context) extends BackendInterface{
val t = new TreeTraverser {
var outerRhs: Tree = tree

def traverse(tree: tpd.Tree): Unit = tree match {
def traverse(tree: tpd.Tree)(implicit ctx: Context): Unit = tree match {
case t: DefDef =>
if (t.symbol is Flags.Label)
res.put(outerRhs, t :: res.getOrElse(outerRhs, Nil))
Expand Down
2 changes: 2 additions & 0 deletions src/dotty/tools/dotc/CompilationUnit.scala
Original file line number Diff line number Diff line change
Expand Up @@ -13,4 +13,6 @@ class CompilationUnit(val source: SourceFile) {
var tpdTree: tpd.Tree = tpd.EmptyTree

def isJava = source.file.name.endsWith(".java")

var pickled: Array[Byte] = Array()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As some phases could want to run after pickler and add additional information:

  • Maybe ArrayBuffer? Though I'd prefer to not allow to mutate already pickled info, and export TastyPickler itself. This will allow to add new sections easily.
  • and an addrOfTree: Tree => Option[Addr] for them to correlate with already pickled info

This will also allow to make PositionPickler a separate phase.
@odersky If you agree, I'll do this in a separate PR

}
4 changes: 2 additions & 2 deletions src/dotty/tools/dotc/Compiler.scala
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,10 @@ class Compiler {
List(new FirstTransform,
new SyntheticMethods),
List(new SuperAccessors),
//List(new Pickler), // Pickler needs to come last in a group since it should not pickle trees generated later
List(new Pickler), // Pickler needs to come last in a group since it should not pickle trees generated later
List(new RefChecks,
new ElimRepeated,
new ElimLocals,
new NormalizeFlags,
new ExtensionMethods),
List(new TailRec), // TailRec needs to be in its own group for now.
// Otherwise it produces -Ycheck incorrect code for
Expand Down
32 changes: 18 additions & 14 deletions src/dotty/tools/dotc/ast/Desugar.scala
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ object desugar {

/** The expansion of a class definition. See inline comments for what is involved */
def classDef(cdef: TypeDef)(implicit ctx: Context): Tree = {
val TypeDef(name, impl @ Template(constr0, parents, self, body)) = cdef
val TypeDef(name, impl @ Template(constr0, parents, self, _)) = cdef
val mods = cdef.mods

val (constr1, defaultGetters) = defDef(constr0, isPrimaryConstructor = true) match {
Expand All @@ -242,7 +242,7 @@ object desugar {
val constr = cpy.DefDef(constr1)(tparams = constrTparams, vparamss = constrVparamss)

// Add constructor type parameters to auxiliary constructors
val normalizedBody = body map {
val normalizedBody = impl.body map {
case ddef: DefDef if ddef.name.isConstructorName =>
cpy.DefDef(ddef)(tparams = constrTparams)
case stat =>
Expand Down Expand Up @@ -342,7 +342,10 @@ object desugar {
val companions =
if (mods is Case) {
val parent =
if (constrTparams.nonEmpty || constrVparamss.length > 1) anyRef
if (constrTparams.nonEmpty ||
constrVparamss.length > 1 ||
mods.is(Abstract) ||
constr.mods.is(Private)) anyRef
// todo: also use anyRef if constructor has a dependent method type (or rule that out)!
else (constrVparamss :\ classTypeRef) ((vparams, restpe) => Function(vparams map (_.tpt), restpe))
val applyMeths =
Expand Down Expand Up @@ -425,10 +428,10 @@ object desugar {
val modul = ValDef(name, clsRef, New(clsRef, Nil))
.withMods(mods | ModuleCreationFlags)
.withPos(mdef.pos)
val ValDef(selfName, selfTpt, selfRhs) = tmpl.self
val ValDef(selfName, selfTpt, _) = tmpl.self
val selfMods = tmpl.self.mods
if (!selfTpt.isEmpty) ctx.error("object definition may not have a self type", tmpl.self.pos)
val clsSelf = ValDef(selfName, SingletonTypeTree(Ident(name)), selfRhs)
val clsSelf = ValDef(selfName, SingletonTypeTree(Ident(name)), tmpl.self.rhs)
.withMods(selfMods)
.withPos(tmpl.self.pos orElse tmpl.pos.startPos)
val clsTmpl = cpy.Template(tmpl)(self = clsSelf, body = tmpl.body)
Expand Down Expand Up @@ -864,18 +867,19 @@ object desugar {
* @param parentType The type of `parent`
*/
def refinedTypeToClass(parent: tpd.Tree, refinements: List[Tree])(implicit ctx: Context): TypeDef = {
def stripToCore(tp: Type): Type = tp match {
case tp: RefinedType if tp.argInfos.nonEmpty => tp // parameterized class type
case tp: TypeRef if tp.symbol.isClass => tp // monomorphic class type
def stripToCore(tp: Type): List[Type] = tp match {
case tp: RefinedType if tp.argInfos.nonEmpty => tp :: Nil // parameterized class type
case tp: TypeRef if tp.symbol.isClass => tp :: Nil // monomorphic class type
case tp: TypeProxy => stripToCore(tp.underlying)
case _ => defn.AnyType
case AndType(tp1, tp2) => stripToCore(tp1) ::: stripToCore(tp2)
case _ => defn.AnyType :: Nil
}
val parentCore = stripToCore(parent.tpe)
val parentCores = stripToCore(parent.tpe)
val untpdParent = TypedSplice(parent)
val (classParent, self) =
if (parent.tpe eq parentCore) (untpdParent, EmptyValDef)
else (TypeTree(parentCore), ValDef(nme.WILDCARD, untpdParent, EmptyTree))
val impl = Template(emptyConstructor, classParent :: Nil, self, refinements)
val (classParents, self) =
if (parentCores.length == 1 && (parent.tpe eq parentCores.head)) (untpdParent :: Nil, EmptyValDef)
else (parentCores map TypeTree, ValDef(nme.WILDCARD, untpdParent, EmptyTree))
val impl = Template(emptyConstructor, classParents, self, refinements)
TypeDef(tpnme.REFINE_CLASS, impl).withFlags(Trait)
}

Expand Down
6 changes: 6 additions & 0 deletions src/dotty/tools/dotc/ast/Positioned.scala
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,12 @@ abstract class Positioned extends DotClass with Product {
*/
def addPos(pos: Position): this.type = withPos(pos union this.pos)

/** Set position of this tree only, without performing
* any checks of consistency with - or updates of - other positions.
* Called from Unpickler when entering positions.
*/
private[dotc] def setPosUnchecked(pos: Position) = curPos = pos

/** If any children of this node do not have positions, set them to the given position,
* and transitively visit their children.
*/
Expand Down
34 changes: 15 additions & 19 deletions src/dotty/tools/dotc/ast/TreeInfo.scala
Original file line number Diff line number Diff line change
Expand Up @@ -24,20 +24,14 @@ trait TreeInfo[T >: Untyped <: Type] { self: Trees.Instance[T] =>
case _ => false
}

/** Is tree legal as a member definition of an interface?
/** Does tree contain an initialization part when seen as a member of a class or trait?
*/
def isPureInterfaceMember(tree: Tree): Boolean = unsplice(tree) match {
def isNoInitMember(tree: Tree): Boolean = unsplice(tree) match {
case EmptyTree | Import(_, _) | TypeDef(_, _) => true
case DefDef(_, _, _, _, rhs) => rhs.isEmpty
case ValDef(_, _, rhs) => rhs.isEmpty
case tree: ValDef => tree.unforcedRhs == EmptyTree
case _ => false
}

/** Is tree legal as a member definition of a no-init trait?
*/
def isNoInitMember(tree: Tree): Boolean =
isPureInterfaceMember(tree) || unsplice(tree).isInstanceOf[DefDef]

def isOpAssign(tree: Tree) = unsplice(tree) match {
case Apply(fn, _ :: Nil) =>
unsplice(fn) match {
Expand Down Expand Up @@ -90,8 +84,8 @@ trait TreeInfo[T >: Untyped <: Type] { self: Trees.Instance[T] =>
}

/** If tree is a closure, it's body, otherwise tree itself */
def closureBody(tree: tpd.Tree): tpd.Tree = tree match {
case Block(DefDef(nme.ANON_FUN, _, _, _, rhs) :: Nil, Closure(_, _, _)) => rhs
def closureBody(tree: tpd.Tree)(implicit ctx: Context): tpd.Tree = tree match {
case Block((meth @ DefDef(nme.ANON_FUN, _, _, _, _)) :: Nil, Closure(_, _, _)) => meth.rhs
case _ => tree
}

Expand Down Expand Up @@ -244,12 +238,14 @@ trait TreeInfo[T >: Untyped <: Type] { self: Trees.Instance[T] =>
/** Is this case guarded? */
def isGuardedCase(cdef: CaseDef) = cdef.guard ne EmptyTree

/** True iff definition if a val or def with no right-hand-side, or it
/** True iff definition is a val or def with no right-hand-side, or it
* is an abstract typoe declaration
*/
def lacksDefinition(mdef: MemberDef)(implicit ctx: Context) = mdef match {
case mdef: ValOrDefDef => mdef.rhs.isEmpty && !mdef.name.isConstructorName && !mdef.mods.is(ParamAccessor)
case mdef: TypeDef => mdef.rhs.isEmpty || mdef.rhs.isInstanceOf[TypeBoundsTree]
case mdef: ValOrDefDef =>
mdef.unforcedRhs == EmptyTree && !mdef.name.isConstructorName && !mdef.mods.is(ParamAccessor)
case mdef: TypeDef =>
mdef.rhs.isEmpty || mdef.rhs.isInstanceOf[TypeBoundsTree]
case _ => false
}

Expand Down Expand Up @@ -287,8 +283,8 @@ trait TypedTreeInfo extends TreeInfo[Type] { self: Trees.Instance[Type] =>
| Import(_, _)
| DefDef(_, _, _, _, _) =>
Pure
case vdef @ ValDef(_, _, rhs) =>
if (vdef.mods is Mutable) Impure else exprPurity(rhs)
case vdef @ ValDef(_, _, _) =>
if (vdef.mods is Mutable) Impure else exprPurity(vdef.rhs)
case _ =>
Impure
}
Expand Down Expand Up @@ -410,7 +406,7 @@ trait TypedTreeInfo extends TreeInfo[Type] { self: Trees.Instance[Type] =>
/** The variables defined by a pattern, in reverse order of their appearance. */
def patVars(tree: Tree)(implicit ctx: Context): List[Symbol] = {
val acc = new TreeAccumulator[List[Symbol]] {
def apply(syms: List[Symbol], tree: Tree) = tree match {
def apply(syms: List[Symbol], tree: Tree)(implicit ctx: Context) = tree match {
case Bind(_, body) => apply(tree.symbol :: syms, body)
case _ => foldOver(syms, tree)
}
Expand Down Expand Up @@ -452,7 +448,7 @@ trait TypedTreeInfo extends TreeInfo[Type] { self: Trees.Instance[Type] =>
def defPath(sym: Symbol, root: Tree)(implicit ctx: Context): List[Tree] = ctx.debugTraceIndented(s"defpath($sym with position ${sym.pos}, ${root.show})") {
require(sym.pos.exists)
object accum extends TreeAccumulator[List[Tree]] {
def apply(x: List[Tree], tree: Tree): List[Tree] = {
def apply(x: List[Tree], tree: Tree)(implicit ctx: Context): List[Tree] = {
if (tree.envelope.contains(sym.pos))
if (definedSym(tree) == sym) tree :: x
else {
Expand All @@ -478,7 +474,7 @@ trait TypedTreeInfo extends TreeInfo[Type] { self: Trees.Instance[Type] =>
if (stats exists (definedSym(_) == sym)) stats else Nil
encl match {
case Block(stats, _) => verify(stats)
case Template(_, _, _, stats) => verify(stats)
case encl: Template => verify(encl.body)
case PackageDef(_, stats) => verify(stats)
case _ => Nil
}
Expand Down
8 changes: 4 additions & 4 deletions src/dotty/tools/dotc/ast/TreeTypeMap.scala
Original file line number Diff line number Diff line change
Expand Up @@ -78,22 +78,22 @@ final class TreeTypeMap(
}

override def transform(tree: tpd.Tree)(implicit ctx: Context): tpd.Tree = treeMap(tree) match {
case impl @ Template(constr, parents, self, body) =>
case impl @ Template(constr, parents, self, _) =>
val tmap = withMappedSyms(localSyms(impl :: self :: Nil))
cpy.Template(impl)(
constr = tmap.transformSub(constr),
parents = parents mapconserve transform,
self = tmap.transformSub(self),
body = body mapconserve tmap.transform
body = impl.body mapconserve tmap.transform
).withType(tmap.mapType(impl.tpe))
case tree1 =>
tree1.withType(mapType(tree1.tpe)) match {
case id: Ident if tpd.needsSelect(id.tpe) =>
ref(id.tpe.asInstanceOf[TermRef]).withPos(id.pos)
case ddef @ DefDef(name, tparams, vparamss, tpt, rhs) =>
case ddef @ DefDef(name, tparams, vparamss, tpt, _) =>
val (tmap1, tparams1) = transformDefs(ddef.tparams)
val (tmap2, vparamss1) = tmap1.transformVParamss(vparamss)
cpy.DefDef(ddef)(name, tparams1, vparamss1, tmap2.transform(tpt), tmap2.transform(rhs))
cpy.DefDef(ddef)(name, tparams1, vparamss1, tmap2.transform(tpt), tmap2.transform(ddef.rhs))
case blk @ Block(stats, expr) =>
val (tmap1, stats1) = transformDefs(stats)
val expr1 = tmap1.transform(expr)
Expand Down
Loading