Skip to content

NPE when inlining a macro that uses a by-name parameter defining a sealed trait #10542

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
hmemcpy opened this issue Nov 28, 2020 · 5 comments · Fixed by #10912
Closed

NPE when inlining a macro that uses a by-name parameter defining a sealed trait #10542

hmemcpy opened this issue Nov 28, 2020 · 5 comments · Fixed by #10912
Assignees
Milestone

Comments

@hmemcpy
Copy link

hmemcpy commented Nov 28, 2020

I found an issue when porting a macro to Scala 3 (using 3.0.0-M1). Took me a while to find it (see it on Scastie), but I managed to narrow it down to:

Minimized code

trait Foo {
  inline def foo[A](t: => A): Unit = ()
}

object Bar extends Foo

object Test {
  Bar.foo {
    sealed trait T1
    case object S1 extends T1
  }
}

Attempting to compile this, gives me an NPE with the following stacktrace:

Output (click arrow to expand)

java.lang.NullPointerException
	at dotty.tools.dotc.core.Symbols$Symbol.computeDenot(Symbols.scala:113)
	at dotty.tools.dotc.core.Symbols$Symbol.denot(Symbols.scala:106)
	at dotty.tools.dotc.core.Symbols$Symbol.isType(Symbols.scala:154)
	at dotty.tools.dotc.core.Types$NamedType$.isType(Types.scala:2563)
	at dotty.tools.dotc.core.Types$NamedType$.apply(Types.scala:2567)
	at dotty.tools.dotc.core.Types$Type.select(Types.scala:1455)
	at dotty.tools.dotc.core.Substituters$.substSym(Substituters.scala:83)
	at dotty.tools.dotc.core.Substituters$SubstSymMap.apply(Substituters.scala:181)
	at dotty.tools.dotc.core.Types$TypeMap.op$proxy11$1(Types.scala:5034)
	at dotty.tools.dotc.core.Types$TypeMap.mapArgs(Types.scala:5034)
	at dotty.tools.dotc.core.Types$TypeMap.mapOver(Types.scala:5069)
	at dotty.tools.dotc.core.Substituters$.substSym(Substituters.scala:103)
	at dotty.tools.dotc.core.Types$Type.substSym(Types.scala:1635)
	at dotty.tools.dotc.ast.TreeTypeMap.mapType(TreeTypeMap.scala:63)
	at dotty.tools.dotc.ast.TreeTypeMap.transform(TreeTypeMap.scala:90)
	at dotty.tools.dotc.ast.TreeTypeMap.apply(TreeTypeMap.scala:149)
	at dotty.tools.dotc.ast.TreeTypeMap.apply(TreeTypeMap.scala:151)
	at dotty.tools.dotc.core.Symbols$$anon$2.complete$$anonfun$1(Symbols.scala:820)
	at scala.collection.immutable.List.mapConserve(List.scala:472)
	at dotty.tools.dotc.core.Symbols$$anon$2.complete(Symbols.scala:820)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.completeFrom(SymDenotations.scala:166)
	at dotty.tools.dotc.core.Denotations$Denotation.completeInfo$1(Denotations.scala:188)
	at dotty.tools.dotc.core.Denotations$Denotation.info(Denotations.scala:190)
	at dotty.tools.dotc.core.SymDenotations$SymDenotation.ensureCompleted(SymDenotations.scala:368)
	at dotty.tools.dotc.core.SymDenotations$ClassDenotation.registeredCompanion(SymDenotations.scala:2147)
	at dotty.tools.dotc.core.Symbols$.mapSymbols$$anonfun$2(Symbols.scala:832)
	at dotty.runtime.function.JProcedure2.apply(JProcedure2.java:15)
	at dotty.runtime.function.JProcedure2.apply(JProcedure2.java:10)
	at scala.collection.LazyZip2.foreach(LazyZipOps.scala:112)
	at dotty.tools.dotc.core.Symbols$.mapSymbols(Symbols.scala:832)
	at dotty.tools.dotc.ast.TreeTypeMap.withMappedSyms(TreeTypeMap.scala:180)
	at dotty.tools.dotc.ast.TreeTypeMap.transformDefs(TreeTypeMap.scala:136)
	at dotty.tools.dotc.ast.TreeTypeMap.transform(TreeTypeMap.scala:107)
	at dotty.tools.dotc.ast.TreeTypeMap.apply(TreeTypeMap.scala:149)
	at dotty.tools.dotc.ast.tpd$TreeOps$.loop$1(tpd.scala:807)
	at dotty.tools.dotc.ast.tpd$TreeOps$.changeOwner$extension(tpd.scala:808)
	at dotty.tools.dotc.typer.Inliner.dotty$tools$dotc$typer$Inliner$$paramBindingDef(Inliner.scala:420)
	at dotty.tools.dotc.typer.Inliner.computeParamBindings$$anonfun$3(Inliner.scala:447)
	at dotty.runtime.function.JProcedure3.apply(JProcedure3.java:15)
	at dotty.runtime.function.JProcedure3.apply(JProcedure3.java:10)
	at scala.collection.LazyZip3.foreach(LazyZipOps.scala:248)
	at dotty.tools.dotc.typer.Inliner.computeParamBindings(Inliner.scala:448)
	at dotty.tools.dotc.typer.Inliner.inlined(Inliner.scala:641)
	at dotty.tools.dotc.typer.Inliner$.inlineCall(Inliner.scala:125)
	at dotty.tools.dotc.typer.Typer.adaptNoArgsOther$4(Typer.scala:3257)
	at dotty.tools.dotc.typer.Typer.adaptNoArgs$1(Typer.scala:3354)
	at dotty.tools.dotc.typer.Typer.adapt1(Typer.scala:3581)
	at dotty.tools.dotc.typer.Typer.adapt(Typer.scala:2920)
	at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2589)
	at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2593)
	at dotty.tools.dotc.typer.Typer.traverse$1(Typer.scala:2641)
	at dotty.tools.dotc.typer.Typer.typedStats(Typer.scala:2664)
	at dotty.tools.dotc.typer.Typer.typedClassDef(Typer.scala:2120)
	at dotty.tools.dotc.typer.Typer.typedTypeOrClassDef$2(Typer.scala:2447)
	at dotty.tools.dotc.typer.Typer.typedNamed$1(Typer.scala:2451)
	at dotty.tools.dotc.typer.Typer.typedUnadapted(Typer.scala:2519)
	at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2589)
	at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2593)
	at dotty.tools.dotc.typer.Typer.traverse$1(Typer.scala:2615)
	at dotty.tools.dotc.typer.Typer.typedStats(Typer.scala:2664)
	at dotty.tools.dotc.typer.Typer.typedPackageDef(Typer.scala:2245)
	at dotty.tools.dotc.typer.Typer.typedUnnamed$1(Typer.scala:2491)
	at dotty.tools.dotc.typer.Typer.typedUnadapted(Typer.scala:2520)
	at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2589)
	at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2593)
	at dotty.tools.dotc.typer.Typer.traverse$1(Typer.scala:2641)
	at dotty.tools.dotc.typer.Typer.typedStats(Typer.scala:2664)
	at dotty.tools.dotc.typer.Typer.typedPackageDef(Typer.scala:2245)
	at dotty.tools.dotc.typer.Typer.typedUnnamed$1(Typer.scala:2491)
	at dotty.tools.dotc.typer.Typer.typedUnadapted(Typer.scala:2520)
	at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2589)
	at dotty.tools.dotc.typer.Typer.typed(Typer.scala:2593)
	at dotty.tools.dotc.typer.Typer.typedExpr(Typer.scala:2708)
	at dotty.tools.dotc.typer.FrontEnd.liftedTree1$1(FrontEnd.scala:79)
	at dotty.tools.dotc.typer.FrontEnd.typeCheck$$anonfun$1(FrontEnd.scala:84)
	at dotty.runtime.function.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
	at dotty.tools.dotc.typer.FrontEnd.monitor(FrontEnd.scala:43)
	at dotty.tools.dotc.typer.FrontEnd.typeCheck(FrontEnd.scala:85)
	at dotty.tools.dotc.typer.FrontEnd.runOn$$anonfun$3(FrontEnd.scala:120)
	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at dotty.tools.dotc.typer.FrontEnd.runOn(FrontEnd.scala:120)
	at dotty.tools.dotc.Run.runPhases$4$$anonfun$4(Run.scala:185)
	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323)
	at dotty.tools.dotc.Run.runPhases$5(Run.scala:195)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:203)
	at dotty.runtime.function.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
	at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:67)
	at dotty.tools.dotc.Run.compileUnits(Run.scala:210)
	at dotty.tools.dotc.Run.compileSources(Run.scala:147)
	at dotty.tools.dotc.Run.compile(Run.scala:129)
	at dotty.tools.dotc.Driver.doCompile(Driver.scala:38)
	at dotty.tools.dotc.Driver.process(Driver.scala:194)
	at dotty.tools.dotc.Main.process(Main.scala)
	at xsbt.CachedCompilerImpl.run(CachedCompilerImpl.java:69)
	at xsbt.CompilerInterface.run(CompilerInterface.java:41)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at sbt.internal.inc.AnalyzingCompiler.invoke(AnalyzingCompiler.scala:330)
	at sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:113)
	at sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$7(MixedAnalyzingCompiler.scala:186)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:241)
	at sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4(MixedAnalyzingCompiler.scala:176)
	at sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4$adapted(MixedAnalyzingCompiler.scala:157)
	at sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:239)
	at sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:157)
	at sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:204)
	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:573)
	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:573)
	at sbt.internal.inc.Incremental$.$anonfun$apply$5(Incremental.scala:173)
	at sbt.internal.inc.Incremental$.$anonfun$apply$5$adapted(Incremental.scala:171)
	at sbt.internal.inc.Incremental$$anon$2.run(Incremental.scala:458)
	at sbt.internal.inc.IncrementalCommon$CycleState.next(IncrementalCommon.scala:116)
	at sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:56)
	at sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:52)
	at sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:261)
	at sbt.internal.inc.Incremental$.$anonfun$incrementalCompile$8(Incremental.scala:413)
	at sbt.internal.inc.Incremental$.withClassfileManager(Incremental.scala:498)
	at sbt.internal.inc.Incremental$.incrementalCompile(Incremental.scala:400)
	at sbt.internal.inc.Incremental$.apply(Incremental.scala:165)
	at sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:573)
	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:491)
	at sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:332)
	at sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:420)
	at sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
	at sbt.Defaults$.compileIncrementalTaskImpl(Defaults.scala:2176)
	at sbt.Defaults$.$anonfun$compileIncrementalTask$2(Defaults.scala:2133)
	at sbt.internal.io.Retry$.apply(Retry.scala:40)
	at sbt.internal.io.Retry$.apply(Retry.scala:23)
	at sbt.internal.server.BspCompileTask$.compute(BspCompileTask.scala:31)
	at sbt.Defaults$.$anonfun$compileIncrementalTask$1(Defaults.scala:2129)
	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
	at sbt.std.Transform$$anon$4.work(Transform.scala:68)
	at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
	at sbt.Execute.work(Execute.scala:291)
	at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
	at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
	at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

Removing the sealed keyword does not cause the issue. Changing the by-name to a lambda () => A also does not cause the issue.

Finally, this might be related somewhat to #10540 which I reported earlier, it's a similar test defining some traits inside a body that is used as a by-name parameter.

@hmemcpy hmemcpy changed the title NPE when inlining a macro when using by-name parameter defining a sealed trait NPE when inlining a macro that uses a by-name parameter defining a sealed trait Nov 28, 2020
@griggt
Copy link
Contributor

griggt commented Nov 28, 2020

Possibly related to #9655 / #9572

@hmemcpy
Copy link
Author

hmemcpy commented Nov 28, 2020

@griggt Ah, looks very similar indeed!
Edit: the original issue #9572 seems to have a workaround by @nicolasstucki (dotty-staging@01abc11) that makes it work by not inlining the body parameter. However, in my case it's a by-name, and it happens without the inline keyword.

@odersky
Copy link
Contributor

odersky commented Dec 2, 2020

The root cause of all this is that we currently cannot reliably copy non-trivial class hierarchies, and inlining implies copying. To fix this we'd have to look at TreeTypeMap and mapSymbols. This is extremely subtle code that is more easily wrecked than fixed. (Analogous functionality never worked remotely correctly over the lifetime of Scala 2). But somebody will have to take it on to plunge into this and fix it.

@smarter
Copy link
Member

smarter commented Dec 21, 2020

I just closed #10540 as a duplicate of this one, the example and crash are different but the root cause is likely the same:

Minimized code

trait Foo {
  inline def foo[A](t: => A): Unit = ()
}

object Bar extends Foo

object Test {
  Bar.foo {
    trait T1
    val array = Array(new T1 {})
  }
}

Output (click arrow to expand)

Scastie output:

java.lang.AssertionError: assertion failed: unresolved symbols: �[33mtrait�[0m �[35mT1�[0m(line 8) when pickling /tmp/scastie7347053754411828221/src/main/scala/main.scala
	at dotty.DottyPredef$.assertFail(DottyPredef.scala:17)
	at dotty.tools.dotc.core.tasty.TreePickler.pickle(TreePickler.scala:765)
	at dotty.tools.dotc.transform.Pickler.run$$anonfun$3$$anonfun$2(Pickler.scala:69)
	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at dotty.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:333)
	at dotty.tools.dotc.transform.Pickler.run$$anonfun$1(Pickler.scala:105)
        ...

@hmemcpy
Copy link
Author

hmemcpy commented Dec 21, 2020

Thanks! Just to emphasize, the assertion error here only happens when using an Array, just something to keep in mind.

odersky added a commit to dotty-staging/dotty that referenced this issue Dec 24, 2020
@Kordyjan Kordyjan modified the milestones: 3.0.0-RC1, 3.0.0 Aug 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants