Skip to content

Parameter untupling with context function args #16994

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ojow opened this issue Feb 22, 2023 · 11 comments · Fixed by #17739
Closed

Parameter untupling with context function args #16994

ojow opened this issue Feb 22, 2023 · 11 comments · Fixed by #17739
Assignees
Labels
Milestone

Comments

@ojow
Copy link

ojow commented Feb 22, 2023

Compiler version

3.2.2

Minimized code

type ZZ = String ?=> Int
def f(xs: ZZ*) = xs.zipWithIndex.foreach((f: ZZ, i) => f(using "s"))

Output (click arrow to expand)

[error] scala.MatchError: Block(List(DefDef($anonfun,List(List(ValDef(evidence$1,TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class scala)),object Predef),type String)],EmptyTree))),TypeTree[TypeRef(TermRef(ThisType(TypeRef(NoPrefix,module class <root>)),object scala),class Int)],Apply(Select(Select(Ident(x$1),_1),apply),List(Ident(evidence$1))))),Closure(List(),Ident($anonfun),EmptyTree)) (of class dotty.tools.dotc.ast.Trees$Block)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.checkStableSelection(PostTyper.scala:268)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transform(PostTyper.scala:378)
[error] dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.loop$2(tpd.scala:1225)
[error] dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformStats(tpd.scala:1225)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transformStats(PostTyper.scala:474)
[error] dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformBlock(tpd.scala:1230)
[error] dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1426)
[error] dotty.tools.dotc.transform.MacroTransform$Transformer.transform(MacroTransform.scala:49)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transform(PostTyper.scala:465)
[error] dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1478)
[error] dotty.tools.dotc.transform.MacroTransform$Transformer.transform(MacroTransform.scala:40)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transform$$anonfun$5(PostTyper.scala:384)
[error] dotty.tools.dotc.transform.SuperAccessors.wrapDefDef(SuperAccessors.scala:223)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transform(PostTyper.scala:384)
[error] dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.loop$2(tpd.scala:1225)
[error] dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformStats(tpd.scala:1225)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transformStats(PostTyper.scala:474)
[error] dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformBlock(tpd.scala:1230)
[error] dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1426)
[error] dotty.tools.dotc.transform.MacroTransform$Transformer.transform(MacroTransform.scala:49)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transform(PostTyper.scala:465)
[error] dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform$$anonfun$1(Trees.scala:1507)
[error] scala.collection.immutable.List.mapConserve(List.scala:472)
[error] dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1507)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.app1$1(PostTyper.scala:323)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transform(PostTyper.scala:337)
[error] dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1478)
[error] dotty.tools.dotc.transform.MacroTransform$Transformer.transform(MacroTransform.scala:40)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transform$$anonfun$5(PostTyper.scala:384)
[error] dotty.tools.dotc.transform.SuperAccessors.wrapDefDef(SuperAccessors.scala:223)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transform(PostTyper.scala:384)
[error] dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.loop$2(tpd.scala:1225)
[error] dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformStats(tpd.scala:1225)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transformStats(PostTyper.scala:474)
[error] dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformStats(tpd.scala:1227)
[error] dotty.tools.dotc.transform.MacroTransform$Transformer.transform(MacroTransform.scala:47)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transform$$anonfun$4$$anonfun$1(PostTyper.scala:371)
[error] dotty.tools.dotc.transform.SuperAccessors.wrapTemplate(SuperAccessors.scala:208)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transform$$anonfun$4(PostTyper.scala:371)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.withNoCheckNews(PostTyper.scala:104)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transform(PostTyper.scala:373)
[error] dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1480)
[error] dotty.tools.dotc.transform.MacroTransform$Transformer.transform(MacroTransform.scala:40)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transform(PostTyper.scala:411)
[error] dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.loop$2(tpd.scala:1225)
[error] dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformStats(tpd.scala:1225)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transformStats(PostTyper.scala:474)
[error] dotty.tools.dotc.ast.tpd$TreeMapWithPreciseStatContexts.transformStats(tpd.scala:1227)
[error] dotty.tools.dotc.ast.Trees$Instance$TreeMap.transform(Trees.scala:1488)
[error] dotty.tools.dotc.transform.MacroTransform$Transformer.transform(MacroTransform.scala:40)
[error] dotty.tools.dotc.transform.PostTyper$PostTyperTransformer.transform(PostTyper.scala:465)
[error] dotty.tools.dotc.transform.MacroTransform.run(MacroTransform.scala:18)
[error] dotty.tools.dotc.core.Phases$Phase.runOn$$anonfun$1(Phases.scala:316)
[error] scala.collection.immutable.List.map(List.scala:246)
[error] dotty.tools.dotc.core.Phases$Phase.runOn(Phases.scala:320)
[error] dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:238)
[error] scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
[error] scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
[error] scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1321)
[error] dotty.tools.dotc.Run.runPhases$1(Run.scala:249)
[error] dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:257)
[error] dotty.tools.dotc.Run.compileUnits$$anonfun$adapted$1(Run.scala:266)
[error] dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:68)
[error] dotty.tools.dotc.Run.compileUnits(Run.scala:266)
[error] dotty.tools.dotc.Run.compileSources(Run.scala:190)
[error] dotty.tools.dotc.Run.compile(Run.scala:174)
[error] dotty.tools.dotc.Driver.doCompile(Driver.scala:35)
[error] dotty.tools.xsbt.CompilerBridgeDriver.run(CompilerBridgeDriver.java:88)
[error] dotty.tools.xsbt.CompilerBridge.run(CompilerBridge.java:22)
[error] sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:91)
[error] sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$7(MixedAnalyzingCompiler.scala:193)
[error] scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[error] sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:248)
[error] sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4(MixedAnalyzingCompiler.scala:183)
[error] sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4$adapted(MixedAnalyzingCompiler.scala:163)
[error] sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:239)
[error] sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:163)
[error] sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:211)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:534)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:534)
[error] sbt.internal.inc.Incremental$.$anonfun$apply$5(Incremental.scala:179)
[error] sbt.internal.inc.Incremental$.$anonfun$apply$5$adapted(Incremental.scala:177)
[error] sbt.internal.inc.Incremental$$anon$2.run(Incremental.scala:463)
[error] sbt.internal.inc.IncrementalCommon$CycleState.next(IncrementalCommon.scala:116)
[error] sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:56)
[error] sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:52)
[error] sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:263)
[error] sbt.internal.inc.Incremental$.$anonfun$incrementalCompile$8(Incremental.scala:418)
[error] sbt.internal.inc.Incremental$.withClassfileManager(Incremental.scala:506)
[error] sbt.internal.inc.Incremental$.incrementalCompile(Incremental.scala:405)
[error] sbt.internal.inc.Incremental$.apply(Incremental.scala:171)
[error] sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:534)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:488)
[error] sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:332)
[error] sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:425)
[error] sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
[error] sbt.Defaults$.compileIncrementalTaskImpl(Defaults.scala:2363)
[error] sbt.Defaults$.$anonfun$compileIncrementalTask$2(Defaults.scala:2313)
[error] sbt.internal.server.BspCompileTask$.$anonfun$compute$1(BspCompileTask.scala:30)
[error] sbt.internal.io.Retry$.apply(Retry.scala:46)
[error] sbt.internal.io.Retry$.apply(Retry.scala:28)
[error] sbt.internal.io.Retry$.apply(Retry.scala:23)
[error] sbt.internal.server.BspCompileTask$.compute(BspCompileTask.scala:30)
[error] sbt.Defaults$.$anonfun$compileIncrementalTask$1(Defaults.scala:2311)
[error] scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error] sbt.std.Transform$$anon$4.work(Transform.scala:68)
[error] sbt.Execute.$anonfun$submit$2(Execute.scala:282)
[error] sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
[error] sbt.Execute.work(Execute.scala:291)
[error] sbt.Execute.$anonfun$submit$1(Execute.scala:282)
[error] sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
[error] java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
[error] java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[error] java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[error] java.base/java.lang.Thread.run(Thread.java:834)
@ojow ojow added itype:bug itype:crash stat:needs triage Every issue needs to have an "area" and "itype" label labels Feb 22, 2023
@Kordyjan Kordyjan added area:transform and removed stat:needs triage Every issue needs to have an "area" and "itype" label labels Feb 22, 2023
@Kordyjan Kordyjan added this to the Future versions milestone Feb 22, 2023
@Kordyjan
Copy link
Contributor

Still reproducible both on 3.3.0-RC3 and 3.3.1-RC1-bin-20230221-847eccc-NIGHTLY.

@Sporarum
Copy link
Contributor

Sporarum commented Mar 7, 2023

Stems from two factors:

  1. checkStableSelection in PostTyper assumes its parameter will be either _n or apply[T](n)
  2. context functions generate useless closures when they are only aliases:
// post typer
type ZZ = String => Int
def f: ZZ = ???
val f2: ZZ = f
// vs
type ZZ = (String) ?=> Int
def f: ZZ =
  {
    def $anonfun(using evidence$1: String): Int = ???
    closure($anonfun)
  }
val f2: ZZ =
  {
    def $anonfun(using evidence$2: String): Int = f.apply(evidence$2)
    closure($anonfun)
  }

Those two clash with each other, as now x$1._1 becomes

{
  def $anonfun(using evidence$1: String): Int =
    x$1._1.apply(evidence$1)
  closure($anonfun)
}

There are thus two ways to solve this problem:

  1. Allow checkStableSelection to accept this expanded form
  2. Make it so this unneeded expansion is never done

We believe option 2 to be superior, but did not have time to implement it, we can however simplify the test case, now all that remains to test is that

type ZZ = String ?=> Int
def f: ZZ = ???
val f2: ZZ = f

Looks like the following after typer (-Xprint:typer):

package <empty> {
  final lazy module val i16994$package: i16994$package = new i16994$package()
  final module class i16994$package() extends Object() {
    this: i16994$package.type =>
    type ZZ = (String) ?=> Int
    val f: ZZ = ???
    val i: Int = f
  }
}

@Sporarum
Copy link
Contributor

Sporarum commented Mar 7, 2023

@Sporarum
Copy link
Contributor

Sporarum commented Mar 7, 2023

Actually, with the reduced test, only knowledge of context functions is needed

@Sporarum Sporarum added the Spree Suitable for a future Spree label Mar 17, 2023
@Sporarum Sporarum self-assigned this Mar 17, 2023
@scala-center-bot
Copy link

This issue was picked for the Issue Spree No. 28 of 28 March 2023 which takes place in a week from now. @Sporarum, @dwijnand, @yawen-guan will be working on it. If you have any insight into the issue or guidance on how to fix it, please leave it here.

@Sporarum
Copy link
Contributor

I had made a branch for this issue:
https://github.com/dotty-staging/dotty/tree/fix-i16994

@Sporarum
Copy link
Contributor

Maybe we could add something to -Ycheck to make sure the invariant is respected ?

@Sporarum
Copy link
Contributor

We realised this is a harder problem to solve than expected, this is due to

val x: String ?=> Int = 4
// becomes:
val x: String ?=> Int = (_: String) ?=> 4

So

type ZZ = String ?=> Int
def f: ZZ = ???
val f2: ZZ = f

// becomes naturally something like:
...
val f2: ZZ = (_: String) ?=> f(using summon[String])

Given this seems to be the specced behaviour we determined the fastest way for this issue to be solved was to instead change checkStableSelection

This allows for this issue to be closed sooner, and in case these useless closures become more of a problem, it can always be changed later

@dwijnand
Copy link
Member

dwijnand commented Mar 31, 2023

@odersky btw, this was the issue I was attempting to describe.

The handling of context function is unique in being sort of "pre-adapted" in typer, which doesn't allow for this simple (but less common) counter-example to be typed in an expected way. I now realise that there are only calls to "readapt" in adapt, there's no re-typing, as far as I can see. So even if this would be moved to adapt, it would be a bit of an outlier there too. And at the cost of cpu and memory for the more common cases: we'd spend time assigning error types and then have to clone trees to be able to assign the right types the second time round.

So perhaps we should leave this as is. One of the cases I looked at with Quentin did end up removing the pointless closure, somewhere in LazyVals's mega phase (didn't find out which mini phase), so it doesn't result in the bytecode. Perhaps we could add it as a case in FirstTransform - but that wouldn't help this error here, as the failure is earlier in PostTyper.

Just jotting down my mental notes on this. I think we should just make checkStableSelection less picky.

@odersky
Copy link
Contributor

odersky commented Mar 31, 2023

Context functions deeply need these alias closures. They will be elimiinated later at erasure. But to do a realiable job of elimination we have to make sure these closures are there in the first place.

So yes, we need to change checkStableSelection.

@Sporarum
Copy link
Contributor

Sporarum commented Mar 31, 2023

Thank you for your insights, it's good to know in particular these are deleted later !

I do wonder however if there are other places where we assume a tree to be of a certain shape, and this can break it ?

In regards to this issue, I believe it to still be suitable for a Spree, as the fix is now relatively well scoped, alternatively, @dwijnand and/or I could fix it directly

@dwijnand dwijnand linked a pull request May 31, 2023 that will close this issue
@SethTisue SethTisue assigned dwijnand and unassigned Sporarum May 31, 2023
@Kordyjan Kordyjan modified the milestones: Future versions, 3.4.0 Aug 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants