Expr.betaReduce does not correctly handle inlining function literals type-ascribed to subclasses of Function

[fhackett]

minimized code

abstract class Test extends (Int => Int) {
  def impl(i: Int): Int
  def apply(i: Int): Int = -1
}
// ...
object Macros {
  inline def dummy : Test =
    (i: Int) => i + 1

  inline def test : Int =
    ${ testImpl }
  def testImpl(using: QuoteContext): Expr[Int]
    Expr.betaReduce('{ dummy })('{ 5 })
}
// ...
object Test {
  def main(argv: Array[String]): Unit =
    println(s"${ Macros.test }")
}

Runtime output

6

expectation

-1

[fhackett]

The rabbit hole goes even deeper - I just learned that type ascriptions in Scala are not algebraically transitive:

abstract class Test1 extends (Int => Int) {
  def apply(s: String): String
}
abstract class Test2 extends Test1 {
  def apply(s: String): String = "gotcha"
}
// ...
( (i: Int) => i + 1 ) : Test2 : Test1

[fhackett]

... actually, yes they are [re: ascription transitivity]. Closure.tpt contains the necessary info to handle SAM cases.

I learned that the inliner has its own betaReduce, here: https://github.com/lampepfl/dotty/blob/a29178ce4ac848e773423f57f45186373f410970/compiler/src/dotty/tools/dotc/typer/Inliner.scala#L804-L824

This other betaReduce rejects SAM closures, and given the amount of complexity my attempt to properly handle SAM closures introduces I think it's probably a good idea to imitate the inliner for now. In any case, that will prevent the crashes and strange behaviours this issue is talking about.