Annotation macros in Dotty

[liufengyun]

We had a discussion before about annotation macros, the idea then was to transform the following annotation macros:

class main extends scala.annotation.StaticAnnotation {
    inline def apply(defn: Any): Any = meta {
       body
    }
}

to:

class main extends scala.annotation.StaticAnnotation {
    inline def apply(defn: Any): Any =  meta {
       main$inline.meta3(defn)
    }
}

object main$inline {
    def meta3(prefix:scala.meta.Stat)(defn: scala.meta.Stat): scala.meta.Stat = body
}

There are some problems with the scheme above. For example, given the code:

@main
class Test {
  println("hello, world!")
}

The namer has a difficulty to determine whether @main is a macros or not and how to expand it. It needs to do as least follows:

• resolve the name main to a symbol (typedIndent)
  • problem 1: it doesn't work reliably if main is defined in the same compilation run
• check if main extends scala.annotation.StaticAnnotation
• check if main has a member inline def apply(defn: Any): Any, and rhs begins with meta.
  • problem 2: Inspect syntactic tree to know some info about a compiled symbol is counter-intuitive, the info should be part of the symbol. Also, ASTs are not always available if @main is defined without tasty (e.g. Scala2).
• inspect the body of meta to get the implementation method.
  • problem 3: This seems to incur unnecessary complexity, a simpler convention can be that the implementation is defined in main$inline.meta, thus no need to inspect ASTs, which can impose a performance penalty by deserializing tasty.

It seems that annotation macros are somewhat special, different from def macros, inlining never happens for annotation macros. It seems to me that current paradise implementation of annotation macros is simpler (avoids problem 2 & 3), it translates the code:

class main extends scala.annotation.StaticAnnotation {
    inline def apply(defn: Any): Any = meta {
       body
    }
}

to:

class main extends scala.annotation.StaticAnnotation {
    inline def apply(defn: Any): Any =  ???
}

object main$inline {
    def apply(prefix:scala.meta.Stat)(defn: scala.meta.Stat): scala.meta.Stat = body
}

My question is:

• Can we assume annotation macros are always compiled in a different run (problem 1)?
• Can we follow paradise's way of handling annotation macros (simpler, avoids problem 2 & 3)?
• During annotation macros transform, can we only transform inline def apply(defn: Any): Any = meta {...}, and ignore other possible inline/meta methods, as they don't make sense for annotation macros?

WDYT @odersky @xeno-by @DarkDimius ?

[odersky]

Good timing. See #1693 for a partial answer :-)

[liufengyun]

From a practical point of view, I'll take the simpler approach and assume macros annotations are pre-compiled. This assumption has been and will continue to be valid for some time. Most importantly, it unblocks me. When time is mature (with an interpreter), we can refactor the impl to weaken the assumption.

To further simplify implementation, I assume all annotations which extend scala.annotations.MacrosAnnotation are annotation macros, it saves us from checking members and trees. Conventions are needed anyway, this is a friendly convention to programmers IMHO.

[xeno-by]

Sorry to have missed this discussion. This all happened right when I was moving to SF, so it seemed to fall through cracks.

Anyway, speaking of problem 2. Why do we need to inspect the AST of the annotation to figure out whether it has inline def apply? Can't we do that using just denotations?

[liufengyun]

This is because checking inline def apply is not enough, it has to check whether the right hand side if a meta block or not.

There could be inline def apply that don't have meta on the right hand side, which is definitely not an annotation macros.

Giving all these complex protocols, scala.annotations.MacrosAnnotation simplifies everything. The mechanism for annotation macros is different from def macros, I think it justifies to have something simpler for annotation macros.

[xeno-by]

I suggest we limit ourselves to just inline def apply. If its right-hand side is non-compliant, we can report that during macro expansion. This is suboptimal, but at least it doesn't leads to unsoundness holes.

The main problem with MacrosAnnotation from the point of view of a Scala implementation is the fact that compiler plugins (that's the only way of delivering an implementation in 2.11 and 2.12 because of binary compatibility) can't add new classes to the classpath. This means that MacrosAnnotation would have to be delivered as a standalone micro-library, which creates an additional complication.

[liufengyun]

@xeno-by if we can afford not checking meta, then it's fine. I'll change it back to scala.annotations.StaticAnnotation.

[liufengyun]

I think it's probably better to reintroduce meta as keyword. Currently, if we introduce meta as keyword, then we need to backquote the import statement:

import scala.`meta`._

That doesn't look nice. We reverted back to apply method in scala.meta to overcome the problem. But that isn't good either, because meta has significant importance in the compiler, it justifies to make it a keyword.

I'm not sure if it's possible to compromise the two? For example, loosen the parser to accept keywords in import statements?

Ideas @xeno-by @DarkDimius @odersky ?

[odersky]

I'd vote for keeping it a method. Other methods also have significant importance in the compiler, e.g. apply, unapply, or copy.

[liufengyun]

Thanks @odersky . While I agree keeping it a method will be the simplest way, meta is different from apply, unapply and copy, because it's a special symbol from outside the compiler, apply and unapply are just special names.

For now I'm happy to keep it as a method, we can come back to this small decision later. It should be easy to change.

[liufengyun]

This discussion is out of date now. Follow the link below for the latest discussion about annotation macros:

https://contributors.scala-lang.org/t/annotation-macros/1211/20