Add Singature information for Semanticdb

Reset local symbol index for each TextDocument, updateExpect

PrettyPrint signature and update metac.expect

Integrate SymbolInformationOps into Scala3#SymbolOps

Model WithType for AndType

#12885 (comment)

Resolve ParamRef by constructing SymbolTable

To construct symbol table before referring the ParamRef
it's required to traverse the child trees before registering the symbol.

Lookup symbols for refinements in RefinedType and RecType

Re-insert assertion to localIdx

Previously, there were the risks that localIdx receives the symbol
that doens't have `span` because `TypeOps` generates dummy symbols for
refinements in `RefinedType` and `RecType`.
However, with the following commits,
7dd91ad
226e26f
now we never generate dummy symbol in `TypeOps`.
So re-insert the assertion we deleted when copying this method from
`ExtractSemanticDB`.
see: https://github.com/lampepfl/dotty/pull/12885/files#r658150249

Convert AndType to IntersectionType

Refactor enterRefined

Refactor: move all extension (of Symbol) method into Scala3

Don't add type parameters as parameters to MethodSignature

Refactor not to use isInstanceOf for designator of TypeRef and TermRef

Use intersection type for the parent of RefinedType

Convert wildcard type as ExistentialType

For wildcard type `C[_ <: T]`, it's internal type representation will be
`AppliedType(TypeBounds(lo = <Nothing>, hi = <T>))`.

As scalameta for Scala2 does, we'll convert the wildcard type to
`ExistentialType(TypeRef(NoPrefix, C, <local0>), Scope(hardlinks = List(<local0>)))`
where `<local0>` has
- display_name: "_" and,
- signature: type_signature(..., lo = <Nothing>, hi = <T>)

See:
https://github.com/lampepfl/dotty/pull/12885/files#r663797616
https://scalameta.org/docs/semanticdb/specification.html#type-2

Now, when we compile the following Scala program to semanticdb

```scala
class Wildcards {
  def e1: List[_ <: Int] = ???
}
```

The semanticdb's SymbolInformation for `e1` looks like:

From scala3
```
  symbols {
    symbol: "advanced/Wildcards#e1()."
    kind: METHOD
    display_name: "e1"
    language: SCALA
    signature {
      value_signature {
        tpe {
          by_name_type {
            tpe {
              existential_type {
                tpe {
                  type_ref {
                    symbol: "scala/collection/immutable/List#"
                    type_arguments {
                      type_ref {
                        symbol: "local0"
                      }
                    }
                  }
                }
                declarations {
                  hardlinks {
                    symbol: "local0"
                    kind: TYPE
                    display_name: "_"
                    language: SCALA
                    signature {
                      type_signature {
                        type_parameters {
                        }
                        lower_bound {
                          type_ref {
                            symbol: "scala/Nothing#"
                          }
                        }
                        upper_bound {
                          type_ref {
                            symbol: "scala/Int#"
                          }
                        }
                      }
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
```

On the other hand, generated from scalameta's metac with scalac 2.13.6

```sh
$ metac --version
Scala compiler version 2.13.6 -- Copyright 2002-2021, LAMP/EPFL and Lightbend, Inc.
```

```
  symbols {
    symbol: "advanced/Wildcards#e()."
    kind: METHOD
    display_name: "e"
    language: SCALA
    signature {
      method_signature {
        type_parameters {
        }
        return_type {
          existential_type {
            tpe {
              type_ref {
                symbol: "scala/package.List#"
                type_arguments {
                  type_ref {
                    symbol: "local0"
                  }
                }
              }
            }
            declarations {
              hardlinks {
                symbol: "local0"
                kind: TYPE
                properties: 4
                display_name: "_"
                language: SCALA
                signature {
                  type_signature {
                    type_parameters {
                    }
                    lower_bound {
                      type_ref {
                        symbol: "scala/Nothing#"
                      }
                    }
                    upper_bound {
                      type_ref {
                        symbol: "scala/Int#"
                      }
                    }
                  }
                }
                access {
                  public_access {
                  }
                }
              }
            }
          }
        }
      }
    }
    access {
      public_access {
      }
    }
  }
```

Use finalResultType for MethodSignature to get actual return type.

Register the symbol of nested method with "actual" binder

For example:
`def foo(x: T)(y: T): T` and for `<y>.owner.info` would be like
`MethodType(...<x>, resType = MethodType(...<y>, resType = <T>))`.
(Let's say the outer `MethodType` "outer", and `MethodType` who is
`resType` of outer "inner")

Before this commit, we register <y> to the symbol table with
`(<y>, outer)`, which should be `(<y>, inner)`.

For such a nested method signature, we have to find the "actual" binder
for parameters and register them to the symbol table.

Remove unused parameter sym from toSemanticType

Fix pprint issue for SingleType

When the prefix of SingleType is Type.Emtpy, we had been printing them
as `<?>.sym.type`, but it should be `sym.type`

Convert symbols to local index for funParamSymbol

Refactor style of if

Co-authored-by: Jamie Thompson <[email protected]>

Fix printing upper type bounds

Add evidence params to SemanticDB

Add refinements to RefinedType generated by opaque types

Fix empty refinement for RefinedType for the result of polymorphic method

Print definition of ExistentialType instead of displaySymbol

Update comment

Co-authored-by: Jamie Thompson <[email protected]>

None for empty decls

Warn if symbol lookup failed for paramRef and RefinedType

Workaround symbol not found for higher kinded type param

Suppress warning for known issue of exporting Polymorphic type

Support multiple type param clauses for extension method

By combining two type param clause into one

Exclude synthetic symbols from occurrences

Fallback to Type.member for refinements who can't access to the tree

For example,
```scala
trait SpecialRefinement {
  def pickOne(as: String*): Option[Any]
}

class PickOneRefinement_1[S <: SpecialRefinement { def pickOne(as: String*): Option[String] }] {
  def run(s: S, as: String*): Option[String] = s.pickOne(as:_*)
}
```

In the typed AST, the SpecialRefinement is inside of the TypeTree and
cannot access to the refinement (pickOne) from tree, therefore we can't
register the symbol of pickOne to symbol table.
In this case, fallback to `Type.member` to find the symbol of `pickOne`.

Fallback to Type.member for ParamRef

Workaround symbol not found for HKTypeLambda in upper bounds

and fix type for curried applied type

Remove unnecessary comment

Do not create newSymbol for wildcard type symbol

instead, create a proxy data that will be converted to semanticdb symbol
who represents wildcard symbol

Warn if unexpected type appear while converting type to SemanticDB type

Dealias LazyRef while converting types to SemanticDB

Do not emit warning for unexpected type

Because there could be an unexpected types for incompilable sources
and if we show the warning for uncompilable sources, it could be confusing.

Do not create a newSymbol for ParamRef

Make wildcard type symbol global

Hardlink fake symbols

Excude symbols that doesn't present in source from occurrence

Make namePresenInSource safe for IndexOutOfRange

Make package objects present in source

Fallback to fake symbol for refinements in RefinedType of upper bound

see:
#12885 (comment)
#12885 (comment)

Add occurrences of secondary constructors

Add fakeSymbols to symbol section and don't hardlink fake syms

Convert tp <:< FromJavaObject to tp <:< Any

Do not add wildcard symbol of existential type to symbol section

Refactor styles

Author: tanishiking

compiler/src/dotty/tools/dotc/semanticdb/ConstantOps.scala

@@ -0,0 +1,25 @@
+package dotty.tools
+package dotc
+package semanticdb
+
+import dotty.tools.dotc.{semanticdb => s}
+
+import core.Contexts.Context
+import core.Constants._
+
+object ConstantOps:
+  extension (const: Constant)
+    def toSemanticConst(using Context): s.Constant = const.tag match {
+      case UnitTag => s.UnitConstant()
+      case BooleanTag => s.BooleanConstant(const.booleanValue)
+      case ByteTag => s.ByteConstant(const.byteValue)
+      case ShortTag => s.ShortConstant(const.shortValue)
+      case CharTag => s.CharConstant(const.charValue)
+      case IntTag => s.IntConstant(const.intValue)
+      case LongTag => s.LongConstant(const.longValue)
+      case FloatTag => s.FloatConstant(const.floatValue)
+      case DoubleTag => s.DoubleConstant(const.doubleValue)
+      case StringTag => s.StringConstant(const.stringValue)
+      case NullTag => s.NullConstant()
+      case _ => throw new Error(s"Constant ${const} can't be converted to Semanticdb Constant.")
+    }

compiler/src/dotty/tools/dotc/semanticdb/Descriptor.scala

@@ -0,0 +1,120 @@
+package dotty.tools.dotc.semanticdb
+
+import java.lang.System.{lineSeparator => EOL}
+import dotty.tools.dotc.semanticdb.{Descriptor => d}
+
+class DescriptorParser(s: String) {
+  var i = s.length
+  def fail() = {
+    val message = "invalid symbol format"
+    val caret = " " * i + "^"
+    sys.error(s"$message$EOL$s$EOL$caret")
+  }
+
+  val BOF = '\u0000'
+  val EOF = '\u001A'
+  var currChar = EOF
+  def readChar(): Char = {
+    if (i <= 0) {
+      if (i == 0) {
+        i -= 1
+        currChar = BOF
+        currChar
+      } else {
+        fail()
+      }
+    } else {
+      i -= 1
+      currChar = s(i)
+      currChar
+    }
+  }
+
+  def parseValue(): String = {
+    if (currChar == '`') {
+      val end = i
+      while (readChar() != '`') {}
+      readChar()
+      s.substring(i + 2, end)
+    } else {
+      val end = i + 1
+      if (!Character.isJavaIdentifierPart(currChar)) fail()
+      while (Character.isJavaIdentifierPart(readChar()) && currChar != BOF) {}
+      s.substring(i + 1, end)
+    }
+  }
+
+  def parseDisambiguator(): String = {
+    val end = i + 1
+    if (currChar != ')') fail()
+    while (readChar() != '(') {}
+    readChar()
+    s.substring(i + 1, end)
+  }
+
+  def parseDescriptor(): Descriptor = {
+    if (currChar == '.') {
+      readChar()
+      if (currChar == ')') {
+        val disambiguator = parseDisambiguator()
+        val value = parseValue()
+        d.Method(value, disambiguator)
+      } else {
+        d.Term(parseValue())
+      }
+    } else if (currChar == '#') {
+      readChar()
+      d.Type(parseValue())
+    } else if (currChar == '/') {
+      readChar()
+      d.Package(parseValue())
+    } else if (currChar == ')') {
+      readChar()
+      val value = parseValue()
+      if (currChar != '(') fail()
+      else readChar()
+      d.Parameter(value)
+    } else if (currChar == ']') {
+      readChar()
+      val value = parseValue()
+      if (currChar != '[') fail()
+      else readChar()
+      d.TypeParameter(value)
+    } else {
+      fail()
+    }
+  }
+
+  def entryPoint(): (Descriptor, String) = {
+    readChar()
+    val desc = parseDescriptor()
+    (desc, s.substring(0, i + 1))
+  }
+}
+
+object DescriptorParser {
+  def apply(symbol: String): (Descriptor, String) = {
+    val parser = new DescriptorParser(symbol)
+    parser.entryPoint()
+  }
+}
+
+sealed trait Descriptor {
+  def isNone: Boolean = this == d.None
+  def isTerm: Boolean = this.isInstanceOf[d.Term]
+  def isMethod: Boolean = this.isInstanceOf[d.Method]
+  def isType: Boolean = this.isInstanceOf[d.Type]
+  def isPackage: Boolean = this.isInstanceOf[d.Package]
+  def isParameter: Boolean = this.isInstanceOf[d.Parameter]
+  def isTypeParameter: Boolean = this.isInstanceOf[d.TypeParameter]
+  def value: String
+}
+object Descriptor {
+  case object None extends Descriptor { def value: String = "" }
+  final case class Term(value: String) extends Descriptor
+  final case class Method(value: String, disambiguator: String) extends Descriptor
+  final case class Type(value: String) extends Descriptor
+  final case class Package(value: String) extends Descriptor
+  final case class Parameter(value: String) extends Descriptor
+  final case class TypeParameter(value: String) extends Descriptor
+}