diff --git a/docs/_assets/images/contribution/breakpoint.jpg b/docs/_assets/images/contribution/breakpoint.jpg new file mode 100644 index 000000000000..748088c269c9 Binary files /dev/null and b/docs/_assets/images/contribution/breakpoint.jpg differ diff --git a/docs/_assets/images/contribution/call-stack.jpg b/docs/_assets/images/contribution/call-stack.jpg new file mode 100644 index 000000000000..8fac2371a6c1 Binary files /dev/null and b/docs/_assets/images/contribution/call-stack.jpg differ diff --git a/docs/_assets/images/contribution/conditional-breakpoint.jpg b/docs/_assets/images/contribution/conditional-breakpoint.jpg new file mode 100644 index 000000000000..11bab89d3f47 Binary files /dev/null and b/docs/_assets/images/contribution/conditional-breakpoint.jpg differ diff --git a/docs/_assets/images/contribution/create-config.jpg b/docs/_assets/images/contribution/create-config.jpg new file mode 100644 index 000000000000..60479233ee70 Binary files /dev/null and b/docs/_assets/images/contribution/create-config.jpg differ diff --git a/docs/_assets/images/contribution/debug-console.jpg b/docs/_assets/images/contribution/debug-console.jpg new file mode 100644 index 000000000000..c9a669019d65 Binary files /dev/null and b/docs/_assets/images/contribution/debug-console.jpg differ diff --git a/docs/_assets/images/contribution/import-build.jpg b/docs/_assets/images/contribution/import-build.jpg new file mode 100644 index 000000000000..79be8450cd4a Binary files /dev/null and b/docs/_assets/images/contribution/import-build.jpg differ diff --git a/docs/_assets/images/contribution/launch-config-file.jpg b/docs/_assets/images/contribution/launch-config-file.jpg new file mode 100644 index 000000000000..4270f6b2326a Binary files /dev/null and b/docs/_assets/images/contribution/launch-config-file.jpg differ diff --git a/docs/_assets/images/contribution/start-debugger.jpg b/docs/_assets/images/contribution/start-debugger.jpg new file mode 100644 index 000000000000..edf17d700afc Binary files /dev/null and b/docs/_assets/images/contribution/start-debugger.jpg differ diff --git a/docs/_assets/images/contribution/toolbar.jpg b/docs/_assets/images/contribution/toolbar.jpg new file mode 100644 index 000000000000..22ae60ba27e1 Binary files /dev/null and b/docs/_assets/images/contribution/toolbar.jpg differ diff --git a/docs/_docs/contributing/architecture/context.md b/docs/_docs/contributing/architecture/context.md new file mode 100644 index 000000000000..cd38ee437867 --- /dev/null +++ b/docs/_docs/contributing/architecture/context.md @@ -0,0 +1,53 @@ +--- +layout: doc-page +title: Contexts +--- + +`dotc` has almost no global state (with the exception of the name table, +which hashes strings into unique names). Instead, all +essential bits of information that can vary over a compiler [run](./lifecycle.md) are collected +in a `Context` (defined in [Contexts]). + +Most methods in the compiler depend on an implicit anonymous `Context` parameter, +and a typical definition looks like the following: +```scala +import dotty.tools.dotc.Contexts.{Context, ctx} + +def doFoo(using Context): Unit = + val current = ctx.run // access the Context parameter with `ctx` +``` + +## Memory Leaks +> **Careful:** Contexts can be heavy so beware of memory leaks + +It is good practice to ensure that implicit contexts are not +captured in closures or other long-lived objects, in order to avoid space leaks +in the case where a closure can survive several compiler runs (e.g. a +lazy completer for a library class that is never required). In that case, the +convention is that the `Context` be an explicit parameter, to track its usage. + +## Context Properties + +| Context property | description | +|-------------------|----------------------------------------| +| `compilationUnit` | current compilation unit | +| `phase` | current phase | +| `run` | current run | +| `period` | current period | +| `settings` | the config passed to the compiler | +| `reporter` | operations for logging errors/warnings | +| `definitions` | the standard built in definitions | +| `platform` | operations for the underlying platform | +| `tree` | current tree | +| `scope` | current scope | +| `typer` | current typer | +| `owner` | current owner symbol | +| `outer` | outer Context | +| `mode` | type checking mode | +| `typerState` | | +| `searchHistory` | | +| `implicits` | | +| ... | and so on | + + +[Contexts]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/Contexts.scala diff --git a/docs/_docs/contributing/architecture/index.md b/docs/_docs/contributing/architecture/index.md new file mode 100644 index 000000000000..9b976cc643cd --- /dev/null +++ b/docs/_docs/contributing/architecture/index.md @@ -0,0 +1,14 @@ +--- +layout: index +title: High Level Architecture +--- + +This chapter of the guide describes the architecture and concepts of `dotc`, +the Scala 3 compiler, including answers to questions such as: +- "What are the transformations that happen to my code?" +- "How do I run a compiler programatically?" +- "What are symbols, denotations, names and types?" +- "What is a compiler phase?" +- "What is the compiler Context?" + +and many more. diff --git a/docs/_docs/contributing/architecture/lifecycle.md b/docs/_docs/contributing/architecture/lifecycle.md new file mode 100644 index 000000000000..2cf58f477da3 --- /dev/null +++ b/docs/_docs/contributing/architecture/lifecycle.md @@ -0,0 +1,90 @@ +--- +layout: doc-page +title: Compiler Overview +--- + +At a high level, `dotc` is an interactive compiler (see [what is a compiler?](../index.md#what-is-a-compiler)), +and can be invoked frequently, for example to answer questions for an IDE, provide REPL completions, +or to manage incremental builds and more. Each of these use cases requires a customised +workflow, but sharing a common core. + +## Introducing the Compiler's Lifecycle + +#### Core +Customisation is provided by extending the [Compiler] class, which maintains an ordered +list of [phases][Phases], and how to [run][Run] them. Each interaction with a compiler +creates a new run, which is a complete iteration of the compiler's phases over a list +of input sources. Each run has the capability to create new definitions or +invalidate older ones, and `dotc` can [track these changes over time](../architecture/time.md). + +#### Runs +During a run, the input sources are converted to [compilation units][CompilationUnit] (i.e. the abstraction of +compiler state associated with each input source); then iteratively: a single phase is applied to +every compilation unit before progressing to the next phase. + +#### Phases +A phase is an abstract transformation over a compilation unit, it is usually responsible +for transforming the trees and types representing the code of a source file. Some phases of +the compiler are: +- `parser`, which converts text that matches Scala's + [syntax] into abstract syntax trees, ASTs +- `typer`, which checks that trees conform to expected types +- `erasure`, which retypes a more simplified program into one that has the same types as the JVM. +- `genBCode`, the JVM backend, which converts erased compiler trees into Java bytecode format. + +[You can read more about phases here](../architecture/phases.md#phase-categories). + +#### Drivers + +The core compiler also requires a lot of state to be initialised before use, such as [settings][ScalaSettings] +and the [Context](../architecture/context.md). For convenience, the [Driver] class contains high level functions for +configuring the compiler and invoking it programatically. The object [Main] inherits from `Driver` +and is invoked by the `scalac` script. + +## Code Structure + +The code of the compiler is found in the package [dotty.tools], +containing the following sub-packages: +```scala +tools // contains helpers and the `scala` generic runner +├── backend // Compiler backends (currently JVM and JS) +├── dotc // The main compiler, with subpackages: +│ ├── ast // Abstract syntax trees +│   ├── classpath +│   ├── config // Compiler configuration, settings, platform specific definitions. +│   ├── core // Core data structures and operations, with specific subpackages for: +│   │   ├── classfile // Reading of Java classfiles into core data structures +│   │   ├── tasty // Reading and writing of TASTY files to/from core data structures +│   │   └── unpickleScala2 // Reading of Scala2 symbol information into core data structures +│   ├── decompiler // pretty printing TASTY as code +│   ├── fromtasty // driver for recompilation from TASTY +│   ├── interactive // presentation compiler and code completions +│   ├── parsing // Scanner and parser +│   ├── plugins // compile plugin definitions +│   ├── printing // Pretty-printing trees, types and other data +│   ├── profile // internals for profiling the compiler +│   ├── quoted // internals for quoted reflection +│   ├── reporting // Reporting of error messages, warnings and other info. +│   ├── rewrites // Helpers for rewriting Scala 2's constructs into Scala 3's. +│   ├── sbt // Helpers for communicating with the Zinc compiler. +│   ├── semanticdb // Helpers for exporting semanticdb from trees. +│   ├── transform // Miniphases and helpers for tree transformations. +│   ├── typer // Type-checking +│   └── util // General purpose utility classes and modules. +├── io // Helper modules for file access and classpath handling. +├── repl // REPL driver and interaction with the terminal +├── runner // helpers for the `scala` generic runner script +└── scripting // scala runner for the -script argument +``` + + +[Phases]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/Phases.scala +[CompilationUnit]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/CompilationUnit.scala + +[dotty.tools]: https://github.com/lampepfl/dotty/tree/master/compiler/src/dotty/tools +[ScalaSettings]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/config/ScalaSettings.scala +[syntax]: https://docs.scala-lang.org/scala3/reference/syntax.html +[Main]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/Main.scala +[Driver]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/Driver.scala +[Compiler]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/Compiler.scala +[Run]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/Run.scala \ No newline at end of file diff --git a/docs/_docs/contributing/architecture/phases.md b/docs/_docs/contributing/architecture/phases.md new file mode 100644 index 000000000000..15690dfe4e4e --- /dev/null +++ b/docs/_docs/contributing/architecture/phases.md @@ -0,0 +1,108 @@ +--- +layout: doc-page +title: Compiler Phases +--- + +As described in the [compiler overview](lifecycle.md#phases), `dotc` is divided into a list of [phases][Phase], +specified in the [Compiler] class. + +#### Printing the phases of the Compiler + +a flattened list of all the phases can be displayed by invoking +the compiler with the `-Xshow-phases` flag: +``` +$ scalac -Xshow-phases +``` + +## Phase Groups + +In class [Compiler] you can access the list of phases with the method `phases`: + +```scala +def phases: List[List[Phase]] = + frontendPhases ::: picklerPhases ::: transformPhases ::: backendPhases +``` + +You can see that phases are actually grouped into sublists, given by the signature +`List[List[Phase]]`; that is, each sublist forms a phase group that is then *fused* into a +single tree traversal when a [Run] is executed. + +Phase fusion allows each phase of a group to be small and modular, +(each performing a single function), while reducing the number of tree traversals +and increasing performance. + +Phases are able to be grouped together if they inherit from [MiniPhase]. + +## Phase Categories + +Phases fall into four categories, allowing customisation by sub-classes of [Compiler]: + +### `frontendPhases` +In the main compiler these include [parser], [typer], [posttyper], +[prepjsinterop] and phases for producing SemanticDB and communicating with the +incremental compiler Zinc. +The [parser] reads source programs and generates untyped abstract syntax trees, which +in [typer] are then typechecked and transformed into typed abstract syntax trees. +Following is [posttyper], performing checks and cleanups that require a fully typed program. +In particular, it +- creates super accessors representing `super` calls in traits +- creates implementations of compiler-implemented methods, +such as `equals` and `hashCode` for case classes. +- marks [compilation units][CompilationUnit] that require inline expansion, or quote pickling +- simplifies trees of erased definitions +- checks variance of type parameters +- mark parameters passed unchanged from subclass to superclass for later pruning. + +### `picklerPhases` +These phases start with [pickler], which serializes typed trees +produced by the `frontendPhases` into TASTy format. Following is [inlining], +which expand calls to inline methods, and [postInlining] providing implementations +of the [Mirror] framework for inlined calls. +Finally are [staging], which ensures that quotes conform to the +[Phase Consistency Principle (PCP)][PCP], and [pickleQuotes] which converts quoted +trees to embedded TASTy strings. + +### `transformPhases` +These phases are concerned with tranformation into lower-level forms +suitable for the runtime system, with two sub-groupings: +- High-level transformations: All phases from [firstTransform] to [erasure]. + Most of these phases transform syntax trees, expanding high-level constructs + to more primitive ones. + - An important transform phase is [patternMatcher], which converts match + trees and patterns into lower level forms, as well as checking the + exhaustivity of sealed types, and unreachability of pattern cases. + - Some phases perform further checks on more primitive trees, + e.g. [refchecks] verifies that no abstract methods exist in concrete classes, + and [initChecker] checks that fields are not used before initialisation. + - The last phase in the group, [erasure] translates all + types into types supported directly by the JVM. To do this, it performs + another type checking pass, but using the rules of the JVM's type system + instead of Scala's. +- Low-level transformations: All phases from `ElimErasedValueType` to + `CollectSuperCalls`. These further transform trees until they are essentially a + structured version of Java bytecode. + +### `backendPhases` +These map the transformed trees to Java classfiles or SJSIR files. + +[CompilationUnit]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/CompilationUnit.scala +[Compiler]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/Compiler.scala +[Phase]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/Phases.scala +[MiniPhase]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/MegaPhase.scala +[Run]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/Run.scala +[parser]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/parsing/ParserPhase.scala +[typer]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/typer/TyperPhase.scala +[posttyper]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/PostTyper.scala +[prepjsinterop]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/sjs/PrepJSInterop.scala +[pickler]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/Pickler.scala +[inlining]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/Inlining.scala +[postInlining]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/PostInlining.scala +[staging]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/Staging.scala +[pickleQuotes]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/PickleQuotes.scala +[refchecks]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/typer/RefChecks.scala +[initChecker]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/init/Checker.scala +[firstTransform]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/FirstTransform.scala +[patternMatcher]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/PatternMatcher.scala +[erasure]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/Erasure.scala +[Mirror]: https://github.com/lampepfl/dotty/blob/master/library/src/scala/deriving/Mirror.scala +[PCP]: {{ site.scala3ref }}/metaprogramming/macros.html#the-phase-consistency-principle diff --git a/docs/_docs/contributing/architecture/symbols.md b/docs/_docs/contributing/architecture/symbols.md new file mode 100644 index 000000000000..c19588a4ff12 --- /dev/null +++ b/docs/_docs/contributing/architecture/symbols.md @@ -0,0 +1,70 @@ +--- +layout: doc-page +title: Symbols +--- + +As discussed previously, `dotc` [maintains time-indexed views](time.md) of various +compiler artifacts. The following sections discuss how they are managed in the compiler. + +## Symbols + +Defined in [Symbols], a `Symbol` is a unique identifier for a definition (e.g. a method, +type, or field). A `ClassSymbol` extends `Symbol` and represents either a +`class`, or a `trait`, or an `object`. A `Symbol` can even refer to non-Scala entities, +such as from the Java standard library. + +## Definitions are Dynamic + +Traditionally, compilers store context-dependent data in a _symbol table_. +Where a symbol then is the central reference to address context-dependent data. +`dotc` instead uses a phase-indexed function (known as +a [Denotation][Denotations]) to compute views of definitions across phases, +as many of attributes associated with definitions are phase-dependent. For example: +- types are gradually simplified by several phases, +- owners change in [lambdaLift] (local methods are lifted to an enclosing class) + and [flatten] (when inner classes are moved to the top level) +- Names are changed when private members need to be accessed from outside + their class (for instance from a nested class or a class implementing + a trait). + +Additionally, symbols are not suitable to be used as a reference to +a definition in another [compilation unit][CompilationUnit]. +In the context of incremental compilation, a symbol from +an external compilation unit may be deleted or changed, making the reference +stale. To counter this, `dotc` types trees of cross-module references with either +a `TermRef` or `TypeRef`. A reference type contains a prefix type and a name. +The denotation that the type refers to is established dynamically based on +these fields. + +## Denotations + +On its own a `Symbol` has no structure. Its semantic meaning is given by being associated +with a [Denotation][Denotations]. + +A denotation is the result of resolving a name during a given period, containing the information +describing some entity (either a term or type), indexed by phase. Denotations usually have a +reference to a selected symbol, but not always, for example if the denotation is overloaded, +i.e. a `MultiDenotation`. + +### SymDenotations +All definition symbols will contain a `SymDenotation`. The denotation, in turn, contains: +- a reverse link to the source symbol +- a reference to the enclosing symbol that defined the source symbol: + - for a local variable, the enclosing method + - for a field or class, the enclosing class +- a set of [flags], describing the definition (e.g. whether it's a trait or mutable). +- the type of the definition (through the `info` method) +- a [signature][Signature1], which uniquely identifies overloaded methods (or else `NotAMethod`). +- and more. + +A class symbol will instead be associated with a `ClassDenotation`, which extends `SymDenotation` +with some additional fields specific for classes. + +[Signature1]: https://github.com/lampepfl/dotty/blob/a527f3b1e49c0d48148ccfb2eb52e3302fc4a349/compiler/src/dotty/tools/dotc/core/Signature.scala#L9-L33 +[Symbols]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/Symbols.scala +[flatten]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/Flatten.scala +[lambdaLift]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/LambdaLift.scala +[CompilationUnit]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/CompilationUnit.scala +[Denotations]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/Denotations.scala +[SymDenotations]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/SymDenotations.scala +[flags]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/Flags.scala diff --git a/docs/_docs/contributing/architecture/time.md b/docs/_docs/contributing/architecture/time.md new file mode 100644 index 000000000000..588b1ce40bb2 --- /dev/null +++ b/docs/_docs/contributing/architecture/time.md @@ -0,0 +1,68 @@ +--- +layout: doc-page +title: Time in the Compiler +--- + +In the [compiler overview](lifecycle.md) section, we saw that `dotc` is an interactive compiler, +and so can answer questions about entities as they come into existance and change throughout time, +for example: +- which new definitions were added in a REPL session? +- which definitions were replaced in an incremental build? +- how are definitions simplified as they are adapted to the runtime system? + +## Hours, Minutes, and Periods + +For the compiler to be able to resolve the above temporal questions, and more, it maintains +a concept of time. Additionally, because interactions are frequent, it is important to +persist knowledge of entities between interactions, allowing the compiler to remain performant. +Knowing about time allows the compiler to efficiently mark entities as being outdated. + +Conceptually, `dotc` works like a clock, where its minutes are represented by [phases](phases.md), +and its hours by [runs]. Like a clock, each run passes once each of its phases have completed +sequentially, and then a new run can begin. Phases are further grouped into [periods], where +during a period certain entities of the compiler remain stable. + +## Time Travel + +During a run, each phase can rewrite the world as the compiler sees it, for example: +- to transform trees, +- to gradually simplify type from Scala types to JVM types, +- to move definitions out of inner scopes to outer ones, fitting the JVM's model, +- and so on. + +Because definitions can [change over time](symbols.md#definitions-are-dynamic), various artifacts associated with them +are stored non-destructively, and views of the definition created earlier, or later +in the compiler can be accessed by using the `atPhase` method, defined in [Contexts]. + +As an example, assume the following definitions are available in a [Context](context.md): +```scala +class Box { type X } + +def foo(b: Box)(x: b.X): List[b.X] = List(x) +``` + +You can compare the type of definition `foo` after the [typer] phase and after the [erasure] phase +by using `atPhase`: +```scala +import dotty.tools.dotc.core.Contexts.{Context, atPhase} +import dotty.tools.dotc.core.Phases.{typerPhase, erasurePhase} +import dotty.tools.dotc.core.Decorators.i + +given Context = … + +val fooDef: Symbol = … // `def foo(b: Box)(x: b.X): List[b.X]` + +println(i"$fooDef after typer => ${atPhase(typerPhase.next)(fooDef.info)}") +println(i"$fooDef after erasure => ${atPhase(erasurePhase.next)(fooDef.info)}") +``` +and see the following output: +``` +method foo after typer => (b: Box)(x: b.X): scala.collection.immutable.List[b.X] +method foo after erasure => (b: Box, x: Object): scala.collection.immutable.List +``` + +[runs]: https://github.com/lampepfl/dotty/blob/a527f3b1e49c0d48148ccfb2eb52e3302fc4a349/compiler/src/dotty/tools/dotc/Run.scala +[periods]: https://github.com/lampepfl/dotty/blob/a527f3b1e49c0d48148ccfb2eb52e3302fc4a349/compiler/src/dotty/tools/dotc/core/Periods.scala +[Contexts]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/Contexts.scala +[typer]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/typer/TyperPhase.scala +[erasure]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/Erasure.scala diff --git a/docs/_docs/contributing/architecture/types.md b/docs/_docs/contributing/architecture/types.md new file mode 100644 index 000000000000..64543e555e69 --- /dev/null +++ b/docs/_docs/contributing/architecture/types.md @@ -0,0 +1,147 @@ +--- +layout: doc-page +title: Compiler Types +--- + +## Common Types and their Representation + +Type representations in `dotc` derive from the class `dotty.tools.dotc.core.Types.Type`, +defined in [Types.scala]. The `toString` method on `Type` will display types in a +format corresponding to the backing data structure, e.g. `ExprType(...)` +corresponds to `class ExprType`, defined in [Types.scala]. + +> You can inspect the representation of any type using the [dotty.tools.printTypes][DottyTypeStealer] +> script, its usage and integration into your debugging workflow is [described here](../issues/inspection.md). + +### Types of Definitions + +The following table describes definitions in Scala 3, followed by the `dotc` representation +of two types - a reference to the definition, and then its underlying type. + +**Note**: in the following types, `p` refers to the self-type of the enclosing scope of +the definition, or `NoPrefix` for local definitions and parameters. + +Definition | Reference | Underlying Type +------------------------|-----------------|------------------------- +`type Z >: A <: B` | `TypeRef(p, Z)` | `RealTypeBounds(A, B)` +`type Z = A` | `TypeRef(p, Z)` | `TypeAlias(A)` +`type F[T] = T match …` | `TypeRef(p, F)` | `MatchAlias([T] =>> T match …)` +`class C` | `TypeRef(p, C)` | `ClassInfo(p, C, …)` +`trait T` | `TypeRef(p, T)` | `ClassInfo(p, T, …)` +`object o` | `TermRef(p, o)` | `TypeRef(p, o$)` where `o$` is a class +`def f(x: A): x.type` | `TermRef(p, f)` | `MethodType(x, A, TermParamRef(x))` +`def f[T <: A]: T` | `TermRef(p, f)` | `PolyType(T, <: A, TypeParamRef(T))` +`def f: A` | `TermRef(p, f)` | `ExprType(A)` +`(x: => A)` | `TermRef(p, x)` | `ExprType(A)` where `x` is a parameter +`val x: A` | `TermRef(p, x)` | `A` + +### Types of Values + +The following types may appear in part of the type of an expression: + +Type | Representation +--------------------------|------------------------------ +`x.y.type` | `TermRef(x, y)` +`X#T` | `TypeRef(X, T)` +`x.y.T` and `x.y.type#T` | `TypeRef(TermRef(x, y), T)` +`this.type` | `ThisType(C)` where `C` is the enclosing class +`"hello"` | `ConstantType(Constant("hello"))` +`A & B` | `AndType(A, B)` +`A | B` | `OrType(A, B)` +`A @foo` | `AnnotatedType(A, @foo)` +`[T <: A] =>> T` | `HKTypeLambda(T, <: A, TypeParamRef(T))` +`x.C[A, B]` | `AppliedType(x.C, List(A, B))` +`C { type A = T }` | `RefinedType(C, A, T)`
when `T` is not a member of `C` +`C { type X = Y }` | `RecType(RefinedType(C, X, z.Y))`
when `X` and `Y` are members of `C`
and `z` is a `RecThis` over the enclosing `RecType` +`super.x.type` | `TermRef(SuperType(…), x)` + +## Constructing Types + +### Method Definition Types + +You can see above that method definitions can have an underlying type of +either `PolyType`, `MethodType`, or `ExprType`. `PolyType` and `MethodType` +may be mixed recursively however, and either can appear as the result type of the other. + +Take this example as given: + +```scala +def f[A, B <: Seq[A]](x: A, y: B): Unit +``` +it can be constructed by the following code: + +```scala +import dotty.tools.dotc.core.Types.* +import dotty.tools.dotc.core.Symbols.* +import dotty.tools.dotc.core.Contexts.* +import dotty.tools.dotc.core.Decorators.* + +given Context = … // contains the definitions of the compiler + +val f: Symbol = … // def f[A, B <: Seq[A]](x: A, y: B): Unit + +f.info = PolyType( + List("A".toTypeName, "B".toTypeName))( + pt => List( + TypeBounds(defn.NothingType, defn.AnyType), + TypeBounds(defn.NothingType, AppliedType(defn.SeqType, List(pt.newParamRef(0)))) + ), + pt => MethodType( + List("x".toTermName, "y".toTermName))( + mt => List(pt.newParamRef(0), pt.newParamRef(1)), + mt => defn.UnitType + ) +) +``` + +Note that `pt.newParamRef(0)` and `pt.newParamRef(1)` refers to the +type parameters `A` and `B` respectively. + +## Proxy Types and Ground Types +Types in `dotc` are divided into two semantic kinds: +- Ground Types (inheriting from either `CachedGroundType` or `UncachedGroundType`) +- Proxy Types (inheriting from `TypeProxy` via either `CachedProxyType` or `UncachedProxyType`) + +A Proxy Type is anything that can be considered to be an abstraction of another type, +which can be accessed by the `underlying` method of the `TypeProxy` class. It's dual, the +Ground Type has no meaningful underlying type, typically it is the type of method and class +definitions, but also union types and intersection types, along with utility types of the +compiler. + +Here's a diagram, serving as the mental model of the most important and distinct types available after the `typer` phase, derived from [dotty/tools/dotc/core/Types.scala][1]: + +``` +Type -+- proxy_type --+- NamedType --------+- TypeRef + | | \ + | +- SingletonType ----+- TermRef + | | +- ThisType + | | +- SuperType + | | +- ConstantType + | | +- TermParamRef + | | +- RecThis + | | +- SkolemType + | +- TypeParamRef + | +- RefinedOrRecType -+-- RefinedType + | | -+-- RecType + | +- AppliedType + | +- TypeBounds + | +- ExprType + | +- AnnotatedType + | +- TypeVar + | +- HKTypeLambda + | +- MatchType + | + +- ground_type -+- AndType + +- OrType + +- MethodOrPoly -----+-- PolyType + | +-- MethodType + +- ClassInfo + +- NoType + +- NoPrefix + +- ErrorType + +- WildcardType + +``` + +[Types.scala]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/Types.scala +[DottyTypeStealer]: https://github.com/lampepfl/dotty/blob/master/compiler/test/dotty/tools/DottyTypeStealer.scala diff --git a/docs/_docs/contributing/contribute-knowledge.md b/docs/_docs/contributing/contribute-knowledge.md deleted file mode 100644 index 7164774ac1df..000000000000 --- a/docs/_docs/contributing/contribute-knowledge.md +++ /dev/null @@ -1,12 +0,0 @@ ---- -layout: doc-page -title: Contributing Knowledge ---- - -# Contribute Internals-related Knowledge -If you know anything useful at all about Dotty, feel free to log this knowledge: - -- [📜Log the Knowledge](https://github.com/lampepfl/dotty-knowledge/issues/new) -- [🎓More about Logging the Knowledge](https://github.com/lampepfl/dotty-knowledge/blob/master/README.md) - -In short, no need to make it pretty, particularly human-readable or give it a particular structure. Just dump the knowledge you have and we'll take it from there. \ No newline at end of file diff --git a/docs/_docs/contributing/getting-started.md b/docs/_docs/contributing/getting-started.md index c842fd0a49d0..239e738bbfbb 100644 --- a/docs/_docs/contributing/getting-started.md +++ b/docs/_docs/contributing/getting-started.md @@ -3,15 +3,41 @@ layout: doc-page title: Getting Started --- +## Scala CLA +Sometime before submitting your pull request you'll want to make sure you have +signed the [Scala CLA][scala-cla]. You can read more about why we require a CLA +and what exactly is included in it [here][scala-cla]. -Requirements ------------- -Make sure that you are using macOS or Linux (or WSL on Windows) with Java 8 or newer. You can determine which version of the JDK is the -default by typing `java -version` in a Terminal window. +## Making sure the team is aware + +Before digging into an issue or starting on a new feature it's a good idea to +make sure an [issue][dotty-issue] or a [discussion][dotty-discussion] has been +created outlining what you plan to work on. This is both for your and the team's +benefit. It ensures you get the help you need, and also gives the compiler team +a heads-up that someone is working on an issue. + +For some small changes like documentation, this isn't always necessary, but it's +never a bad idea to check. + +## Requirements + +- [git] is essential for managing the Scala 3 code, and contributing to GitHub, + where the code is hosted. +- A Java Virtual Machine (JDK 8 or higher), required for running the build tool. + - download Java from [Oracle Java 8][java8], [Oracle Java 11][java11], + or [AdoptOpenJDK 8/11][adopt]. Refer to [JDK Compatibility][compat] for Scala/Java compatibility detail. + - Verify that the JVM is installed by running the following command in a terminal: `java -version`. +- [sbt][sbt-download], the build tool required to build the Scala 3 compiler and libraries. + +## Nice To Have + +An IDE, such as [Metals] will help you develop in Scala 3 with features such as autocompletion or goto-definition, +and with the [VS Code][vs-code] text editor you can even use the Scala debugger, or create interactive worksheets for an +iterative workflow. + +## Compiling and Running -Compiling and Running ---------------------- Start by cloning the repository: ```bash @@ -48,8 +74,8 @@ $ scala HelloWorld ``` -Starting a REPL ---------------- +## Starting a REPL + ```bash $ sbt > repl @@ -64,8 +90,9 @@ or via bash: ```bash $ scala ``` -Publish to local repository ---------------------------------- + +## Publish to local repository + To test our cloned compiler on local projects: ```bash @@ -79,8 +106,8 @@ ThisBuild / scalaVersion := "-bin-SNAPSHOT" where `dotty-version` can be found in the file `project/Build.scala`, like `3.0.0-M2` -Generating Documentation -------------------------- +## Generating Documentation + To generate this page and other static page docs, run ```bash $ sbt @@ -92,9 +119,22 @@ Before contributing to Dotty, we invite you to consult the [Dotty Developer Guidelines](https://github.com/lampepfl/dotty/blob/main/CONTRIBUTING.md). -Community -------------- +## Community + The main development discussion channels are: - [github.com/lampepfl/dotty/discussions](https://github.com/lampepfl/dotty/discussions) - [contributors.scala-lang.org](https://contributors.scala-lang.org) - [gitter.im/scala/contributors](https://gitter.im/scala/contributors) + +[git]: https://git-scm.com +[Metals]: https://scalameta.org/metals/ +[vs-code]: https://code.visualstudio.com +[lampepfl/dotty]: https://github.com/lampepfl/dotty +[sbt-download]: https://www.scala-sbt.org/download.html +[java8]: https://www.oracle.com/java/technologies/javase-jdk8-downloads.html +[java11]: https://www.oracle.com/java/technologies/javase-jdk11-downloads.html +[adopt]: https://adoptopenjdk.net/ +[compat]: /overviews/jdk-compatibility/overview.html +[scala-cla]: https://www.lightbend.com/contribute/cla/scala +[dotty-issue]: https://github.com/lampepfl/dotty/issues +[dotty-discussion]: https://github.com/lampepfl/dotty/discussions diff --git a/docs/_docs/contributing/index.md b/docs/_docs/contributing/index.md index 6cf0def2d5e2..27954aefd7a1 100644 --- a/docs/_docs/contributing/index.md +++ b/docs/_docs/contributing/index.md @@ -2,3 +2,48 @@ layout: index title: Contributing --- + +This guide is intended to give new contributors the knowledge they need to +become productive and fix issues or implement new features in Scala 3. It +also documents the inner workings of the Scala 3 compiler, `dotc`. + +### This is a living document + +Keep in mind that the code for `dotc` is continually changing, so the ideas +discussed in this guide may fall out of date. This is a living document, so +please consider contributing to it on +[GitHub](https://github.com/scala/docs.scala-lang/tree/main/_overviews/scala3-contribution) +if you notice anything out of date, or report any issues +[here](https://github.com/scala/docs.scala-lang/issues). + +### Get the Most from This Guide + +`dotc` is built with Scala 3, fully utilising its [new +features](/scala3/new-in-scala3.html). It is recommended that you first have +some familiarity with Scala 3 to get the most out of this guide. You can learn +more in the [language reference]({{ site.scala3ref }}). + +Many code snippets in this guide make use of shell commands (a line beginning +with `$`), and in this case a `bash` compatible shell is assumed. You may have +to look up how to translate commands to your shell. + +### What is a Compiler? + +Let's start at the beginning and first look at the question of "what is a +compiler?". A compiler is a program that takes as input text, representing a +program in one language and produces as output the same program, written in +another programming language. + +#### The Scala Compiler + +As an example, `dotc` takes text input, verifies that it is a valid Scala program +and then produces as output the same program, but written in Java bytecode, and optionally +in SJSIR when producing Scala.js output. + +### Contribute Internals-related Knowledge +If you know anything useful at all about Dotty, feel free to log this knowledge: + +- [📜Log the Knowledge](https://github.com/lampepfl/dotty-knowledge/issues/new) +- [🎓More about Logging the Knowledge](https://github.com/lampepfl/dotty-knowledge/blob/master/README.md) + +In short, no need to make it pretty, particularly human-readable or give it a particular structure. Just dump the knowledge you have and we'll take it from there. \ No newline at end of file diff --git a/docs/_docs/contributing/issues/areas.md b/docs/_docs/contributing/issues/areas.md new file mode 100644 index 000000000000..4f9adf79ba77 --- /dev/null +++ b/docs/_docs/contributing/issues/areas.md @@ -0,0 +1,70 @@ +--- +layout: doc-page +title: Common Issue Locations +--- + +Many issues are localised to small domains of the compiler and are self-contained, +here is a non-exhaustive list of such domains, and the files associated with them: + +### Pretty Printing of Types and Trees + +Objects in the compiler that inherit from [Showable] can be pretty printed. +The pretty-printing of objects is used in many places, from debug output, +to user-facing error messages and printing of trees after each phase. + +Look in [RefinedPrinter] (or its parent class [PlainPrinter]) for the implementation of pretty printing. + +### Content of Error Messages + +You can find the definitions of most error messages in [messages] (with IDs +defined in [ErrorMessageID]). If the message is not defined there, try the +`-Ydebug-error` compiler flag, which will print a stack trace leading to the +production of the error, and the contents of the message. + +### Compiler Generated Given Instances + +If the issue lies in given instances provided by the compiler, such as `scala.reflect.ClassTag`, +`scala.deriving.Mirror`, `scala.reflect.TypeTest`, `scala.CanEqual`, `scala.ValueOf`, +`scala.reflect.Manifest`, etc, look in [Synthesizer], which provides factories for +given instances. + +### Compiler Generated Methods + +Members can be generated for many classes, such as `equals` and `hashCode` +for case classes and value classes, and `ordinal` and `fromProduct` for Mirrors. +To change the implementation, see [SyntheticMembers]. + +### Code Completions +For suggestions to auto-complete method selections, see [Completion]. + +### Enum Desugaring +See [Desugar] and [DesugarEnums]. + +### Pattern Match Exhaustivity +See [Space]. + +### Metaprogramming + +#### Quotes Reflection +See the [quoted runtime package][quotes-impl]. + +#### Inline match +See [Inliner]. + +#### Compiletime Ops Types +See `tryCompiletimeConstantFold` in [Types]. + +[Showable]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/printing/Showable.scala +[PlainPrinter]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/printing/PlainPrinter.scala +[RefinedPrinter]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/printing/RefinedPrinter.scala +[ErrorMessageID]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/reporting/ErrorMessageID.scala +[messages]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/reporting/messages.scala +[Synthesizer]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/typer/Synthesizer.scala +[SyntheticMembers]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/SyntheticMembers.scala +[quotes-impl]: https://github.com/lampepfl/dotty/tree/master/compiler/src/scala/quoted/runtime/impl +[Inliner]: https://github.com/lampepfl/dotty/blob/main/compiler/src/dotty/tools/dotc/inlines/Inliner.scala +[Types]: https://github.com/lampepfl/dotty/tree/master/compiler/src/dotty/tools/dotc/core/Types.scala +[Completion]: https://github.com/lampepfl/dotty/tree/master/compiler/src/dotty/tools/dotc/interactive/Completion.scala +[DesugarEnums]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/ast/DesugarEnums.scala +[Desugar]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/ast/Desugar.scala +[Space]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/patmat/Space.scala diff --git a/docs/_docs/contributing/issues/cause.md b/docs/_docs/contributing/issues/cause.md new file mode 100644 index 000000000000..5bb04e894f70 --- /dev/null +++ b/docs/_docs/contributing/issues/cause.md @@ -0,0 +1,115 @@ +--- +layout: doc-page +title: Finding the Cause of an Issue +--- + +In this section, you will be able to answer questions such as: +- where does an error happen in a codebase? +- when during compilation was a particular tree introduced? +- where is a particular object created? +- where is a particular value assigned to a variable? + +> You may be able to quickly find the source responsible for an issue by consulting [common issue locations](../issues/areas.md) + +## What phase generated a particular tree? + +As described in the [compiler lifecycle](../architecture/lifecycle.md#phases), each phase transforms the trees +and types that represent your code in a certain way. + +To print the code as it is transformed through the compiler, use the compiler flag `-Xprint:all`. +After each phase group is completed, you will see the resulting trees representing the code. + +> It is recommended to test `-Xprint:all` on a single, small file, otherwise a lot of unnecessary +> output will be generated. + +### Trace a Tree Creation Site + +When you see a problematic tree appear after a certain phase group, you know to isolate the rest of +your search to the code of that phase. For example if you found a problematic tree after phase +`posttyper`, the problem most likely appears in the code of [PostTyper]. We can trace the exact point +the tree was generated by looking for its unique ID, and then generating a stack trace at its creation: + +1. Run the compiler with `-Xprint:posttyper` and `-Yshow-tree-ids` flags. + This will only print the trees of the `posttyper` phase. This time you should see the tree + in question be printed alongside its ID. You'll see something like `println#223("Hello World"#37)`. +2. Copy the ID of the desired tree. +3. Run the compiler with `-Ydebug-tree-with-id ` flag. The compiler will print a stack trace + pointing to the creation site of the tree with the provided ID. + +### Enhanced Tree Printing + +As seen above `-Xprint:` can be enhanced with further configuration flags, found in +[ScalaSettings]. For example, you can additionally print the type of a tree with `-Xprint-types`. + +## Increasing Logging Output +Once you have identified the phase that generated a certain tree, you can then increase +logging in that phase, to try and detect erroneous states: + +- general logging within a phase can be enabled with the `-Ylog` compiler flag, such as + - `-Ylog:,,...` for individual phases + - `-Ylog:all` for all phases. +- Additionally, various parts of the compiler have specialised logging objects, defined in [Printers]. + Change any of the printers of interest from `noPrinter` to `default` and increase output specialised + to that domain. + +## Navigating to Where an Error is Generated + +The compiler issues user facing errors for code that is not valid, such as the type mismatch +of assigning an `Int` to a `Boolean` value. Sometimes these errors do not match what is expected, which could be a bug. + +To discover why such a *spurious* error is generated, you can trace the code that generated the error by +adding the `-Ydebug-error` compiler flag, e.g. `scala3/scalac -Ydebug-error Test.scala`. +This flag forces a stack trace to be printed each time an error happens, from the site where it occurred. + +Analysing the trace will give you a clue about the objects involved in producing the error. +For example, you can add some debug statements before the error is issued to discover +the state of the compiler. [See some useful ways to debug values.](./inspection.md) + +### Where was a particular object created? + +If you navigate to the site of the error, and discover a problematic object, you will want to know +why it exists in such a state, as it could be the cause of the error. You can discover the +creation site of that object to understand the logic that created it. + +You can do this by injecting a *tracer* into the class of an instance in question. +A tracer is the following variable: +```scala +val tracer = Thread.currentThread.getStackTrace.mkString("\n") +``` +When placed as a member definition at a class, it will contain a stack trace pointing at where exactly +its particular instance was created. + +Once you've injected a tracer into a class, you can `println` that tracer from the error site or +other site you've found the object in question. + +#### Procedure + +1. Determine the type of the object in question. You can use one of the following techniques to do so: + - Use an IDE to get the type of an expression, or save the expression to a `val` + and see its inferred type. + - Use `println` to print the object or use `getClass` on that object. +2. Locate the type definition for the type of that object. +3. Add a field `val tracer = Thread.currentThread.getStackTrace.mkString("\n")` to that type definition. +4. `println(x.tracer)` (where `x` is the name of the object in question) from the original site where you + encountered the object. This will give you the stack trace pointing to the place where the + constructor of that object was invoked. + +### Where was a particular value assigned to a variable? + +Say you have a certain [type](../architecture/types.md) assigned to a [Denotation] and you would like to know why it has that +specific type. The type of a denotation is defined by `var myInfo: Type`, and can be assigned multiple times. +In this case, knowing the creation site of that `Type`, as described above, is not useful; instead, you need to +know the *assignment* (not *creation*) site. + +This is done similarly to how you trace the creation site. Conceptually, you need to create a proxy for that variable that will log every write operation to it. Practically, if you are trying to trace the assignments to a variable `myInfo` of type `Type`, first, rename it to `myInfo_debug`. Then, insert the following at the same level as that variable: + +```scala +var tracer = "", +def myInfo: Type = myInfo_debug, +def myInfo_=(x: Type) = { tracer = Thread.currentThread.getStackTrace.mkString("\n"); myInfo_debug = x } +``` + +[Printers]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/config/Printers.scala +[Denotation]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/Denotations.scala +[PostTyper]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/transform/PostTyper.scala +[ScalaSettings]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/config/ScalaSettings.scala diff --git a/docs/_docs/contributing/issues/checklist.md b/docs/_docs/contributing/issues/checklist.md new file mode 100644 index 000000000000..e2fcf32531de --- /dev/null +++ b/docs/_docs/contributing/issues/checklist.md @@ -0,0 +1,135 @@ +--- +layout: doc-page +title: Pull Request Checklist +--- + +Once you solved the issue you were working on, you'll likely want to see your +changes added to the [Scala 3 repo][lampepfl/dotty]. To do that, you need to +prepare a [pull request][pull-request] with your changes. Assuming that the team +is aware of what you've been working, here are some final steps that you'll want +to keep in mind as you create your PR. + +### 1. Sign the CLA + +Make sure you have signed the [Scala CLA][cla]. If you have any questions about +what this is and why it's required you can read further about it [here][cla]. + +### 2. Make sure your work is on its own branch + +When submitting your pull request it's always best to ensure the branch name is +unique to the changes you're working on. It's important not to submit your PR on +your `main` branch as this blocks maintainers from making any changes to your PR +if necessary. + +### 3: Add Tests + +Add at least one test that replicates the problem in the issue, and that shows it is now resolved. + +You may of course add variations of the test code to try and eliminate edge cases. +[Become familiar with testing in Scala 3](./testing.md). + +### 4: Add Documentation + +Please ensure that all code is documented to explain its use, even if only internal +changes are made. This refers to scaladocs and also any changes that might be +necessary in the reference docs. + +### 5: Double check everything + +Here are a couple tips to keep in mind. + +- [DRY (Don't Repeat Yourself)][dry] +- [Scouts Rule][scouts] +- When adding new code try use [optional braces]. If you're rewriting old code, + you should also use optional braces unless it introduces more code changes + that necessary. + +### 6: Commit Messages + +Here are some guidelines when writing commits for Dotty. + +1. If your work spans multiple local commits (for example; if you do safe point + commits while working in a feature branch or work in a branch for long time + doing merges/rebases etc.) then please do not commit it all but rewrite the + history by squashing the commits into one large commit which is accompanied + by a detailed commit message for (as discussed in the following sections). + For more info, see the article: [Git Workflow][git-workflow]. Additionally, + every commit should be able to be used in isolation—that is, each commit must + build and pass all tests. + +2. The first line should be a descriptive sentence about what the commit is + doing. It should be possible to fully understand what the commit does by just + reading this single line. It is **not ok** to only list the ticket number, + type "minor fix" or similar. If the commit has a corresponding ticket, + include a reference to the ticket number, prefixed with "Closes #", at the + beginning of the first line followed by the title of the ticket, assuming + that it aptly and concisely summarizes the commit in a single line. If the + commit is a small fix, then you are done. If not, go to 3. + +3. Following the single line description (ideally no more than 70 characters + long) should be a blank line followed by an enumerated list with the details + of the commit. + +4. Add keywords for your commit (depending on the degree of automation we reach, + the list may change over time): + * ``Review by @githubuser`` - will notify the reviewer via GitHub. Everyone + is encouraged to give feedback, however. (Remember that @-mentions will + result in notifications also when pushing to a WIP branch, so please only + include this in your commit message when you're ready for your pull + request to be reviewed. Alternatively, you may request a review in the + pull request's description.) + * ``Fix/Fixing/Fixes/Close/Closing/Refs #ticket`` - if you want to mark the + ticket as fixed in the issue tracker (Assembla understands this). + * ``backport to _branch name_`` - if the fix needs to be cherry-picked to + another branch (like 2.9.x, 2.10.x, etc) + +Example: + +``` +fix: here is your pr title briefly mentioning the topic + +Here is the body of your pr with some more information + - Details 1 + - Details 2 + - Details 3 + +Closes #2 +``` + +### 7: Create your PR! + +When the feature or fix is completed you should open a [Pull +Request](https://help.github.com/articles/using-pull-requests) on GitHub. + +If you're not actually finished yet and are just looking for some initial input +on your approach, feel free to open a [Draft PR][draft]. This lets reviewers +know that you're not finished yet. It's also a good idea to put a [wip] in front +of your pr title to make this extra clear. + +Shortly after creating your pull request a maintainer should assign someone to +review it. If this doesn't happen after a few days, feel free to ping someone on +the [Scala Contributors Discor][discord] or tag someone on the PR. Depending on +the type of pull request there might be multiple people that take a look at your +changes. There might also be community input as we try to keep the review +process as open as possible. + +### 8: Addressing feedback + +More than likely you'll get feedback from the reviewers, so you'll want to make +sure to address everything. When in doubt, don't hesitate to ask for +clarification or more information. + +Once you finally see the "LGTM" (Looks Good To Me or Let's Get This Merged) +you're PR will be merged in! + +[pull-request]: https://docs.github.com/en?query=pull+requests +[lampepfl/dotty]: https://github.com/lampepfl/dotty +[cla]: http://typesafe.com/contribute/cla/scala +[issues]: https://github.com/lampepfl/dotty/issues +[full-list]: https://github.com/lampepfl/dotty/blob/master/CONTRIBUTING.md +[discord]: https://discord.gg/TSmY9zkHar +[dry]: https://www.oreilly.com/library/view/97-things-every/9780596809515/ch30.html +[scouts]: https://www.oreilly.com/library/view/97-things-every/9780596809515/ch08.html +[optional-braces]: https://docs.scala-lang.org/scala3/reference/other-new-features/indentation.html +[draft]: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests#draft-pull-requests +[git-workflow]: http://sandofsky.com/blog/git-workflow.html diff --git a/docs/_docs/contributing/issues/debugging.md b/docs/_docs/contributing/issues/debugging.md new file mode 100644 index 000000000000..2d8a9e5941e4 --- /dev/null +++ b/docs/_docs/contributing/issues/debugging.md @@ -0,0 +1,189 @@ +--- +layout: doc-page +title: Debugging the Compiler +--- + +The debugger is a powerful tool to navigate the internals of the compiler and track bugs. + +You can start the Scala debugger in VSCode using [Metals](https://scalameta.org/metals/). +In this page you will learn how to configure it, and how to use it. + +## Importing the project in VSCode using Metals + +The first step is to import the build in Metals, if it has not yet been imported. + +To do so you can open the [lampefl/dotty][lampepfl/dotty] repository in VSCode and click `Import build` in Metals view. +It may take a few minutes to import, compile and index the full project. + +![Import build](/images/contribution/import-build.jpg) + +If you have any trouble with importing, you can try to switch the build server from Bloop to sbt, +by running the `Metals: Switch build server` command from VSCode command palette. + +## Configuring the debugger + +To configure the debugger in VSCode, you can go to the `Run and Debug` view and click `create a launch.json file`. +It creates the `launch.json` file in the `.vscode` folder, in which we will define the debug configurations. + +![Create launch.json file](/images/contribution/launch-config-file.jpg) + +To create a debug configuration: +- Open the `.vscode/launch.json` file +- Click the `Add Configuration` button +- Go down the list of templates and select `Scala: Run main class` + +![Create configuration](/images/contribution/create-config.jpg) + +The added configuration should look like this: +```json +{ + "type": "scala", + "request": "launch", + "name": "Untitled", + "mainClass": "???", + "args": [], + "jvmOptions": [], + "env": {} +} +``` + +This is a template that you need to fill out. +First You can give a `name` to your configuration, for instance `Debug Scala 3 Compiler`. + +The two most important parameters, to debug the compiler, are `mainClass` and `args`. +The `mainClass` of the compiler is `dotty.tools.dotc.Main`. +In the `args` you need to specify the compiler arguments, which must contain at least a Scala file to compile and a `-classpath` option. + +To start with, we can compile the `../tests/pos/HelloWorld.scala` file. +In the classpath, we always need at least the `scala-library_2.13` and the bootstrapped `scala3-library_3`. +To locate them on your filesystem you can run the `export scala3-library-bootstrapped/fullClasspath` command in sbt. + +``` +$ sbt +> export scala3-library-bootstrapped/fullClasspath +/home/user/lampepfl/dotty/out/bootstrap/scala3-library-bootstrapped/scala-3.3.1-RC1-bin-SNAPSHOT-nonbootstrapped/classes:/home/user/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.10/scala-library-2.13.10.jar +[success] Total time: 1 s, completed Mar 10, 2023, 4:37:43 PM +``` + +Note that it is important to use the bootstrapped version of the `scala3-library` to get the correct TASTy version. + +Additionally you can add the `-color` and `never` arguments to prevent the compiler from printing ANSI codes as strings in the debug console. + +Here is the final configuration: +```json +{ + "type": "scala", + "request": "launch", + "name": "Debug Scala 3 Compiler", + "mainClass": "dotty.tools.dotc.Main", + "args": [ + "../tests/pos/HelloWorld.scala", + "-classpath", + // To replace with your own paths + "/home/user/lampepfl/dotty/out/bootstrap/scala3-library-bootstrapped/scala-3.3.1-RC1-bin-SNAPSHOT-nonbootstrapped/classes:/home/user/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.10/scala-library-2.13.10.jar", + "-color", + "never" + ], + "jvmOptions": [], + "env": {} +} +``` + +## Customizing the debug configurations + +### Compiling several files at once + +You can compile more than one Scala file, by adding them in the `args`: +```json +"args": [ + "file1.scala", + "file2.scala", + "-classpath", + "/home/user/lampepfl/dotty/out/bootstrap/scala3-library-bootstrapped/scala-3.3.1-RC1-bin-SNAPSHOT-nonbootstrapped/classes:/home/user/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.10/scala-library-2.13.10.jar" +] +``` + +### Depending on a library + +To add a dependency to an external library you need to download it and all its transitive dependencies, and to add them in the classpath. +The Coursier CLI can help you to do that. +For instance to add a dependency to cats you can run: +``` +$ cs fetch org.typelevel::cats-core:2.+ --classpath --scala-version 3 --exclude org.scala-lang:scala-library --exclude org.scala-lang:scala3-library +/home/user/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-core_3/2.9.0/cats-core_3-2.9.0.jar:/home/user/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-kernel_3/2.9.0/cats-kernel_3-2.9.0.jar +``` + +And concatenate the output into the classpath argument, which should already contain the scala-library_2.13 and the bootstrapped scala3-library: + +```json +"args": [ + "using-cats.scala", + "-classpath", + "/home/user/lampepfl/dotty/out/bootstrap/scala3-library-bootstrapped/scala-3.3.1-RC1-bin-SNAPSHOT-nonbootstrapped/classes:/home/user/.cache/coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.13.10/scala-library-2.13.10.jar:/home/user/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-core_3/2.9.0/cats-core_3-2.9.0.jar:/home/user/.cache/coursier/v1/https/repo1.maven.org/maven2/org/typelevel/cats-kernel_3/2.9.0/cats-kernel_3-2.9.0.jar" +] +``` + +### Add more compiler options + +In the `args` you can add any additional compiler option you want. + +For instance you can add `-Xprint:all` to print all the generated trees after each mega phase. + +Run `scalac -help` to get an overview of the available compiler options. + +### Defining more than one launch configuration + +You can create as many debug configurations as you need: to compile different files, with different compiler options or different classpaths. + +## Starting the debugger + +Before starting the debugger you need to put a breakpoint in the part of the code that you want to debug. +If you don't know where to start, you can put a breakpoint in the `main` method of the `dotty.tools.dotc.Driver` trait. + +![First breakpoint](/images/contribution/breakpoint.jpg) + +Now to start the debugger, open the debug view, find the drop-down list of all the debug configurations and click on yours. +The debugger should start and pause on your breakpoint. + +![Start debugger](/images/contribution/start-debugger.jpg) + +## Using the debugger + +### Navigating the call stack + +When the debugger has paused, you can see the current call stack in the `Debug and Run` view. +Each frame of the call stack contains different variables, whose values you can see in the `Variables` section of the `Debug and Run` view. + +![Call stack](/images/contribution/call-stack.jpg) + +Analysing the call stack and the variables can help you understand the path taken by the compiler to reach that state. + +### The debugging steps + +The debug toolbar contains the `Continue / Pause`, `Step Over`, `Step Into`, `Step Out`, `Restart` and `Stop` buttons. + +![Debugging steps](/images/contribution/toolbar.jpg) + +You can use the step buttons to execute the code step by step and get a precise understanding of the program. + +### The debug console + +When the debugger has paused, you can evaluate any Scala 3 expression in the debug console. +This is useful to inspect some values or to execute some parts of the code. + +For instance, you can evaluate `tree.show` to pretty-print a tree. + +![Import build](/images/contribution/debug-console.jpg) + +### Conditional breakpoints + +In a breakpoint you can define a condition, in the form of a Boolean expression written in Scala. +The program will stop on the breakpoint as soon as the condition is met. + +To add a condition, right-click on a breakpoint and pick `Edit breakpoint...`. + +For instance, if you know that a bug happens on typing a method `foo`, you can use the condition `tree.symbol.name.show == "foo"` in a breakpoint in the `Typer`. + +![Import build](/images/contribution/conditional-breakpoint.jpg) + +[lampepfl/dotty]: https://github.com/lampepfl/dotty diff --git a/docs/_docs/contributing/issues/efficiency.md b/docs/_docs/contributing/issues/efficiency.md new file mode 100644 index 000000000000..07307646a4bb --- /dev/null +++ b/docs/_docs/contributing/issues/efficiency.md @@ -0,0 +1,24 @@ +--- +layout: doc-page +title: Improving Your Workflow +--- + +In the previous sections of this chapter, you saw some techniques for +working with the compiler. Some of these techniques can be used +repetitively, e.g.: + +- Navigating stack frames +- Printing variables in certain ways +- Instrumenting variable definitions with tracers + +The above procedures often take a lot of time when done manually, reducing productivity: +as the cost (in terms of time and effort) is high, you may avoid attempting to do so, +and possibly miss valuable information. + +If you're doing those things really frequently, it is recommended to script your editor +to reduce the number of steps. E.g. navigating to the definition of a stack frame +part when you click it, or instrumenting variables for printing. + +An example of how it is done for Sublime Text 3 is [here](https://github.com/anatoliykmetyuk/scala-debug-sublime). + +True, it takes some time to script your editor, but if you spend a lot of time with issues, it pays off. diff --git a/docs/_docs/contributing/issues/index.md b/docs/_docs/contributing/issues/index.md new file mode 100644 index 000000000000..db348d7edd9d --- /dev/null +++ b/docs/_docs/contributing/issues/index.md @@ -0,0 +1,17 @@ +--- +layout: index +title: Finding the Cause of an Issue +--- + +An issue found in the [GitHub repo][lampepfl/dotty] usually describes some code that +manifests undesired behaviour. + +This chapter of the guide describes the different steps to contribute to Dotty: +- [Reproducing an Issue](./reproduce.md) +- [Finding the Cause of an Issue](./cause.md) +- [Debugging the Compiler](./debugging.md) +- [Other debugging techniques](./other-debugging.md) +- [Inspect the values](./inspection.md) +- [Improving your workflow](./efficiency.md) +- [Testing a Fix](./testing.md) +- [Checklist](./checklist.md) diff --git a/docs/_docs/contributing/issues/inspection.md b/docs/_docs/contributing/issues/inspection.md new file mode 100644 index 000000000000..abedc09ecd3b --- /dev/null +++ b/docs/_docs/contributing/issues/inspection.md @@ -0,0 +1,181 @@ +--- +layout: doc-page +title: How to Inspect Values +--- + +In this section, you will find out how to debug the contents of certain objects +while the compiler is running, and inspect produced artifacts of the compiler. + +## Inspecting variables in-place + +Frequently you will need to inspect the content of a particular variable. +You can either use `println`s or the debugger, more info on how to setup the latter. + +In the remeainder of this article we'll use `println()` inserted in the code, but the same effect can be accomplished by stopping at a breakpoint, and typing `` in the [debug console](./debugging.md#the-debug-console) of the debugger. + +When printing a variable, it's always a good idea to call `show` on that variable: `println(x.show)`. +Many objects of the compiler define `show`, returning a human-readable string. +e.g. if called on a tree, the output will be the tree's representation as source code, rather than +the underlying raw data. + +Sometimes you need to print flags. Flags are metadata attached to [symbols] containing information such as whether a +class is abstract, comes from Java, what modifiers a variable has (private, protected etc) and so on. +Flags are stored in a single `Long` value, each bit of which represents whether a particular flag is set. + +To print flags, you can use the `flagsString` method, e.g. `println(x.flagsString)`. + +## Pretty Printing with a String Interpolator + +You can also pretty print objects with string interpolators, +these default to call `.show` when possible, avoiding boilerplate +and also helping format error messages. + +Import them with the following: + +```scala +import dotty.tools.dotc.core.Decorators.* +``` + +Here is a table of explanations for their use: + +| Usage | Description | +|--------|-----------------------------------| +|`i""` | General purpose string formatting. It calls `.show` on objects
mixing in Showable, `String.valueOf` otherwise | +|`em""` | Formatting for error messages: Like `i` but suppress
follow-on, error messages after the first one if some
of their arguments are "non-sensical". | +|`ex""` | Formatting with added explanations: Like `em`, but add
explanations to give more info about type variables
and to disambiguate where needed. | + + +## Obtaining debug output from the compiler + +As explained in [navigation](../issues/cause.md), we can debug the code being generated as it is transformed +through the compiler. As well as plain tree output, there are many compiler options that +add extra debug information to trees when compiling a file; you can find the full list +in [ScalaSettings]. + +## Stopping the compiler early +Sometimes you may want to stop the compiler after a certain phase, for example to prevent +knock-on errors from occurring from a bug in an earlier phase. Use the flag +`-Ystop-after:` to prevent any phases executing afterwards. + +> e.g. `-Xprint:` where `phase` is a miniphase, will print after +> the whole phase group is complete, which may be several miniphases after `phase`. +> Instead you can use `-Ystop-after: -Xprint:` to stop +> immediately after the miniphase and see the trees that you intended. + +## Printing TASTy of a Class + +If you are working on an issue related to TASTy, it is good to know how to inspect +the contents of a TASTy file, produced from compilation of Scala files. + +The next example uses an [issue directory](../issues/reproduce.md#dotty-issue-workspace) to compile a class and print its TASTy. +In the directory, you should create a file `tasty/Foo.scala` (with contents of `class Foo`), +and create a file `tasty/launch.iss` with the following contents: + +``` +$ (rm -rv out || true) && mkdir out # clean up compiler output, create `out` dir. + +scala3/scalac -d $here/out $here/Foo.scala + +scala3/scalac -print-tasty $here/out/Foo.tasty +``` + +With sbt command `issue tasty` you will see output such as the following: + +``` +-------------------------------------------------------------------------------- +local/foo/out/Foo.tasty +-------------------------------------------------------------------------------- +Names: + 0: ASTs + 1: + 2: Foo + 3: +... +``` +and so on. + +## Inspecting The Representation of Types + +> [learn more about types](../architecture/types.md) in `dotc`. + +If you are curious about the representation of a type, say `[T] =>> List[T]`, +you can use a helper program [dotty.tools.printTypes][DottyTypeStealer], +it prints the internal representation of types, along with their class. It can be +invoked from the sbt shell with three arguments as follows: +```bash +sbt:scala3> scala3-compiler/Test/runMain + dotty.tools.printTypes + + + +``` + +- The first argument, `source`, is an arbitrary string that introduces some Scala definitions. +It may be the empty string `""`. +- The second argument, `kind`, determines the format of the following arguments, +accepting one of the following options: + - `rhs` - accept return types of definitions + - `class` - accept signatures for classes + - `method` - accept signatures for methods + - `type` - accept signatures for type definitions + - The empty string `""`, in which case `rhs` will be assumed. +- The remaining arguments are type signature strings, accepted in the format determined by +`kind`, and collected into a sequence `typeStrings`. Signatures are the part of a definition +that comes after its name, (or a simple type in the case of `rhs`) and may reference +definitions introduced by the `source` argument. + +Each one of `typeStrings` is then printed, displaying their internal structure, alongside their class. + +### Examples + +Here, given a previously defined `class Box { type X }`, you can inspect the return type `Box#X`: +```bash +sbt:scala3> scala3-compiler/Test/runMain +> dotty.tools.printTypes +> "class Box { type X }" +> "rhs" +> "Box#X" +[info] running (fork) dotty.tools.printTypes "class Box { type X }" rhs Box#X +TypeRef(TypeRef(ThisType(TypeRef(NoPrefix,module class )),class Box),type X) [class dotty.tools.dotc.core.Types$CachedTypeRef] +``` + +Here are some other examples you can try: +- `...printTypes "" "class" "[T] extends Seq[T] {}"` +- `...printTypes "" "method" "(x: Int): x.type"` +- `...printTypes "" "type" "<: Int" "= [T] =>> List[T]"` + +### Don't just print: extracting further information + +`dotty.tools.printTypes` is useful to to see the representation +of a type at a glance, but sometimes you want to extract more. Instead, you can use the +method `dotty.tools.DottyTypeStealer.stealType`. With the same inputs as `printTypes`, +it returns both a `Context` containing the definitions passed, along with the list of types. + +As a worked example let's create a test case to verify the structure of `Box#X` that you saw earlier: +```scala +import dotty.tools.dotc.core.Contexts.Context +import dotty.tools.dotc.core.Types.* + +import org.junit.Test + +import dotty.tools.DottyTypeStealer, DottyTypeStealer.Kind + +class StealBox: + + @Test + def stealBox: Unit = + val (ictx, List(rhs)) = + DottyTypeStealer.stealType("class Box { type X }", Kind.rhs, "Box#X") + + given Context = ictx + + rhs match + case X @ TypeRef(Box @ TypeRef(ThisType(empty), _), _) => + assert(Box.name.toString == "Box") + assert(X.name.toString == "X") + assert(empty.name.toString == "") +``` + +[DottyTypeStealer]: https://github.com/lampepfl/dotty/blob/master/compiler/test/dotty/tools/DottyTypeStealer.scala +[ScalaSettings]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/config/ScalaSettings.scala +[symbols]: https://github.com/lampepfl/dotty/blob/master/compiler/src/dotty/tools/dotc/core/SymDenotations.scala diff --git a/docs/_docs/contributing/debugging.md b/docs/_docs/contributing/issues/other-debugging.md similarity index 94% rename from docs/_docs/contributing/debugging.md rename to docs/_docs/contributing/issues/other-debugging.md index 959ad6706290..1aa0fb85e5f8 100644 --- a/docs/_docs/contributing/debugging.md +++ b/docs/_docs/contributing/issues/other-debugging.md @@ -1,26 +1,8 @@ --- layout: doc-page -title: Debugging Techniques +title: Other Debugging Techniques --- -# Debugging Techniques -- [Setting up the playground](#setting-up-the-playground) -- [Show for human readable output](#show-for-human-readable-output) -- [How to disable color](#how-to-disable-color) -- [Reporting as a non-intrusive println](#reporting-as-a-non-intrusive-println) -- [Printing out trees after phases](#printing-out-trees-after-phases) -- [Printing out stack traces of compile time errors](#printing-out-stack-traces-of-compile-time-errors) -- [Configuring the printer output](#configuring-the-printer-output) -- [Figuring out an object creation site](#figuring-out-an-object-creation-site) - * [Via ID](#via-id) - * [Via tracer](#via-tracer) -- [Built-in Logging Architecture](#built-in-logging-architecture) - * [Printers](#printers) - * [Tracing](#tracing) - * [Reporter](#reporter) - -Table of contents generated with markdown-toc - ## Setting up the playground Consider the `../issues/Playground.scala` (relative to the Dotty directory) file is: diff --git a/docs/_docs/contributing/issues/reproduce.md b/docs/_docs/contributing/issues/reproduce.md new file mode 100644 index 000000000000..41d96327ef24 --- /dev/null +++ b/docs/_docs/contributing/issues/reproduce.md @@ -0,0 +1,127 @@ +--- +layout: doc-page +title: Reproducing an Issue +--- + +To try fixing it, you will first need to reproduce the issue, so that +- you can understand its cause +- you can verify that any changes made to the codebase have a positive impact on the issue. + +Say you want to reproduce locally issue [#7710], you would first copy the code from the *"Minimised Code"* +section of the issue to a file named e.g. `local/i7710.scala`, +and then try to compile it from the sbt console opened in the dotty root directory: +```bash +$ sbt +sbt:scala3> scala3/scalac -d local/out local/i7710.scala +``` +> Here, the `-d` flag specifies a directory `local/out` where generated code will be output. + +You can then verify that the local reproduction has the same behaviour as originally reported in the issue. +If so, then you can start to try and fix it. Otherwise, perhaps the issue is out of date, or +is missing information about how to accurately reproduce the issue. + +## Dotty Issue Workspace + +Sometimes you will need more complex commands to reproduce an issue, and it is useful to script these, which +can be done with [dotty-issue-workspace]. It allows to bundle sbt commands for issue reproduction in one +file and then run them from the Dotty project's sbt console. + +### Try an Example Issue + +Let's use [dotty-issue-workspace] to reproduce issue [#7710]: +1. Follow [the steps in the README][workspace-readme] to install the plugin. +2. In your Issue Workspace directory (as defined in the plugin's README file, + "Getting Started" section, step 2), create a subdirectory for the + issue: `mkdir i7710`. +3. Create a file for the reproduction: `cd i7710; touch Test.scala`. In that file, + insert the code from the issue. +4. In the same directory, create a file `launch.iss` with the following content: + ```bash + $ (rm -rv out || true) && mkdir out # clean up compiler output, create `out` dir. + + scala3/scalac -d $here/out $here/Test.scala + ``` + + - The first line, `$ (rm -rv out || true) && mkdir out` specifies a shell command + (it starts with `$`), in this case to ensure that there is a fresh `out` + directory to hold compiler output. + - The next line, `scala3/scalac -d $here/out $here/Test.scala` specifies an sbt + command, which will compile `Test.scala` and place any output into `out`. + `$here` is a special variable that will be replaced by the path of the parent + directory of `launch.iss` when executing the commands. +5. Now, from a terminal you can run the issue from sbt in the dotty directory + ([See here](../getting-started.md#compiling-and-running) for a reminder if you have not cloned the repo.): + ```bash + $ sbt + sbt:scala3> issue i7710 + ``` + This will execute all the commands in the `i7710/launch.iss` file one by one. + If you've set up `dotty-issue-workspace` as described in its README, + the `issue` task will know where to find the folder by its name. + +### Using Script Arguments + +You can use script arguments inside `launch.iss` to reduce the number of steps when +working with issues. + +Say you have an issue `foo`, with two alternative files that are very similar: +`original.scala`, which reproduces the issue, and `alt.scala`, which does not, +and you want to compile them selectively? + +You can achieve this via the following `launch.iss`: + +```bash +$ (rm -rv out || true) && mkdir out # clean up compiler output, create `out` dir. + +scala3/scalac -d $here/out $here/$1.scala # compile the first argument following `issue foo ` +``` + +It is similar to the previous example, except now you will compile a file `$1.scala`, referring +to the first argument passed after the issue name. The command invoked would look like +`issue foo original` to compile `original.scala`, and `issue foo alt` for `alt.scala`. + +In general, you can refer to arguments passed to the `issue ` command using +the dollar notation: `$1` for the first argument, `$2` for the second and so on. + +### Multiline Commands + +Inside a `launch.iss` file, one command can be spread accross multiple lines. For example, +if your command has multiple arguments, you can put each argument on a new line. + +Multiline commands can even have comments inbetween lines. This is useful +if you want to try variants of a command with optional arguments (such as configuration). +You can put the optional arguments on separate lines, and then decide when they are passed to +the command by placing `#` in front to convert it to a comment (i.e. the argument will +not be passed). This saves typing the same arguments each time you want to use them. + +The following `launch.iss` file is an example of how you can use multiline commands as a +template for solving issues that [run compiled code](../issues/testing.md#checking-program-output). It demonstrates configuring the +`scala3/scalac` command using compiler flags, which are commented out. +Put your favourite flags there for quick usage. + +```bash +$ (rm -rv out || true) && mkdir out # clean up compiler output, create `out` dir. + +scala3/scalac # Invoke the compiler task defined by the Dotty sbt project + -d $here/out # All the artefacts go to the `out` folder created earlier + # -Xprint:typer # Useful debug flags, commented out and ready for quick usage. Should you need one, you can quickly access it by uncommenting it. + # -Ydebug-error + # -Yprint-debug + # -Yprint-debug-owners + # -Yshow-tree-ids + # -Ydebug-tree-with-id 340 + # -Ycheck:all + $here/$1.scala # Invoke the compiler on the file passed as the second argument to the `issue` command. E.g. `issue foo Hello` will compile `Hello.scala` assuming the issue folder name is `foo`. + +scala3/scala -classpath $here/out Test # Run main method of `Test` generated by the compiler run. +``` + +## Conclusion + +In this section, you have seen how to reproduce an issue locally, and next you will see +how to try and detect its root cause. + +[lampepfl/dotty]: https://github.com/lampepfl/dotty/issues +[#7710]: https://github.com/lampepfl/dotty/issues/7710 +[dotty-issue-workspace]: https://github.com/anatoliykmetyuk/dotty-issue-workspace +[workspace-readme]: https://github.com/anatoliykmetyuk/dotty-issue-workspace#getting-started \ No newline at end of file diff --git a/docs/_docs/contributing/issues/testing.md b/docs/_docs/contributing/issues/testing.md new file mode 100644 index 000000000000..1f7c35c6d58a --- /dev/null +++ b/docs/_docs/contributing/issues/testing.md @@ -0,0 +1,212 @@ +--- +layout: doc-page +title: Testing Your Changes +--- + +It is important to add tests before a pull request, to verify that everything is working as expected, +and act as proof of what is valid/invalid Scala code (in case it is broken in the future). +In this section you will see the testing procedures in Scala 3. + +## Running all Tests + +Running all tests in Dotty is as simple as: + +```bash +$ sbt test +``` +Specifically, `sbt test` runs all tests that do _not_ require a bootstrapped +compiler. In practice, this means that it runs all compilation tests meeting +this criterion, as well as all non-compiler tests. + +To run all tests of Scala 3, including for compiler, REPL, libraries and more, run the following in sbt: + +```bash +$ sbt +sbt:scala3> scala3-bootstrapped/test +``` + +Often however it is not necessary to test everything if your changes are localised to one area, +you will see in the following sections the different kinds of tests, and how +to run individual tests. + +## Compilation Tests + +Compilation tests run the compiler over input files, using various settings. Input files +are found within the `tests/` directory at the root of the compiler repo. + +Test input files are categorised further by placing them in the subdirectories +of the `tests/` directory. A small selection of test categories include: + +- `tests/pos` – tests that should compile: pass if compiles successfully. +- `tests/neg` – should not compile: pass if fails compilation. Useful, e.g., to test an expected compiler error. +- `tests/run` – these tests not only compile but are also run. + +### Naming and Running a Test Case + +Tests are, by convention, named after the number of the issue they are fixing. +e.g. if you are fixing issue 101, then the test should be named `i101.scala`, for a single-file test, +or be within a directory called `i101/` for a multi-file test. + +To run the test, invoke the sbt command `testCompilation i101` (this will match all tests with `"i101"` in +the name, so it is useful to use a unique name) + +The test groups – `pos`, `neg`, etc. – are defined in [CompilationTests]. If you want to run a group +of tests, e.g. `pos`, you can do so via `testOnly *CompilationTests -- *pos` command. + +### Testing a Single Input File + +If your issue is reproducible by only one file, put that file under an appropriate category. +For example, if your issue is about getting rid of a spurious compiler error (that is a code that doesn't compile should, in fact, compile), you can create a file `tests/pos/i101.scala`. + +### Testing Multiple Input Files + +If you need more than one file to reproduce an issue, create a directory instead of a file +e.g. `tests/pos/i101/`, and put all the Scala files that are needed to reproduce the issue there. +There are two ways to organise the input files within: + +**1: Requiring classpath dependency:** Sometimes issues require one file to be compiled after the other, +(e.g. if the issue only happens with a library dependency, like with Java interop). In this case, +the outputs of the first file compiled will be available to the next file compiled, available via the classpath. +This is called *separate compilation*. + +To achieve this, within `tests/pos/i101/`, add a suffix `_n` to each file name, where `n` is an integer defining the +order in which the file will compile. E.g. if you have two files, `Lib.scala` and `Main.scala`, and you need them +compiled separately – Lib first, Main second, then name them `Lib_1.scala` and `Main_2.scala`. + +**2: Without classpath dependency:** If your issue does not require a classpath dependency, your files can be compiled +in a single run, this is called *joint compilation*. In this case use file names without the `_n` suffix. + +### Checking Program Output + +`tests/run` tests verify the run-time behaviour of a test case. The output is checked by invoking a main method +on a class `Test`, this can be done with either +```scala +@main def Test: Unit = assert(1 > 0) +``` +or +```scala +object Test extends scala.App: + assert(1 > 0) +``` + +If your program also prints output, this can be compared against `*.check` files. +These contain the expected output of a program. Checkfiles are named after the issue they are checking, +e.g. `tests/run/i101.check` will check either `tests/run/i101.scala` or `tests/run/i101/`. + +### Checking Compilation Errors + +`tests/neg` tests verify that a file does not compile, and user-facing errors are produced. There are other neg +categories such as `neg-custom-args`, i.e. with `neg` prefixing the directory name. Test files in the `neg*` +categories require annotations for the lines where errors are expected. To do this add one `// error` token to the +end of a line for each expected error. For example, if there are three expected errors, the end of the line should contain +`// error // error // error`. + +You can verify the content of the error messages with a `*.check` file. These contain the expected output of the +compiler. Checkfiles are named after the issue they are checking, +e.g. `i101.check` will check either `tests/neg/i101.scala` or `tests/neg/i101/`. +*Note:* checkfiles are not required for the test to pass, however they do add stronger constraints that the errors +are as expected. + +### If Checkfiles do not Match Output + +If the actual output mismatches the expected output, the test framework will dump the actual output in the file +`*.check.out` and fail the test suite. It will also output the instructions to quickly replace the expected output +with the actual output, in the following format: + +``` +Test output dumped in: tests/neg/Sample.check.out + See diff of the checkfile + > diff tests/neg/Sample.check tests/neg/Sample.check.out + Replace checkfile with current output + > mv tests/neg/Sample.check.out tests/neg/Sample.check +``` + +### Tips for creating Checkfiles + +To create a checkfile for a test, you can do one of the following: + +1. Create an empty checkfile + - then add arbitrary content + - run the test + - when it fails, use the `mv` command reported by the test to replace the initial checkfile with the actual output. +2. Manually compile the file you are testing with `scala3/scalac` + - copy-paste whatever console output the compiler produces to the checkfile. + +### Automatically Updating Checkfiles + +When complex or many checkfiles must be updated, `testCompilation` can run in a mode where it overrides the +checkfiles with the test outputs. +```bash +$ sbt +> testCompilation --update-checkfiles +``` + +Use `--help` to see all the options +```bash +$ sbt +> testCompilation --help +``` + +### Bootstrapped-only tests + +To run `testCompilation` on a bootstrapped Dotty compiler, use +`scala3-compiler-bootstrapped/testCompilation` (with the same syntax as above). +Some tests can only be run in bootstrapped compilers; that includes all tests +with `with-compiler` in their name. + +### From TASTy tests + +`testCompilation` has an additional mode to run tests that compile code from a `.tasty` file. +Modify the lists in [compiler/test/dotc] to enable or disable tests from `.tasty` files. + +```bash +$ sbt +> testCompilation --from-tasty +``` + +## Unit Tests + +Unit tests cover the other areas of the compiler, such as interactions with the REPL, scripting tools and more. +They are defined in [compiler/test], so if your use case isn't covered by this guide, +you may need to consult the codebase. Some common areas are highlighted below: + +### SemanticDB tests + +To test the SemanticDB output from the `extractSemanticDB` phase (enabled with the `-Xsemanticdb` flag), run the following sbt command: +```bash +$ sbt +sbt:scala3> scala3-compiler-bootstrapped/testOnly + dotty.tools.dotc.semanticdb.SemanticdbTests +``` + +[SemanticdbTests] uses source files in `tests/semanticdb/expect` to generate "expect files": +these verify both +- SemanticDB symbol occurrences inline in sourcecode (`*.expect.scala`) +- complete output of all SemanticDB information (`metac.expect`). + +Expect files are used as regression tests to detect changes in the compiler. +Their correctness is determined by human inspection. + +If expect files change then [SemanticdbTests] will fail, and generate new expect files, providing instructions for +comparing the differences and replacing the outdated expect files. + +If you are planning to update the SemanticDB output, you can do it in bulk by running the command +```bash +$ sbt +sbt:scala3> scala3-compiler/Test/runMain + dotty.tools.dotc.semanticdb.updateExpect +``` + +then compare the changes via version control. + +## Troubleshooting + +Some of the tests depend on temporary state stored in the `out` directory. In rare cases, that directory +can enter an inconsistent state and cause spurious test failures. If you suspect a spurious test failure, +you can run `rm -rf out/*` from the root of the repository and run your tests again. If that fails, you +can try `git clean -xfd`. + +[CompilationTests]: https://github.com/lampepfl/dotty/blob/master/compiler/test/dotty/tools/dotc/CompilationTests.scala +[compiler/test]: https://github.com/lampepfl/dotty/blob/master/compiler/test/ +[compiler/test/dotc]: https://github.com/lampepfl/dotty/tree/master/compiler/test/dotc +[SemanticdbTests]: https://github.com/lampepfl/dotty/blob/master/compiler/test/dotty/tools/dotc/semanticdb/SemanticdbTests.scala diff --git a/docs/_docs/contributing/procedures/index.md b/docs/_docs/contributing/procedures/index.md index 01c76f72c00c..db2b09dbe80f 100644 --- a/docs/_docs/contributing/procedures/index.md +++ b/docs/_docs/contributing/procedures/index.md @@ -2,3 +2,7 @@ layout: index title: Procedures --- + +This chapter of the guide describes: +- [How to release a procedure](./release.md) +- [How to test the vulpix framework](./vulpix.md) \ No newline at end of file diff --git a/docs/_docs/contributing/procedures/vulpix.md b/docs/_docs/contributing/procedures/vulpix.md index 5e8a2eab425b..1eea2fa24778 100644 --- a/docs/_docs/contributing/procedures/vulpix.md +++ b/docs/_docs/contributing/procedures/vulpix.md @@ -3,7 +3,6 @@ layout: doc-page title: Test Vulpix Framework --- -# Test Vulpix Framework If you are modifying the Vulpix framework and need a playground with dummy tests to try out your modifications, do the following. Create the directory structure for the playground: diff --git a/docs/_docs/contributing/testing.md b/docs/_docs/contributing/testing.md deleted file mode 100644 index a01cdb08f8ab..000000000000 --- a/docs/_docs/contributing/testing.md +++ /dev/null @@ -1,207 +0,0 @@ ---- -layout: doc-page -title: Testing in Dotty ---- - -Running all tests in Dotty is as simple as: - -```bash -$ sbt test -``` - -Specifically, `sbt test` runs all tests that do _not_ require a bootstrapped -compiler. In practice, this means that it runs all compilation tests meeting -this criterion, as well as all non-compiler tests. - -The entire suite of tests can be run using the bootstrapped compiler as follows: - -```bash -$ sbt -> scala3-bootstrapped/test -``` - -There are currently several forms of tests in Dotty. These can be split into -two categories: - -## Unit tests -These tests can be found in `/test` and are used to check -functionality of specific parts of the codebase in isolation e.g: parsing, -scanning and message errors. - -To run all tests in e.g., for the compiler test-suite you can write: - -```bash -$ sbt -> scala3-compiler/test -``` - -To run a single test class you use `testOnly` and the fully qualified class name. -For example: - -```bash -> testOnly dotty.tools.dotc.transform.TreeTransformerTest -``` - -The test command follows a regular expression-based syntax `testOnly * -- *`. -The right-hand side picks a range of names for methods and the left-hand side picks a range of class names and their -fully-qualified paths. - -Consequently, you can restrict the aforementioned executed test to a subset of methods by appending ``-- *method_name``. -The example below picks up all methods with the name `canOverwrite`: - -```bash -> testOnly dotty.tools.dotc.transform.TreeTransformerTest -- *canOverwrite -``` - -Additionally, you can run all tests named `method_name`, in any class, without providing a class name: - -```bash -> testOnly -- *canOverwrite -``` - -You can also run all paths of classes of a certain name: - -```bash -> testOnly *.TreeTransformerTest -``` - -### Testing with checkfiles -Some tests support checking the output of the run or the compilation against a checkfile. A checkfile is a file in which the expected output of the compilation or run is defined. A test against a checkfile fails if the actual output mismatches the expected output. - -Currently, the `run` and `neg` (compilation must fail for the test to succeed) tests support the checkfiles. `run`'s checkfiles contain an expected run output of the successfully compiled program. `neg`'s checkfiles contain an expected error output during compilation. - -Absence of a checkfile is **not** a condition for the test failure. E.g. if a `neg` test fails with the expected number of errors and there is no checkfile for it, the test still passes. - -Checkfiles are located in the same directories as the tests they check, have the same name as these tests with the extension `*.check`. E.g. if you have a test named `tests/neg/foo.scala`, you can create a checkfile for it named `tests/neg/foo.check`. And if you have a test composed of several files in a single directory, e.g. `tests/neg/manyScalaFiles`, the checkfile will be `tests/neg/manyScalaFiles.check`. - -If the actual output mismatches the expected output, the test framework will dump the actual output in the file `*.check.out` and fail the test suite. It will also output the instructions to quickly replace the expected output with the actual output, in the following format: - -``` -Test output dumped in: tests/playground/neg/Sample.check.out - See diff of the checkfile - > diff tests/playground/neg/Sample.check tests/playground/neg/Sample.check.out - Replace checkfile with current output - > mv tests/playground/neg/Sample.check.out tests/playground/neg/Sample.check -``` - -To create a checkfile for a test, you can do one of the following: - -- Create a dummy checkfile with a random content, run the test, and, when it fails, use the `mv` command reported by the test to replace the dummy checkfile with the actual output. -- Manually compile the file you are testing with `scalac` and copy-paste whatever console output the compiler produces to the checkfile. - -## Integration tests -These tests are Scala source files expected to compile with Dotty (pos tests), -along with their expected output (run tests) or errors (neg tests). - -All of these tests are contained in the `./tests/*` directories and can be run with the `testCompilation` command. Tests in folders named `with-compiler` are an exception, see next section. - -Currently to run these tests you need to invoke from sbt: - -```bash -$ sbt -> testCompilation -``` - -(which is effectively the same with `testOnly dotty.tools.dotc.CompilationTests`) - -It is also possible to run tests filtered, again from sbt: - -```bash -$ sbt -> testCompilation companions -``` - -This will run both the test `./tests/pos/companions.scala` and -`./tests/neg/companions.scala` since both of these match the given string. -This also means that you could run `testCompilation` with no arguments to run all integration tests. - -When complex checkfiles must be updated, `testCompilation` can run in a mode where it overrides the checkfiles with the test outputs. -```bash -$ sbt -> testCompilation --update-checkfiles -``` - -Use `--help` to see all the options -```bash -$ sbt -> testCompilation --help -``` - -### Joint and separate sources compilation - -When the sources of a test consist of multiple source files places in a single directory they are passed to the compiler in a single run and the compiler decides in which order to compile them. In some cases, however, to reproduce a specific test scenario it might be necessary to compile the source files in several steps in a specified order. To achieve that one can add a `_${step_index}` suffix to a file name (before the `.scala` or `.java` extension) indicating the order of compilation. E.g. if the test directory contains files named `Foo_1.scala`, `Bar_2.scala` and `Baz_2.scala` then `Foo_1.scala` will be compiled first and after that `Bar_2.scala` together with `Baz_2.scala`. - -The other kind of suffix that can modify how particular files are compiled is `_c${compilerVersion}`. When specified, the file will be compiled with a specific version of the compiler instead of the one developed on the current branch. - -Different suffixes can be mixed together (their order is not important although consistency is advised), e.g. `Foo_1_c3.0.2`, `Bar_2_c3.1.0`. - -### Bootstrapped-only tests - -To run `testCompilation` on a bootstrapped Dotty compiler, use -`scala3-compiler-bootstrapped/testCompilation` (with the same syntax as above). -Some tests can only be run in bootstrapped compilers; that includes all tests -with `with-compiler` in their name. - -### From TASTy tests - -`testCompilation` has an additional mode to run tests that compile code from a `.tasty` file. - Modify blacklist and whitelists in `compiler/test/dotc` to enable or disable tests from `.tasty` files. - - ```bash - $ sbt - > testCompilation --from-tasty - ``` - - This mode can be run under `scala3-compiler-bootstrapped/testCompilation` to test on a bootstrapped Dotty compiler. - -### SemanticDB tests - -```bash -$ sbt -> scala3-compiler-bootstrapped/testOnly dotty.tools.dotc.semanticdb.SemanticdbTests -``` - -The output of the `extractSemanticDB` phase, enabled with `-Xsemanticdb` is tested with the bootstrapped JUnit test -`dotty.tools.dotc.semanticdb.SemanticdbTests`. It uses source files in `tests/semanticdb/expect` to generate -two kinds of output file that are compared with "expect files": placement of semanticdb symbol occurrences inline in -sourcecode (`*.expect.scala`), for human verification by inspection; and secondly metap formatted output which outputs -all information stored in semanticdb (`metac.expect`). -Expect files are used as regression tests to detect changes in the compiler. - -The test suite will create a new file if it detects any difference, which can be compared with the -original expect file, or if the user wants to globally replace all expect files for semanticdb they can use -`scala3-compiler-bootstrapped/test:runMain dotty.tools.dotc.semanticdb.updateExpect`, and compare the changes via version -control. - -### Test regimes - -Continuous integration, managed by GitHub Actions, does not run all jobs when a pull request is created. -In particular, test jobs for testing under JDK 8 and Windows are not run. Those jobs are run only for the nightly build. - -If a PR may fail differentially under either JDK 8 or Windows, the test jobs may be triggered by adding -a special command to the PR comment text: - -``` -[test_java8] -[test_windows_full] -``` -Furthermore, CI tests are bootstrapped. A job to also run tests non-bootstrapped may be triggered manually: -``` -[test_non_bootstrapped] -``` -A trivial PR, such as a fix for a typo in a comment or when contributing other documentation, may benefit by skipping CI tests altogether: -``` -[skip ci] -``` -Other jobs which are normally run can also be selectively skipped: -``` -[skip community_build] -[skip test_windows_fast] -``` - -## Troubleshooting - -Some of the tests depend on temporary state stored in the `out` directory. In rare cases, that directory -can enter an inconsistent state and cause spurious test failures. If you suspect a spurious test failure, -you can run `rm -rf out/*` from the root of the repository and run your tests again. If that fails, you -can try `git clean -xfd`. diff --git a/docs/_docs/contributing/tools/index.md b/docs/_docs/contributing/tools/index.md index 92503ee82013..e784e3e15d61 100644 --- a/docs/_docs/contributing/tools/index.md +++ b/docs/_docs/contributing/tools/index.md @@ -2,3 +2,8 @@ layout: index title: IDEs and Tools --- + +This chapter of the guide describes how to use Dotty with IDEs and other tools: +- [IDEs](./ide.md) +- [Use Mill](./mill.md) +- [Use Scalafix](./scalafix.md) diff --git a/docs/_docs/contributing/tools/scalafix.md b/docs/_docs/contributing/tools/scalafix.md index 58c7d0eb7b3a..30c7050f8b3e 100644 --- a/docs/_docs/contributing/tools/scalafix.md +++ b/docs/_docs/contributing/tools/scalafix.md @@ -3,8 +3,6 @@ layout: doc-page title: Working with Scalafix --- -# Working with Scalafix - First, create a new rule as follows (command from https://scalacenter.github.io/scalafix/docs/developers/setup.html): ```bash diff --git a/docs/_docs/contributing/workflow.md b/docs/_docs/contributing/workflow.md index 956ce2998c75..36d91bf02707 100644 --- a/docs/_docs/contributing/workflow.md +++ b/docs/_docs/contributing/workflow.md @@ -103,8 +103,27 @@ The basics of working with Dotty codebase are documented [here](https://dotty.ep | Command | Description | |------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------| +| `scala3/scalac` | Run the compiler directly, with any current changes. | +| `scala3/scala` | Run the main method of a given class name. | | `scalac ../issues/Playground.scala` | Compile the given file – path relative to the Dotty directory. Output the compiled class files to the Dotty directory itself. | | `scala Playground` | Run the compiled class `Playground`. Dotty directory is on classpath by default. | | `repl` | Start REPL | +| `scala3/scalac -print-tasty Foo.tasty` | Print the TASTy of top-level class `Foo` | +| `scala3-bootstrapped/test` | Run all tests for Scala 3. (Slow, recommended for CI only) | +| `scala3-bootstrapped/publishLocal` | Build Scala 3 locally. (Use to debug a specific project) | +| `scalac ../issues/Playground.scala` | Compile the given file – path relative to the Dotty directory. Output the compiled class files to the Dotty directory itself.| | `testOnly dotty.tools.dotc.CompilationTests -- *pos` | Run test (method) `pos` from `CompilationTests` suite. | | `testCompilation sample` | In all test suites, run test files containing the word `sample` in their title. | +| `scala3-compiler/Test/runMain dotty.tools.printTypes`| Print types underlying representation | + + +## Shell Commands + +| Command | Description | +|--------------------------------------|------------------------------------------------------------------| +| `rm -rv *.tasty *.class out || true` | clean all compiled artifacts, from root dotty directory | + + + + + diff --git a/docs/sidebar.yml b/docs/sidebar.yml index 1e791472bceb..d6fd65ab9475 100644 --- a/docs/sidebar.yml +++ b/docs/sidebar.yml @@ -166,11 +166,18 @@ subsection: directory: docs/contributing index: contributing/index.md subsection: - - page: contributing/contribute-knowledge.md - page: contributing/getting-started.md - - page: contributing/workflow.md - - page: contributing/testing.md - - page: contributing/debugging.md + - index: contributing/workflow.md + subsection: + - page: contributing/issues/reproduce.md + - page: contributing/issues/cause.md + - page: contributing/issues/areas.md + - page: contributing/issues/debugging.md + - page: contributing/issues/other-debugging.md + - page: contributing/issues/inspection.md + - page: contributing/issues/efficiency.md + - page: contributing/issues/testing.md + - page: contributing/issues/checklist.md - title: IDEs and Tools directory: tools index: contributing/tools/index.md @@ -179,10 +186,21 @@ subsection: - page: contributing/tools/mill.md - page: contributing/tools/scalafix.md - title: Procedures + directory: procedures index: contributing/procedures/index.md subsection: - page: contributing/procedures/release.md - page: contributing/procedures/vulpix.md + - title: High Level Architecture + directory: architecture + index: contributing/architecture/index.md + subsection: + - page: contributing/architecture/lifecycle.md + - page: contributing/architecture/context.md + - page: contributing/architecture/phases.md + - page: contributing/architecture/types.md + - page: contributing/architecture/time.md + - page: contributing/architecture/symbols.md - title: Internals directory: docs/internals index: internals/index.md