Merge pull request #2153 from dotty-staging/topic/dottest-userman

Add documentation for new parallel testing suite

Author: felixmulder

doc-tool/resources/css/dottydoc.css

@@ -334,3 +334,23 @@ blockquote {
     color: #777;
     border-left: 0.25em solid #ddd;
 }
+
+aside {
+    padding: 15px;
+    margin: 10px 0;
+}
+
+aside.warning {
+    border-left: 3px solid #d62c2c;
+    background-color: #ffe4e4;
+}
+
+aside.notice {
+    border-left: 3px solid #4c97e4;
+    background-color: #e4ebff;
+}
+
+aside.success {
+    border-left: 3px solid #36bf1d;
+    background-color: #ebfddd;
+}

docs/docs/contributing/testing.md

@@ -0,0 +1,89 @@
+---
+layout: doc-page
+title: Testing in Dotty
+---
+
+<aside class="warning">
+This page should be updated as soon as scala-partest is removed
+</aside>
+
+Running all tests in Dotty is as simple as:
+
+```bash
+$ sbt test
+```
+
+There are currently several forms of tests in Dotty. These can be split into
+two categories:
+
+## Unit tests
+These tests can be found in `<sub-project>/test` and are used to check
+functionality of specific parts of the codebase in isolation e.g: parsing,
+scanning and message errors.
+
+Running a single unit test class from sbt is as simple as:
+
+```bash
+> testOnly absolute.path.to.TestClass
+```
+
+You can further restrict the executed tests to a subset of `TestClass` methods
+as follows:
+
+```bash
+> testOnly absolute.path.to.TestClass -- *methodName
+```
+
+## Integration tests
+These tests are Scala source files expected to compile with Dotty (pos tests),
+along with their expected output (run tests) or errors (neg tests).
+
+All of these tests are contained in the `./tests/*` directories.
+
+## scala-partest
+Historically these tests needed a structure which was generated by running the
+unit tests, and then that structure was in turn used by
+[scala-partest](http://github.com/scala/scala-partest) to run compilation tests
+in parallel.
+
+This test suite can still be used (and is currently a part of the CI to check
+that it has the same outcome as the new test suite). It is invoked from sbt by
+running one of the following commands:
+
+```bash
+> partest-only-no-bootstrap
+> partest-only
+> partest
+```
+
+- `partest-only-no-bootstrap` will only run the integration tests
+- `partest-only` will bootstrap the compiler and run the integration tests
+- `partest` will bootstrap the compiler, run the unit tests and then the
+  integration tests
+
+## dotty parallel test suite
+The new test suite will soon become the standard integration test runner. It
+has several advantages over the old implementation:
+
+- integrates with JUnit, without the need for setup
+- reuses the same VM for compilation
+- allows filtering of tests
+- runs much faster (almost 2x)
+
+Currently to run these tests you need to invoke from sbt:
+
+```bash
+> testOnly dotty.tools.dotc.CompilationTests
+```
+
+This might be aliased in the future. It is also possible to run tests filtered
+by using:
+
+```bash
+> filterTest .*i2147.scala
+```
+
+This will run both the test `./tests/pos/i2147.scala` and
+`./tests/partest-test/i2147.scala` since both of these match the given regular
+expression. This also means that you could run `filterTest .*` to run all
+integration tests.

docs/docs/contributing/workflow.md

@@ -57,11 +57,21 @@ $ sbt
 To test a specific test tests/x/y.scala (for example tests/pos/t210.scala):
 
 ```bash
-> partest-only-no-bootstrap --show-diff --verbose tests/partest-generated/x/y.scala
+> filterTest .*pos/t210.scala
 ```
 
-Currently this will re-run some unit tests and do some preprocessing because of
-the way partest has been set up.
+The filterTest task takes a regular expression as its argument. For example,
+you could run a negative and a positive test with:
+
+```bash
+> filterTest (.*pos/t697.scala)|(.*neg/i2101.scala)
+```
+
+or if they have the same name, the equivalent can be achieved with:
+
+```bash
+> filterTest .*/i2101.scala
+```
 
 ## Inspecting Trees with Type Stealer ##
 

docs/docs/internals/higher-kinded-v2.md

@@ -3,10 +3,11 @@ layout: doc-page
 title: "Higher-Kinded Types in Dotty"
 ---
 
-**This page is out of date and preserved for posterity. Please see [Implementing
-Higher-Kinded Types in
-Dotty](http://guillaume.martres.me/publications/dotty-hk.pdf) for a more up to
-date version**
+<aside class="warning">
+    This page is out of date and preserved for posterity. Please see
+    <a href="http://guillaume.martres.me/publications/dotty-hk.pdf">
+    Implementing Higher-Kinded Types in Dotty</a> for a more up to date version
+</aside>
 
 Higher-Kinded Types in Dotty V2
 ===============================

docs/sidebar.yml

@@ -23,6 +23,8 @@ sidebar:
           url: docs/contributing/intellij-idea.html
         - title: Workflow
           url: docs/contributing/workflow.html
+        - title: Testing
+          url: docs/contributing/testing.html
     - title: Internals
       subsection:
         - title: Backend