Make SBT based benchmarking more flexible

  - Allow injection of extra settings in to the build
  - Parameterize the command name used to compile.
  - Create a custom "compileRaw" task in the build (for SBT 0.13.x, the
    API I'm using must have shifted in 1.0.x)

Testing a local build of SBT that disables its metadata collection compiler phases
when `antStyle = true`.

```
sbt> set scalaVersion in compilation := "2.12.3-bin-d1ec01a-SNAPSHOT"

sbt> compilation/jmh:run HotSbtBenchmark -psource=scalap -psbtVersion=0.13.16-bin-7b5d818 -pextraBuild=incOptions:=incOptions.value.withNameHashing(false).withAntStyle(true)
...
[info] Benchmark                                (compileCmd)  (corpusVersion)  (extraArgs)  (extraBuild)  (sbtVersion)  (source)    Mode  Cnt     Score   Error  Units
[info] HotSbtBenchmark.compile                          a8c43dc               incOptions:=incOptions.value.withNameHashing(false).withAntStyle(true)  0.13.16-bin-7b5d818    scalap  sample  330   956.082 ± 7.116  ms/op
```

Bypassing the `compile` task altogether in favour of user of SBT's `RawCompiler`.

```
sbt> compilation/jmh:run HotSbtBenchmark -psource=scalap -psbtVersion=0.13.15 -pcompileCmd=compileRaw

[info] Benchmark                                (compileCmd)  (corpusVersion)  (extraArgs)  (extraBuild)  (sbtVersion)  (source)    Mode  Cnt     Score   Error  Units
[info] HotSbtBenchmark.compile                    compileRaw          a8c43dc                                  0.13.15    scalap  sample  357   888.775 ± 7.243  ms/op
```

The base and top-lines that we're comparing against:

  - `HotScalacBenchmark` for the same Scala revision: 765ms.
  - `HotSbtBenchmark` using default config and compile task from 0.13.15: 1063ms

Discussion:

  - The overhead of the terminal scraping between the JMH benchmark process and the child SBT process might be a factor here. Try a larger codebases to reduce that effect.
  - The SBT compiler phases appear to levy only ~12% overhead, which seems quite reasonable.
  - We need to find out what other factors are preventing the results above from coming in closer to the direct compiler usage.

Author: retronym

compilation/src/main/scala/scala/tools/nsc/HotSbtBenchmark.scala

@@ -28,6 +28,12 @@ class HotSbtBenchmark {
   @Param(value = Array("0.13.15"))
   var sbtVersion: String = _
 
+  @Param(value = Array(""))
+  var extraBuild: String = _
+
+  @Param(value= Array("compile"))
+  var compileCmd: String = _
+
   // This parameter is set by ScalacBenchmarkRunner / UploadingRunner based on the Scala version.
   // When running the benchmark directly the "latest" symlink is used.
   @Param(value = Array("latest"))
@@ -42,6 +48,19 @@ class HotSbtBenchmark {
   var processInputReader: BufferedWriter = _
   var output= new java.lang.StringBuilder()
 
+  def compileRawSetting = if (sbtVersion.startsWith("1.")) "" else
+    """
+      |val compileRaw = taskKey[Unit]("Compile directly, bypassing the incremental compiler")
+      |
+      |def addCompileRaw = compileRaw := {
+      |    val compiler = new sbt.compiler.RawCompiler(scalaInstance.value, sbt.ClasspathOptions.auto, streams.value.log)
+      |    classDirectory.value.mkdirs()
+      |    compiler.apply(sources.value, classDirectory.value +: dependencyClasspath.value.map(_.data), classDirectory.value, scalacOptions.value)
+      |  }
+      |
+      |inConfig(Compile)(List(addCompileRaw))
+    """.stripMargin
+
   def buildDef =
     s"""
        |scalaHome := Some(file("${scalaHome.toAbsolutePath.toString}"))
@@ -55,6 +74,12 @@ class HotSbtBenchmark {
        |libraryDependencies += "org.scala-lang" % "scala-compiler" % scalaVersion.value
        |libraryDependencies += "org.scala-lang" % "scala-reflect" % scalaVersion.value
        |
+       |traceLevel := Int.MaxValue
+       |traceLevel in runMain := Int.MaxValue
+       |
+       |$compileRawSetting
+       |
+       |$extraBuild
        |// TODO support .java sources
     """.stripMargin
 
@@ -79,7 +104,7 @@ class HotSbtBenchmark {
 
   @Benchmark
   def compile(): Unit = {
-    issue(";cleanClasses;compile")
+    issue(s";cleanClasses;$compileCmd")
     awaitPrompt()
   }