Skip to content

making scoverage work with classpaths #267

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 8 commits into from
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,10 @@ lazy val runtime = CrossProject("scalac-scoverage-runtime", file("scalac-scovera
.settings(appSettings: _*)
.jvmSettings(
fork in Test := true,
libraryDependencies += "org.scalatest" %% "scalatest" % ScalatestVersion % "test"
envVars in Test := Map("SCOVERAGE_MEASUREMENT_PATH" -> "scoverageMeasurementFiles"),
libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % ScalatestVersion % "test"
)
)
.jsSettings(
libraryDependencies += "org.scalatest" %%% "scalatest" % ScalatestVersion % "test",
Expand Down
89 changes: 72 additions & 17 deletions scalac-scoverage-plugin/src/main/scala/scoverage/plugin.scala
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ import scala.tools.nsc.transform.{Transform, TypingTransformers}

/** @author Stephen Samuel */
class ScoveragePlugin(val global: Global) extends Plugin {

override val name: String = "scoverage"
override val description: String = "scoverage code coverage compiler plugin"
private val (extraAfterPhase, extraBeforePhase) = processPhaseOptions(pluginOptions)
Expand All @@ -35,16 +34,23 @@ class ScoveragePlugin(val global: Global) extends Plugin {
options.dataDir = opt.substring("dataDir:".length)
} else if (opt.startsWith("extraAfterPhase:") || opt.startsWith("extraBeforePhase:")) {
// skip here, these flags are processed elsewhere
} else {
} else if (opt.startsWith("useEnvironment:")) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

userEnvironment should be renamed to something more meaningful related to classpath --use-classpath-dir.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This option should also be clear that it'll override dataDir.

options.useEnvironment = opt.substring("useEnvironment:".length).toBoolean
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have some error checking here? .toBoolean could potentially throw an error here.

} else{
error("Unknown option: " + opt)
}
}
if (!opts.exists(_.startsWith("dataDir:")))
throw new RuntimeException("Cannot invoke plugin without specifying <dataDir>")

if (opts.exists{p: String => p.contains("useEnvironment") && p.contains("false")} || (!opts.exists(_.startsWith("useEnvironment")))) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this meant to replace dataDir in the future?

Copy link
Author

@sammy-1234 sammy-1234 Jul 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this is just a check that if the option to write to the classpath is either set to false p.contains("useEnvironment") && p.contains("false") or is not present (!opts.exists(_.startsWith("useEnvironment")), then make sure that dataDir attribute is given by the user.

Overriding dataDir takes place in def setOptions(options: ScoverageOptions): Unit

if (!opts.exists(_.startsWith("dataDir:")))
throw new RuntimeException("Cannot invoke plugin without specifying <dataDir>")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or without specifying new option?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is being checked by !opts.exists(_.startsWith("useEnvironment"))

}
instrumentationComponent.setOptions(options)
}

override val optionsHelp: Option[String] = Some(Seq(
"-P:scoverage:useEnvironment:<boolean> if true, use the environment variable to store measurements" +
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we include some description for overrides dataDir?

"and store instruments to the output classpath where compiler emits classfiles.",
"-P:scoverage:dataDir:<pathtodatadir> where the coverage files should be written\n",
"-P:scoverage:excludedPackages:<regex>;<regex> semicolon separated list of regexs for packages to exclude",
"-P:scoverage:excludedFiles:<regex>;<regex> semicolon separated list of regexs for paths to exclude",
Expand Down Expand Up @@ -82,6 +88,13 @@ class ScoverageOptions {
var excludedFiles: Seq[String] = Nil
var excludedSymbols: Seq[String] = Seq("scala.reflect.api.Exprs.Expr", "scala.reflect.api.Trees.Tree", "scala.reflect.macros.Universe.Tree")
var dataDir: String = IOUtils.getTempPath

// Adding additional option for https://github.com/scoverage/scalac-scoverage-plugin/issues/265
// If the option is false, scoverage works the usual way.
// If the option is true, the instruments are stored on to the classpath and
// the measurements are written to the directory specified by the environment variable
// `SCOVERAGE_MEASUREMENT_PATH`. By default, the option is set to false.
var useEnvironment: Boolean = false
}

class ScoverageInstrumentationComponent(val global: Global, extraAfterPhase: Option[String], extraBeforePhase: Option[String])
Expand All @@ -108,9 +121,20 @@ class ScoverageInstrumentationComponent(val global: Global, extraAfterPhase: Opt
private var coverageFilter: CoverageFilter = AllCoverageFilter

def setOptions(options: ScoverageOptions): Unit = {

this.options = options

// If useEnvironment is true, we set the output directory to output classpath,
// i.e to the directory where compiler is going to output the classfiles.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may not be unique, unfortunately. Even if you include the absolute path to the output directory, two compiles happening on different machines might use the same output directory, and not be identical.

I don't have any great ideas here. Hashing the source files themselves relative to the current working directory might work? Using a random number or UUID would make builds non-deterministic.

Maybe this information does need to be user provided, and you could require that the dataDir is a relative path that you'd use in this position?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. I have currently set the dataDir to be the targetID in case the writeToClasspath option is set to true.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should update this comment, and below.

if(options.useEnvironment){
settings.outputDirs.getSingleOutput match {
case Some(dest) => options.dataDir = dest.toString() + "/META-INF/scoverage"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make META-INF/scoverage a configurable override option that could be passed in?

case None => throw new RuntimeException("No output classpath specified.")
}
}
coverageFilter = new RegexCoverageFilter(options.excludedPackages, options.excludedFiles, options.excludedSymbols)
new File(options.dataDir).mkdirs() // ensure data directory is created

}

override def newPhase(prev: scala.tools.nsc.Phase): Phase = new Phase(prev) {
Expand All @@ -128,7 +152,12 @@ class ScoverageInstrumentationComponent(val global: Global, extraAfterPhase: Opt

Serializer.serialize(coverage, Serializer.coverageFile(options.dataDir))
reporter.echo(s"Wrote instrumentation file [${Serializer.coverageFile(options.dataDir)}]")
reporter.echo(s"Will write measurement data to [${options.dataDir}]")

// Measurements are not necessarily written to
// [options.dataDir] in case [useEnvironment] is set to true.
if(!options.useEnvironment) {
reporter.echo(s"Will write measurement data to [${ options.dataDir }]")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should there be a helpful hint to use the new option instead?

}
}
}

Expand All @@ -153,23 +182,48 @@ class ScoverageInstrumentationComponent(val global: Global, extraAfterPhase: Opt
def safeSource(tree: Tree): Option[SourceFile] = if (tree.pos.isDefined) Some(tree.pos.source) else None

def invokeCall(id: Int): Tree = {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not familiar with this, hoping someone with more knowledge can take a look at this one.

Apply(
Select(
/**
* We still pass in [options.dataDir] as one of the instruments with
* [invokedUseEnvironment] as it helps differentiating the source
* files at runtime, i.e helps in creating a unique subdir for each source file.
*/
if (options.useEnvironment){
Apply(
Select(
Ident("scoverage"),
newTermName("Invoker")
Select(
Ident("scoverage"),
newTermName("Invoker")
),
newTermName("invokedUseEnvironment")
),
newTermName("invoked")
),
List(
Literal(
Constant(id)
List(
Literal(
Constant(id)
),
Literal(
Constant(options.dataDir)
)
)
)
}else {
Apply(
Select(
Select(
Ident("scoverage"),
newTermName("Invoker")
),
newTermName("invoked")
),
Literal(
Constant(options.dataDir)
List(
Literal(
Constant(id)
),
Literal(
Constant(options.dataDir)
)
)
)
)
}
}

override def transform(tree: Tree) = process(tree)
Expand Down Expand Up @@ -288,6 +342,7 @@ class ScoverageInstrumentationComponent(val global: Global, extraAfterPhase: Opt
def allConstArgs(args: List[Tree]) = args.forall(arg => arg.isInstanceOf[Literal] || arg.isInstanceOf[Ident])

def process(tree: Tree): Tree = {

tree match {

// // non ranged inside ranged will break validation after typer, which only kicks in for yrangepos.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
package scoverage

import java.nio.file.{Files, Paths}
import org.scalatest.{BeforeAndAfter, FunSuite}
import scoverage.Platform.File

/**
* Verify that [[Invoker.invoked()]] can handle a multi-module project
*/
class InvokerUseEnvironmentTest extends FunSuite with BeforeAndAfter {


val instrumentsDir = Array(
new File("target/invoker-test.instrument0"),
new File("target/invoker-test.instrument1")
)

before {
deleteInstrumentFiles()
instrumentsDir.foreach(_.mkdirs())
createNewInstruments()
}

test("calling Invoker.invokedUseEnvironments puts the data on env var SCOVERAGE_MEASUREMENT_PATH" +
" and copies coverage files from instruments directory.") {

val testIds: Set[Int] = (1 to 10).toSet

testIds.map { i: Int => Invoker.invokedUseEnvironment(i, instrumentsDir(i % 2).toString)}

// Verify measurements went to correct directory under the environment variable.
val dir0 = s"${System.getenv("SCOVERAGE_MEASUREMENT_PATH")}/${Invoker.md5HashString(instrumentsDir(0).toString)}"
val measurementFiles3 = Invoker.findMeasurementFiles(dir0)
val idsFromFile3 = Invoker.invoked(measurementFiles3.toIndexedSeq)
idsFromFile3 === testIds.filter { i: Int => i % 2 == 0}


val dir1 = s"${System.getenv("SCOVERAGE_MEASUREMENT_PATH")}/${Invoker.md5HashString(instrumentsDir(1).toString)}"
val measurementFiles4 = Invoker.findMeasurementFiles(dir1)
val idsFromFile4 = Invoker.invoked(measurementFiles4.toIndexedSeq)
idsFromFile3 === testIds.filter { i: Int => i % 2 == 1}

// Verify that coverage files have been copied correctly.
assert(Files.exists(Paths.get(s"$dir0/scoverage.coverage")))
assert(Files.exists(Paths.get(s"$dir1/scoverage.coverage")))
}

after {
deleteInstrumentFiles()
instrumentsDir.foreach(_.delete())
deleteMeasurementFolders()
}

private def deleteInstrumentFiles(): Unit = {
instrumentsDir.foreach((md) => {
if (md.isDirectory)
md.listFiles().foreach(_.delete())
})
}

private def deleteMeasurementFolders(): Unit = {
val d = s"${System.getenv("SCOVERAGE_MEASUREMENT_PATH")}"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test should not be expect this env var to already be set.

What I'd recommend instead is changing the code in the invoker to consume a Java "system property", rather than an env var, and to then set the property during the test. System properties can be changed at runtime (see "Writing system properties" here: https://docs.oracle.com/javase/tutorial/essential/environment/sysprop.html).

This would avoid setting it in the build.sbt file, which affects all tests: not just this one.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

...but before doing this, see the comment below about how the dataDir is initialized. That change would also allow for setting the dataDir in tests.

instrumentsDir.foreach (i => {
val f = new File(s"$d/${Invoker.md5HashString(i.toString)}")
if (f.isDirectory)
f.listFiles().foreach(_.delete())
})

val f2 = new File(d)
if(f2.isDirectory)
f2.listFiles().foreach(_.delete())
f2.delete()
}

private def createNewInstruments(): Unit = {
new File("target/invoker-test.instrument0/scoverage.coverage").createNewFile()
new File("target/invoker-test.instrument1/scoverage.coverage").createNewFile()
}

}
Loading