ClassfileManager complains that a file does not exist

[rkuhn]

Using the lampepfl/dotty.g8 template with the following in Main.scala:

object Main {

  trait Op[S, +T] {
    def flatMap[U](f: T => Op[S, U]): Op[S, U]
  }

  sealed trait X[Y] extends Any {
    type S = Y
  }

  object X {
    private val _unit: Op[Nothing, Null] = ???
    private def unit[S, T]: Op[S, T] = _unit.asInstanceOf[Op[S, T]]

    def apply[T]: Next[T] = next.asInstanceOf[Next[T]]
    trait Next[T] {
      def apply[U](body: X[T] => Op[T, U]): Op[T, U] =
        unit[T, X[T]].flatMap(body)
    }
    private object next extends Next[Nothing]
  }

}

Running sbt shell and after a few compile runs while editing the following happens:

[debug] [naha] 
[debug] [naha] Initial source changes: 
[debug] [naha] 	removed:Set()
[debug] [naha] 	added: Set()
[debug] [naha] 	modified: Set(/Users/rkuhn/comp/test/dotty/dotty-project-template/src/main/scala/Main.scala)
[debug] [naha] Invalidated products: Set()
[debug] [naha] External API changes: API Changes: Set()
[debug] [naha] Modified binary dependencies: Set()
[debug] [naha] Initial directly invalidated sources: Set(/Users/rkuhn/comp/test/dotty/dotty-project-template/src/main/scala/Main.scala)
[debug] [naha] 
[debug] [naha] Sources indirectly invalidated by:
[debug] [naha] 	product: Set()
[debug] [naha] 	binary dep: Set()
[debug] [naha] 	external source: Set()
[debug] All initially invalidated sources: Set(/Users/rkuhn/comp/test/dotty/dotty-project-template/src/main/scala/Main.scala)
[debug] [naha] Initial set of included nodes: Set(/Users/rkuhn/comp/test/dotty/dotty-project-template/src/main/scala/Main.scala)
[debug] [naha] Recompiling all 1 sources: invalidated sources (1) exceeded 50.0% of all sources
[debug] Forcing garbage collection...
java.lang.IllegalArgumentException: requirement failed: Source file '/Users/rkuhn/comp/test/dotty/dotty-project-template/target/scala-0.1/classes/Main$X$next.class' does not exist.
	at scala.Predef$.require(Predef.scala:233)
	at sbt.IO$.copyFile(IO.scala:650)
	at sbt.IO$.move(IO.scala:853)
	at sbt.inc.ClassfileManager$$anonfun$transactional$1$$anon$2.sbt$inc$ClassfileManager$$anonfun$$anon$$move(ClassfileManager.scala:75)
	at sbt.inc.ClassfileManager$$anonfun$transactional$1$$anon$2$$anonfun$delete$3.apply(ClassfileManager.scala:52)
	at sbt.inc.ClassfileManager$$anonfun$transactional$1$$anon$2$$anonfun$delete$3.apply(ClassfileManager.scala:51)
	at scala.collection.immutable.HashSet$HashSet1.foreach(HashSet.scala:153)
	at scala.collection.immutable.HashSet$HashTrieSet.foreach(HashSet.scala:306)
	at sbt.inc.ClassfileManager$$anonfun$transactional$1$$anon$2.delete(ClassfileManager.scala:51)
	at dotty.tools.sbtplugin.DottyPlugin$$anonfun$2$$anon$1.delete(DottyPlugin.scala:99)
	at sbt.inc.Incremental$.prune(Incremental.scala:92)
	at sbt.inc.IncrementalCommon.cycle(IncrementalCommon.scala:29)
	at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:72)
	at sbt.inc.Incremental$$anonfun$1.apply(Incremental.scala:71)
	at sbt.inc.Incremental$.manageClassfiles(Incremental.scala:99)
	at sbt.inc.Incremental$.compile(Incremental.scala:71)
	at sbt.inc.IncrementalCompile$.apply(Compile.scala:54)
	at sbt.compiler.IC$.compileInternal(IncrementalCompiler.scala:160)
	at sbt.compiler.IC$.incrementalCompile(IncrementalCompiler.scala:138)
	at sbt.Compiler$.compile(Compiler.scala:155)
	at sbt.Compiler$.compile(Compiler.scala:141)
	at sbt.Defaults$.sbt$Defaults$$compileIncrementalTaskImpl(Defaults.scala:886)
	at sbt.Defaults$$anonfun$compileIncrementalTask$1.apply(Defaults.scala:877)
	at sbt.Defaults$$anonfun$compileIncrementalTask$1.apply(Defaults.scala:875)
	at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
	at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
	at sbt.std.Transform$$anon$4.work(System.scala:63)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:228)
	at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
	at sbt.Execute.work(Execute.scala:237)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:228)
	at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
	at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)

[smarter]

Running sbt shell and after a few compile runs while editing the following happens:

I tried adding/removing spaces and commenting/uncommenting a subset of the code but couldn't get it to crash, do you have a series of step that reliably reproduces the problem for you?

[rkuhn]

Unfortunately not, it happens sporadically during the edit/compile cycle. It just recurred on the larger project where this code comes from, and all occurrences were about a class whose name corresponds to a private object in a nested object as in the code shown above. I have not seen this error with any other class.

[smarter]

Next time it happens, could you zip the whole project, and upload that somewhere?

[rkuhn]

It happened on the bigger project, ZIP file is here.

[smarter]

Hmm, this is pretty annoying, I think sbt incremental compilation output is not platform-independent so when I try compiling your project in the zip it just recompiles everything. Ideally we would need a VM image here :). Anyway, for now let's see if we can at least rule some things out, we have a custom incOptions in the sbt-dotty plugin that appears in your stacktrace, could you try using an old version of sbt-dotty that didn't have this custom incOptions? In projects/plugins.sbt you need:

addSbtPlugin("ch.epfl.lamp" % "sbt-dotty" % "0.1.0-RC5")

instead of 0.1.1. Then just work as normal and see if the problem pops up again (if it does, please provide an updated stacktrace).

[smarter]

@sjrd Either seen something like this with Scala.js use of ClassfileManager? Any clue what could be going wrong?

[rkuhn]

Okay, will do.

[smarter]

@rkuhn Haha, I think I got it: sbt/sbt#1777 It seems the issue is that you're using a non-case-sensitive filesystem and have an object and a trait in the same package whose name only differ by case (next and Next in your original example). In Scala 2 this is not a problem because no companion class is generated for an inner object, but in dotty it seems that we do generate it for some reason. I would suggest simply avoiding identifiers that only differ in case for objects/traits/classes, but we should investigate why we generate these companions.

[sjrd]

No, never seen anything like that.

[rkuhn]

Ah, nice catch, yes that makes sense. (well, not really, not in 2017, but still)

[rkuhn]

So, should the compiler emit an error in this case, or do users of broken filesystems get to keep the pieces?

[DarkDimius]

@rkuhn we should emit a warning #2679

[smarter]

Opened #2680 to track the useless companion emitted, I think we can close this issue now, thanks @rkuhn !

[DarkDimius]

We have machinery that should have prevented overwriting files. See here: #313 (comment)