Stack overflow when processing many command line arguments

[oyvindberg]

Hey there.

I noticed the compiler fails with a StackOverflowError after around 6000 arguments. I had a quick look, and the problem seems to be non stack safe recursive method. I didn't have time to fix it myself, but I did come up with a failling test.

I happened upon this because I setup compilation with bloop, which somewhere along the way produces a list of all input files, for hashing purposes it seems. All these file names are then passed to the compiler as command line arguments.

Compiler version

3.0.0

Failing test case

--- a/compiler/test/dotty/tools/dotc/SettingsTests.scala
+++ b/compiler/test/dotty/tools/dotc/SettingsTests.scala
@@ -54,6 +54,13 @@ class SettingsTests {
       assertEquals(1, Settings.bar.value)
     }
 
+  @Test def `dont crash on many files`: Unit =
+    inContext {
+      val args = List.fill(6000)("file.scala")
+      val summary = new config.ScalaSettings().processArguments(args, true)
+      assertEquals(Nil, summary.errors)
+    }
+
   @Test def validateChoices: Unit =
     object Settings extends SettingGroup:
       val foo = ChoiceSetting("-foo", "foo", "Foo", List("a", "b"), "a")

Output

java.lang.StackOverflowError
	at scala.collection.immutable.List.equals(List.scala:612)
	at dotty.tools.dotc.config.Settings$SettingGroup.processArguments(Settings.scala:227)
	at dotty.tools.dotc.config.Settings$SettingGroup.processArguments(Settings.scala:242)
	at dotty.tools.dotc.config.Settings$SettingGroup.processArguments(Settings.scala:242)

Expectation

[som-snytt]

Thanks for the test case. It turns out you did have time to DYI, but I appreciate the opportunity to remember how to run a test in dotty. Just adding final was sufficient, even though other calls are not tailrec.

What took so long was that you add tailrec, it says it can't because not final, so you add final, then it says it can't because of the other calls, then you remove tailrec, and the test passes and you think you must have broken the test, or sbt didn't run it or incremental compilation is broken.

[oyvindberg]

Great! I started down that path too, but didn't notice the test started passing. Started rewriting the entire thing and figured somebody else would be able to jump in and fix it with a smaller change. Happy I was proved right :)