scaladoc bash scripts abstract away actual issues

[ckipp01]

Compiler version

HEAD

Minimized example

There isn't a great way to minimize this, but this is related to the issues I'm having in #16846. If you're making changes to scaladoc and you need to test it, the scaladoc CI has a huge mess of bash scripts. The issue is that when something is going wrong, it gives no indication about what is happening or why something is failing. For example when you run the following:

./project/scripts/cmdScaladocTests

which is ran in CI, if something is failing you'll just see the following:

+ SOURCE_LINKS_VERSION=3.3.1-RC1-bin-SNAPSHOT
+ /Users/ckipp/Documents/scala-workspace/dotty/project/scripts/../../project/scripts/sbt scaladoc/generateTestcasesDocumentation
+ dist/target/pack/bin/scaladoc -d /var/folders/fq/nx_jsnyd6550xp03czx898d40000gn/T/tmp.J4Ht4qwN -project 'scaladoc testcases' -source-links:out/bootstrap/stdlib-bootstrapped/scala-3.3.1-RC1-bin-SNAPSHOT-nonbootstrapped/src_managed/main/scala-library-src=github://scala/scala/v2.13.10#src/library -source-links:out/bootstrap/stdlib-bootstrapped/scala-3.3.1-RC1-bin-SNAPSHOT-nonbootstrapped/src_managed/main/dotty-library-src=github://lampepfl/dotty/3.3.1-RC1-bin-SNAPSHOT#library/src -source-links:github://lampepfl/dotty/3.3.1-RC1-bin-SNAPSHOT '-external-mappings:.*scala/.*::scaladoc3::https://dotty.epfl.ch/api/,.*java/.*::javadoc::https://docs.oracle.com/javase/8/docs/api/' '-skip-by-regex:.+\.internal($|\..+)' '-skip-by-regex:.+\.impl($|\..+)' -project-logo docs/_assets/images/logo.svg -social-links:github::https://github.com/lampepfl/dotty,discord::https://discord.com/invite/scala,twitter::https://twitter.com/scala_lang -Ygenerate-inkuire -skip-by-id:scala.runtime.stdLibPatches -skip-by-id:scala.runtime.MatchCase -snippet-compiler:scaladoc-testcases/docs=compile -comment-syntax:scaladoc-testcases/src/example/comment-md=markdown,scaladoc-testcases/src/example/comment-wiki=wiki -siteroot scaladoc-testcases/docs -project-footer 'Copyright (c) 2002-2023, LAMP/EPFL' -default-template static-site-main -author -groups -revision main -project-version 3.3.1-RC1-bin-SNAPSHOT -quick-links:Learn::https://docs.scala-lang.org/,Install::https://www.scala-lang.org/download/,Playground::https://scastie.scala-lang.org,Find A Library::https://index.scala-lang.org,Community::https://www.scala-lang.org/community/,Blog::https://www.scala-lang.org/blog/, out/bootstrap/scaladoc-testcases/scala-3.3.1-RC1-bin-SNAPSHOT-nonbootstrapped/classes
+ echo 'generated testcases project with scripts'
generated testcases project with scripts
+ diff -rq /var/folders/fq/nx_jsnyd6550xp03czx898d40000gn/T/tmp.J4Ht4qwN scaladoc/output/testcases
diff: /var/folders/fq/nx_jsnyd6550xp03czx898d40000gn/T/tmp.J4Ht4qwN: No such file or directory

So the problem is that no such file or directory exists? Well why not? You have to manually re-run this command with some slight changes to actually see that scaladoc is throwing:

6 warnings found
Exception in thread "main" java.lang.NoClassDefFoundError: com/vladsch/flexmark/util/data/MutableDataSetter
        at dotty.tools.scaladoc.ScalaModuleProvider$.mkModule(ScalaModuleProvider.scala:42)
        at dotty.tools.scaladoc.Scaladoc$.run(Scaladoc.scala:236)
        at dotty.tools.scaladoc.Scaladoc$.run$$anonfun$1(Scaladoc.scala:68)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
        at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
        at scala.Option.map(Option.scala:242)
        at dotty.tools.scaladoc.Scaladoc$.run(Scaladoc.scala:72)
        at dotty.tools.scaladoc.Main.run(Main.scala:8)
        at dotty.tools.scaladoc.Main$.main(Main.scala:14)
        at dotty.tools.scaladoc.Main.main(Main.scala)
Caused by: java.lang.ClassNotFoundException: com.vladsch.flexmark.util.data.MutableDataSetter
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520)
        ... 10 more

This is a really difficult workflow for someone unfamiliar with scaladoc or how it's all supposed to work together and it buries the actual issue.

Expectation

I'd expect to be able to run the scaladoc tests locally and somewhat easily see why something is wrong without having to dive through a bunch of bash scripts and recreate the command that I need to run from scratch to mimic it.

[ckipp01]

I just hit on this again, which is always frustrating. For example when trying to figure out #17469 you would just get the following error when running ./project/scripts/cmdScaladocTests:

+ /Users/ckipp/Documents/scala-workspace/dotty/project/scripts/../../project/scripts/sbt scaladoc/generateTestcasesDocumentation
+ echo 'generated testcases project with sbt'
generated testcases project with sbt

Which made it seem like everything was fine, but then it'd fail with:

+ diff -rq /var/folders/fq/nx_jsnyd6550xp03czx898d40000gn/T/tmp.QIHVcfwc scaladoc/output/testcases
diff: scaladoc/output/testcases: No such file or directory

You'd have to actually step through everything to find that the issue was really this command:

sbt:scala3> scaladoc/generateTestcasesDocumentation
[error] stack trace is suppressed; run last scaladoc / generateTestcasesDocumentation for the full output
[error] (scaladoc / generateTestcasesDocumentation) sbt.internal.util.Init$RuntimeUndefined: References to undefined settings at runtime.
[error] ScopedKey(Scope(Select(ProjectRef(file:/Users/ckipp/Documents/scala-workspace/dotty/,dist)), Zero, Zero, Zero),pack) referenced from setting(ScopedKey(Scope(Select(ProjectRef(file:/Users/ckipp/Documents/scala-workspace/dotty/,scaladoc)), Zero, Zero, Zero),generateTestcasesDocumentation)) at LinePosition(/Users/ckipp/Documents/scala-workspace/dotty/project/Build.scala,1501)
[error] Total time: 0 s, completed 11 May 2023, 11:20:53