[POC] sbt 1.5.0 validation

[adpi2]

The purpose of this PR is not to be merged. It is a peek in the future to check that sbt 1.5 will be able to replace sbt-dotty.

sbt/sbt#6342 (comment)

For this purpose:

1. sbt is bumped to 1.5.0-bin-20210302T081602
2. the use of sbt-dotty in the main build is removed
3. sbt-dotty is removed from all scripted tests:
   Including the scripted tests of the sbt-dotty project itself to show that sbt-dotty is not needed anymore. All the tests passed with no trouble.
4. sbt-dotty is removed from almost all the community sbt projects (46 out of 52), except the 6 ones below

- cats-effect-2
- cats-effect-3
- cats-mtl
- coop
- scodec-bits
- scodec

These projects use the SpiewakPlugin which requires the DottyPlugin (see here) and it was too much of a hassle for me to make things work without the SpiewakPlugin.

As an alternative, I bumped the version of sbt-dotty to 0.5.3 explicity in each one of them. So it shows that sbt-dotty is still compatible with sbt 1.5 when it comes to compiling, running and testing. However it appears that sbt-dotty doc task is not compatible with sbt 1.5.

I temporarily removed the doc checking of cats-effects and scodecs projects.

What I have learned during the process:

Most often than not, removing sbt-dotty is straightforward:

• replace isDotty.value with one of the alternatives

isScala.value.startsWith("3")
VersionNumber(scalaVersion.value).matchesSemVer(SemanticSelector(">= 3.0.0-M1"))

• replace .withDottyCompat(scalaVersion.value) with .cross(CrossVersion.for3Use2_13)
• remove occurences of useScaladoc := true

There are 2 small difficulties:

1. libraryDependencies.map(_.withDottyCompat(scalaVersion.value))

This pattern cannot be naively rewritten into

libraryDependencies.map(_.cross(CrossVersion.for3Use2_13))

because:

• sbt 1.5.0 adds the scala3-library to the libraryDependencies
• CrossVersion.for3Use2_13 cannot be used on java dependencies:

("org.scala-sbt" % "test-interface" % "1.0").cross(CrossVersion.for3Use2_13) // incorrect, try to resolve test-interface_2.13
"org.scala-sbt" % "test-interface" % "1.0" // correct

One general solution is

libraryDependencies.value.map { module =>
  if (module.name != ScalaArtifacts.Scala3LibraryID && module.crossVersion == CrossVersion.binary)
    module.cross(CrossVersion.for3Use2_13)
  else module
}

2. dottyLatestNightlyBuild

The code is here.

This feature is obviously not supported by sbt 1.5. It may deserve to be published as an independent library.

[griggt]

@adpi2 FYI the community_build_a failures are due to #11604 (there is a race condition in the scalatest build)

[smarter]

Could this method be made public upstream?

[adpi2]

Don't think that's worth it. It's only useful for works on the Scala compilers and we probably don't want to maintain compatibility just in case other people use it.

[smarter]

Your call, but It seems better than having to deal with the complicated classloader caching logic API directly. And does this need to be in package sbt?

[adpi2]

It does because the ScalaInstance constructor is package private. But we could as well create our own ScalaInstance implementation.

This code is a quick and dirty copy-paste, it can be made simpler.

[smarter]

It does because the ScalaInstance constructor is package private.

It looks public to me (https://github.com/sbt/zinc/blob/13ddb10938237263f706174439d8ed055e8e4ac2/internal/zinc-classpath/src/main/scala/sbt/internal/inc/ScalaInstance.scala#L34-L43), and weren't we already using it before this PR?

[smarter]

This PR has a commit which adds the pdbp submodule directory (which was removed from the dotty repo in a previous PR) without adding it to .gitmodules, the result is that git explodes with "fatal: No url found for submodule path 'community-build/community-projects/pdbp' in .gitmodules" and because 1) the git repo is not cleaned up afterwards and 2) the git checkout action runs "git submodule foreach ..." before it pulls the current git branch, this affected other PRs hence why the CI exploded: https://github.com/lampepfl/dotty/commits/master

The pdbp submodule has been restored on master so if you rebase things should work out.