unsafeNulls import doesn't work properly for java methods with generic return type

[rochala]

Compiler version

3.4.0-RC1-bin-20231024-15033c7-NIGHTLY

Java methods with generic return type fails when we return them inside anonymous functions and rely on type inference.

Minimized code

Test.scala

//> using scala 3.4.0-RC1-bin-20231024-15033c7-NIGHTLY
//> using option "-Yexplicit-nulls"
//> using file Test.java

import scala.language.unsafeNulls

object A {
  protected def decodeJson[T](test: Test): Option[T] =
    val x = Option(???).map(_ => test.fromJson[T]())
    x
}

Test.java

class Test {
  public <T> T fromJson() {
    return null;
  }
}

And it is working for:

//> using scala 3.4.0-RC1-bin-20231024-15033c7-NIGHTLY
//> using option "-Yexplicit-nulls"
//> using file Test.java

import scala.language.unsafeNulls

object A {
  protected def decodeJson[T](test: Test): Option[T] =
    Option(???).map(_ => test.fromJson[T]())
}

The origin problem was found in: https://github.com/scalameta/metals/blob/0d2739f6dfd96c455dadeca56dc85581e3bc48de/mtags-shared/src/main/scala/scala/meta/internal/mtags/CommonMtagsEnrichments.scala#L41-L60

Output

Compilation fails even with import scala.language.unsafeNulls

❯ scala-cli compile Test.scala Test.java
Compiling project (Scala 3.4.0-RC1-bin-20231024-15033c7-NIGHTLY, JVM)
[error] ./Test.scala:10:5
[error] Found:    (x : Option[T | Null])
[error] Required: Option[T]
[error]     x
[error]     ^
Error compiling project (Scala 3.4.0-RC1-bin-20231024-15033c7-NIGHTLY, JVM)
Compilation failed

Expectation

It should compile.

[noti0na1]

These errors are expected according to rules of unsafeNulls. The inferred type arguments for map are different, which causes different behaviours.

• In the first example, the type argument of map is inferred as T | Null. The type of test.fromJson[T] is T | Null, and the type of x is Option[T | Null]. The unsafeNulls only makes Null a subtype of all reference types in the scope. Since T is bound by >: Nothing <: Any, Null is still not a subtype of T.

• In the seound example, the type argument of map is inferred as T, using the information from the result type. The expected return type of the closure is T, but the type of test.fromJson[T] is T | Null. When a type mismatch like this happens on calling java methods, unsafeNulls will try to mitigate the error by stripping the outmost Null and type check again. https://github.com/lampepfl/dotty/blob/c0eae68577f00f0e14c6331a173d8b8750b05107/compiler/src/dotty/tools/dotc/typer/Typer.scala#L4260-L4267

One workaround is to define the type parameter as T >: Null <: AnyRef | Null.