Skip to content

Commit f21303f

Browse files
LuciferYangdongjoon-hyun
authored andcommitted
[SPARK-40497][BUILD] Upgrade Scala to 2.13.11
### What changes were proposed in this pull request? This PR aims to upgrade Scala to 2.13.11 - https://www.scala-lang.org/news/2.13.11 Additionally, this pr adds a new suppression rule for warning message: `Implicit definition should have explicit type`, this is a new compile check introduced by scala/scala#10083, we must fix it when we upgrading to use Scala 3, ### Why are the changes needed? This release improves collections, adds support for JDK 20 and 21, adds support for JDK 17 `sealed`: - scala/scala#10363 - scala/scala#10184 - scala/scala#10397 - scala/scala#10348 - scala/scala#10105 There are 2 known issues in this version: - scala/bug#12800 - scala/bug#12799 For the first one, there is no compilation warning messages related to `match may not be exhaustive` in Spark compile log, and for the second one, there is no case of `method.isAnnotationPresent(Deprecated.class)` in Spark code, there is just https://github.com/apache/spark/blob/8c84d2c9349d7b607db949c2e114df781f23e438/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala#L130 in Spark Code, and I checked `javax.annotation.Nonnull` no this issue. So I think These two issues will not affect Spark itself, but this doesn't mean it won't affect the code written by end users themselves The full release notes as follows: - https://github.com/scala/scala/releases/tag/v2.13.11 ### Does this PR introduce _any_ user-facing change? Yes, this is a Scala version change. ### How was this patch tested? - Existing Test - Checked Java 8/17 + Scala 2.13.11 using GA, all test passed Java 8 + Scala 2.13.11: https://github.com/LuciferYang/spark/runs/14337279564 Java 17 + Scala 2.13.11: https://github.com/LuciferYang/spark/runs/14343012195 Closes #41626 from LuciferYang/SPARK-40497. Authored-by: yangjie01 <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 8c84d2c commit f21303f

File tree

2 files changed

+8
-2
lines changed

2 files changed

+8
-2
lines changed

pom.xml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3569,7 +3569,7 @@
35693569
<profile>
35703570
<id>scala-2.13</id>
35713571
<properties>
3572-
<scala.version>2.13.8</scala.version>
3572+
<scala.version>2.13.11</scala.version>
35733573
<scala.binary.version>2.13</scala.binary.version>
35743574
</properties>
35753575
<build>
@@ -3628,6 +3628,10 @@
36283628
-->
36293629
<arg>-Wconf:cat=unused-imports&amp;src=org\/apache\/spark\/graphx\/impl\/VertexPartitionBase.scala:s</arg>
36303630
<arg>-Wconf:cat=unused-imports&amp;src=org\/apache\/spark\/graphx\/impl\/VertexPartitionBaseOps.scala:s</arg>
3631+
<!--
3632+
SPARK-40497 Upgrade Scala to 2.13.11 and suppress `Implicit definition should have explicit type`
3633+
-->
3634+
<arg>-Wconf:msg=Implicit definition should have explicit type:s</arg>
36313635
</args>
36323636
<compilerPlugins combine.self="override">
36333637
</compilerPlugins>

project/SparkBuild.scala

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -286,7 +286,9 @@ object SparkBuild extends PomBuild {
286286
// TODO(SPARK-43850): Remove the following suppression rules and remove `import scala.language.higherKinds`
287287
// from the corresponding files when Scala 2.12 is no longer supported.
288288
"-Wconf:cat=unused-imports&src=org\\/apache\\/spark\\/graphx\\/impl\\/VertexPartitionBase.scala:s",
289-
"-Wconf:cat=unused-imports&src=org\\/apache\\/spark\\/graphx\\/impl\\/VertexPartitionBaseOps.scala:s"
289+
"-Wconf:cat=unused-imports&src=org\\/apache\\/spark\\/graphx\\/impl\\/VertexPartitionBaseOps.scala:s",
290+
// SPARK-40497 Upgrade Scala to 2.13.11 and suppress `Implicit definition should have explicit type`
291+
"-Wconf:msg=Implicit definition should have explicit type:s"
290292
)
291293
}
292294
}

0 commit comments

Comments
 (0)