You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
title: SIP XX - Improving binary compatibility with @binaryCompatible
3
+
title: SIP XX - Improving binary compatibility with @stableABI
4
4
disqus: true
5
5
---
6
6
7
7
__Dmitry Petrashko__
8
8
9
9
__first submitted 13 January 2017__
10
10
11
-
## Introduction ##
11
+
## Introduction
12
12
13
-
Scala is a language which evolves fast and thus made a decision to only promise binary compatibility across minor releases.
13
+
Scala is a language which evolves fast and thus made a decision to only promise binary compatibility across minor releases\[[3]\].
14
14
At the same time, there is a demand to develop APIs that live longer than a major release cycle of Scala.
15
-
This SIP introduces an annotation `@binaryCompatible` that checks that `what you write is what you get`.
16
-
It will fail compilation in case emitted methods or their signatures
17
-
are different from those written by users.
18
-
As long as signatures of methods in source is not changed, `@binaryCompatible` annotated class
19
-
will be compatible across major version of Scala.
15
+
This SIP introduces an annotation `@stableABI` that checks that `what you write is what you get`.
16
+
It will fail compilation in case compilation scheme of this class requires changes to methods signatures,
17
+
making them different from those written by developers and introducing implicit dependency on version of this desugaring.
20
18
21
-
## Use Cases
19
+
As long as declarations in source have not changed, `@stableABI` annotated class will be compatible across major versions of Scala.
20
+
21
+
22
+
## Term definitions
23
+
* #####Binary descriptors
24
+
25
+
As defined by the JVM spec\[[4]\]:
26
+
> A descriptor is a string representing the type of a field or method. Descriptors are represented in the class file format using modified UTF-8 strings (§4.4.7)
27
+
and thus may be drawn, where not further constrained, from the entire Unicode codespace.
28
+
>
29
+
> A method descriptor contains zero or more parameter descriptors, representing the types of parameters that the method takes, and a return descriptor, representing the type of the value (if any) that the method returns.
30
+
31
+
Binary descriptors are used in the bytecode to indicate what fields and methods are accessed or invoked.
32
+
If method or field has his descriptor changed, previously compiled classes that used different descriptor will fail in
33
+
runtime as they no longer link to the changed field.
34
+
35
+
In this document we use term `binary descriptor` to refer to both method and field descriptors used by the JVM.
36
+
* #####Public API
37
+
38
+
Methods and fields marked with `ACC_PUBLIC`\[[5]\] may be accessed from any class and package.
39
+
This loosely corresponds to absence of AccessModifier\[[6]\] in Scala source.
40
+
Changing a binary descriptor of a method or a field marked with `ACC_PUBLIC` is a binary incompatible change
41
+
which may affect all classes in all packages leading to a runtime linkage failure.
42
+
43
+
Methods and fields marked with `ACC_PROTECTED`\[[5]\] may be accessed within subclasses.
44
+
This loosely corresponds to presence of `protected` AccessModifier\[[6]\] in Scala source.
45
+
Changing a binary descriptor of a method or a field marked with `ACC_PROTECTED` is a binary incompatible change
46
+
which may affect all subclasses of this class leading to a runtime linkage failure.
47
+
48
+
In this document we use term `Public API` to refer both to methods and fields defined as `ACC_PUBLIC` and `ACC_PROTECTED`.
49
+
Changes do binary descriptors of Public API may lead to runtime linkage failures.
50
+
* ####Binary compatibility
51
+
52
+
Two versions of the same class are called binary compatible if there are no changes to the Public API of this class,
53
+
meaning that those two classes can be substituted in runtime without linkage errors.
54
+
55
+
## Use cases
56
+
57
+
2. Defining a class which is supposed to be also used from Java\Kotlin.
58
+
`@stableABI` will ensure both binary compatibility and that there are no unexpected methods
59
+
that would show up in members of a class or an interface.
60
+
61
+
## Current Status
22
62
In case there's a need to develop an API that will be used by clients compiled using different major versions of Scala,
23
63
the current approach is to either develop them in Java or to use best guess to restrict what Scala features should be used.
24
-
There's also a different approach which is used by SBT: instead of publishing a binary `compiler-interface`, sources are published instead
25
-
that would be locally compiled.
26
64
27
-
There's also a use-case of defining a class which is supposed to be also used from Java.
28
-
`@binaryCompatible` will ensure that there are no not-expected methods that would show up in members of a class or an interface.
65
+
There's also a different approach which is used by SBT: instead of publishing a binary `compiler-interface`, sources are published instead that would be locally compiled.
66
+
67
+
Examples:
68
+
1. Zinc\[[8]\] is writing their interfaces in Java because the interface has to be Scala version agnostic, as it is shipped in every sbt release, independently of Scala version that was used to compiler zinc or will be used in to compile the project.
69
+
SBT additionally compiles on demand the compiler bridge, which implements this Java interface.
70
+
2. Dotty\[[7]\] currently uses java defined interfaces as public API for IntelliJ in order to ensure binary compatibility.
71
+
These interfaces can be replaced by `@stableABI` annotated traits to reach the same goal.
72
+
73
+
29
74
30
-
Dotty currently uses java defined interfaces as public API for IntelliJ in order to ensure binary compatibility.
31
-
These interfaces can be replaced by `@binaryCompatible` annotated traits to reach the same goal.
32
75
33
76
## Design Guidelines
34
-
`@binaryCompatible` is a feature which is supposed to be used by a small subset of the ecosystem to be binary compatible across major versions of Scala.
77
+
`@stableABI` is a feature which is supposed to be used by a small subset of the ecosystem to be binary compatible across major versions of Scala.
35
78
Thus this is designed as an advanced feature that is used rarely and thus is intentionally verbose.
36
79
It's designed to provide strong guarantees, in some cases sacrificing ease of use.
37
80
38
-
The limitations enforced by `@binaryCompatible` are designed to be an overapproximation:
39
-
instead of permitting a list of features known to be compatible, `@binaryCompatible` enforces a stronger
81
+
The limitations enforced by `@stableABI` are designed to be an overapproximation:
82
+
instead of permitting a list of features known to be compatible, `@stableABI` enforces a stronger
40
83
check which is sufficient to promise binary compatibility.
41
84
85
+
This SIP intentionally goes follows a very conservative approach.
86
+
This is because we will be able to allow more features later, but we won't have an opportunity to remove them.
87
+
42
88
## Overview ##
43
-
In order for a classor a trait to succeed compilation with the `@binaryCompatible` annotation it has to be:
89
+
In order for a class, trait or an object to succeed compilation with the `@stableABI` annotation it has to be:
44
90
- defined on the top level;
91
+
- if a class or an object has a companion annotated with `@stableABI`, than annotation applies to both of them;
45
92
- use a subset of Scala that during compilation does not require changes to public API of the class, including
46
93
- synthesizing new members, either concrete or abstract;
47
94
- changing binary signatures of existing members, either concrete or abstract;
48
95
49
-
`@binaryCompatible` does not change the compilation scheme of a class:
50
-
compiling a class previously annotated with the `@binaryCompatible`, will produce the same bytecode with or without `@binaryCompatible` annotation.
96
+
`@stableABI` does not change the compilation scheme of a class:
97
+
compiling a class previously annotated with the `@stableABI`, will produce the same bytecode with or without `@stableABI` annotation.
51
98
52
-
Below are several examples of classes and traits that succeed compilation with `@binaryCompatible`
99
+
Below are several examples of classes and traits that succeed compilation with `@stableABI`
53
100
```scala
54
101
{% highlight scala %}
55
-
@binaryCompatible
102
+
@stableABI
56
103
traitAbstractFile {
57
104
defname():String
58
105
@@ -61,12 +108,12 @@ trait AbstractFile {
61
108
defjfile():Optional[File]
62
109
}
63
110
64
-
@binaryCompatible
111
+
@stableABI
65
112
traitSourceFileextendsAbstractFile {
66
113
defcontent():Array[Char]
67
114
}
68
115
69
-
@binaryCompatible
116
+
@stableABI
70
117
traitDiagnostic {
71
118
defmessage():String
72
119
@@ -75,14 +122,14 @@ trait Diagnostic {
75
122
defposition():Optional[SourcePosition]
76
123
}
77
124
78
-
@binaryCompatible
125
+
@stableABI
79
126
objectDiagnostic {
80
127
@static finalvalERROR:Int=2
81
128
@static finalvalWARNING:Int=1
82
129
@static finalvalINFO:Int=0
83
130
}
84
131
85
-
@binaryCompatible
132
+
@stableABI
86
133
classFeaturesInBodies {
87
134
defapiMethod:Int= {
88
135
// as body of the method isn't part of the public interface, one can use all features of Scala here.
@@ -93,23 +140,23 @@ class FeaturesInBodies {
93
140
{% endhighlight %}
94
141
```
95
142
96
-
## Features that will fail compilation with `@binaryCompatible`
143
+
## Features that will fail compilation with `@stableABI`
97
144
The features listed below have complex encodings that may change in future versions. We prefer not to compromise on them.
98
-
Most of those features can be simulated in a binary compatible way by writing a verbose re-impelemtation
145
+
Most of those features can be simulated in a binary compatible way by writing a verbose re-implemtation
99
146
which won't rely on desugaring performed inside compiler.
100
147
Note that while those features are prohibited in the public API, they can be safely used inside bodies of the methods.
101
148
102
149
- public fields. Can be simulated by explicitly defining public getters and setters that access a private field;
103
150
- lazy vals. Can be simulated by explicitly writing an implementation in source;
104
151
- case classes. Can be simulated by explicitly defining getters and other members synthesized for a case class(`copy`, `productArity`, `apply`, `unApply`, `unapply`).
105
152
106
-
The features listed below cannot be easily re-implemented in a class or trait annotated with `@binaryCompatible`.
153
+
The features listed below cannot be easily re-implemented in a class or trait annotated with `@stableABI`.
107
154
- default arguments;
108
155
- default methods. See Addendum;
109
156
- constant types(both explicit and inferred);
110
157
- inline.
111
158
112
-
## `@binaryCompatible` and Scala.js
159
+
## `@stableABI` and Scala.js
113
160
114
161
Allowing to write API-defining classes in Scala instead of Java will allow them to compile with Scala.js,
115
162
which would have benefit of sharing the same source for two ecosystems.
@@ -122,26 +169,26 @@ Providing stronger binary compatibility guarantees for JVM will automatically pr
122
169
The Migration Manager for Scala (MiMa in short) is a tool for diagnosing binary incompatibilities for Scala libraries.
123
170
MiMa allows to compare binary APIs of two already compiled classfiles and reports errors if APIs do not match perfectly.
124
171
125
-
MiMa and `@binaryCompatible` complement each other, as `@binaryCompatible` helps to develop APIs that stay compatible
172
+
MiMa and `@stableABI` complement each other, as `@stableABI` helps to develop APIs that stay compatible
126
173
across major versions, while MiMa checks that previously published artifacts indeed have the same API.
127
174
128
-
`@binaryCompatible` does not compare the currently compiled class or trait against previous version,
175
+
`@stableABI` does not compare the currently compiled class or trait against previous version,
129
176
so introduction of new members won't be prohibited. This is a use-case for MiMa.
130
177
131
178
MiMa does not indicate how hard, if possible, would it be to maintain compatibility of a class across future versions of Scala.
132
179
Multiple features of Scala, most notably lazy vals and traits, has been compiled diffently by different Scala versions
133
180
making porting existing compiled bytecode across versions very hard.
134
181
MiMa will complain retroactively that the new version is incompatible with the old one.
135
-
`@binaryCompatible` will instead indicate at compile time that the old version had used features whose encoding is prone to change.
182
+
`@stableABI` will instead indicate at compile time that the old version had used features whose encoding is prone to change.
136
183
This provides early guidance and warning when designing long-living APIs before they are publicly released.
137
184
138
185
## Compilation scheme ##
139
186
No modification of typer or any existing phase is planned. The current proposed scheme introduces a late phase that runs before the very bytecode emission that checks that:
140
-
- classes and traits annotated as `@binaryCompatible` are on the top level;
141
-
- no non-private members where introduced inside classes and traits annotated as `@binaryCompatible` by compiler using phase travel;
142
-
- no non-private members inside classes and traits annotated as `@binaryCompatible` has changed their signature from the one written by developer.
187
+
- classes and traits annotated as `@stableABI` are on the top level;
188
+
- no non-private members where introduced inside classes and traits annotated as `@stableABI` by compiler using phase travel;
189
+
- no non-private members inside classes and traits annotated as `@stableABI` has changed their binary descriptor from the one written by developer.
143
190
144
-
The current prototype is implemented for Dotty and supports everything descibed in this SIP.
191
+
The current prototype is implemented for Dotty and supports everything described in this SIP.
145
192
The implementation is simple with less than 50 lines of non-boilerplate code.
146
193
The current implementation has a scope for improvement of error messages that will report domain specific details for disallowed features, but it already prohibits them.
147
194
@@ -161,8 +208,25 @@ The classes which have been correctly inheriting those traits compiled by previo
161
208
may need recompilation if trait has been recompiled with a new major version of Scala.
162
209
163
210
Thus, the authors of this SIP has decided not to allow default methods in the
164
-
`@binaryCompatible` traits.
211
+
`@stableABI` traits.
165
212
166
213
## See Also ##
167
-
*[dotty#1900](https://github.com/lampepfl/dotty/pull/1900) is an implementation for Dotty
0 commit comments