Skip to content

Commit cc28752

Browse files
spastorinospastorino
committed
Add RPITIT documentation
1 parent 218da21 commit cc28752

File tree

2 files changed

+289
-0
lines changed

2 files changed

+289
-0
lines changed

‎src/SUMMARY.md

+1
Original file line numberDiff line numberDiff line change
@@ -131,6 +131,7 @@
131131
- [Variance](./variance.md)
132132
- [Opaque Types](./opaque-types-type-alias-impl-trait.md)
133133
- [Inference details](./opaque-types-impl-trait-inference.md)
134+
- [Return Position Impl Trait In Trait](./return-position-impl-trait-in-trait.md)
134135
- [Pattern and Exhaustiveness Checking](./pat-exhaustive-checking.md)
135136
- [MIR dataflow](./mir/dataflow.md)
136137
- [Drop elaboration](./mir/drop-elaboration.md)

‎src/return-position-impl-trait-in-trait.md

+288
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,288 @@
1+
# Return Position Impl Trait In Trait
2+
3+
Return-position impl trait in trait (RPITIT) is conceptually (and as of [#112988], literally) sugar that turns RPITs in trait methods into generic associated types (GATs) without the user having to define that GAT either on the trait side or impl side.
4+
5+
RPITIT was originally implemented in [#101224], which added support for async fn in trait (AFIT), since the implementation for RPITIT came for free as a part of implementing AFIT which had been RFC'd previously. It was then RFC'd independently in [RFC 3425], which was recently approved by T-lang.
6+
7+
## How does it work?
8+
9+
This doc is ordered mostly via the compilation pipeline. AST -> HIR -> astconv -> typeck.
10+
11+
### AST and HIR
12+
13+
AST -> HIR lowering for RPITITs is almost the same as lowering RPITs. We still lower them as [`hir::ItemKind::OpaqueTy`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/struct.OpaqueTy.html). The two differences are that:
14+
15+
We record `in_trait` for the opaque. This will signify that the opaque is an RPITIT for astconv, diagnostics that deal with HIR, etc.
16+
17+
We record `lifetime_mapping`s for the opaque type, described below.
18+
19+
#### Aside: Opaque lifetime duplication
20+
21+
*All opaques* (not just RPITITs) end up duplicating their captured lifetimes into new lifetime parameters local to the opaque. The main reason we do this is because RPITs need to be able to "reify"[^1] any captured late-bound arguments, or make them into early-bound ones. This is so they can be used as substs for the opaque, and later to instantiate hidden types. Since we don't know which lifetimes are early- or late-bound during AST lowering, we just do this for all lifetimes.
22+
23+
[^1]: This is compiler-errors terminology, I'm not claiming it's accurate :^)
24+
25+
The main addition for RPITITs is that during lowering we track the relationship between the captured lifetimes and the corresponding duplicated lifetimes in an additional field, [`OpaqueTy::lifetime_mapping`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir/hir/struct.OpaqueTy.html#structfield.lifetime_mapping). We use this lifetime mapping later on in `predicates_of` to install bounds that enforce equality between these duplicated lifetimes and their source lifetimes in order to properly typecheck these GATs, which will be discussed below.
26+
27+
##### spastorino note:
28+
29+
It would be better if we were able to lower without duplicates and for that I think we would need to stop distinguishing between early and late bound lifetimes. So we would need a solution like [Account for late-bound lifetimes in generics #103448](https://github.com/rust-lang/rust/pull/103448) and then also a PR similar to [Inherit function lifetimes for impl-trait #103449](https://github.com/rust-lang/rust/pull/103449).
30+
31+
### Astconv
32+
33+
The main change to astconv is that we lower `hir::TyKind::OpaqueDef` for an RPITIT to a projection instead of an opaque, using a newly synthesized def-id for a new associated type in the trait. We'll describe how exactly we get this def-id in the next section.
34+
35+
This means that any time we call `ast_ty_to_ty` on the RPITIT, we end up getting a projection back instead of an opaque. This projection can then be normalized to the right value -- either the original opaque if we're in the trait, or the inferred type of the RPITIT if we're in an impl.
36+
37+
#### Lowering to synthetic associated types
38+
39+
Using query feeding, we synthesize new associated types on both the trait side and impl side for RPITITs that show up in methods.
40+
41+
##### Lowering RPITITs in traits
42+
43+
When `tcx.associated_item_def_ids(trait_def_id)` is called on a trait to gather all of the trait's associated types, the query previously just returned the def-ids of the HIR items that are children of the trait. After [#112988], additionally, for each method in the trait, we add the def-ids returned by `tcx.associated_types_for_impl_traits_in_associated_fn(trait_method_def_id)`, which walks through each trait method, gathers any RPITITs that show up in the signature, and then calls `associated_type_for_impl_trait_in_trait` for each RPITIT, which synthesizes a new associated type.
44+
45+
##### Lowering RPITITs in impls
46+
47+
Similarly, along with the impl's HIR items, for each impl method, we additionally add all of the `associated_types_for_impl_traits_in_associated_fn` for the impl method. This calls `associated_type_for_impl_trait_in_impl`, which will synthesize an associated type definition for each RPITIT that comes from the corresponding trait method.
48+
49+
#### Synthesizing new associated types
50+
51+
We use query feeding ([`TyCtxtAt::create_def`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/query/plumbing/struct.TyCtxtAt.html#method.create_def)) to synthesize a new def-id for the synthetic GATs for each RPITIT.
52+
53+
Locally, most of rustc's queries match on the HIR of an item to compute their values. Since the RPITIT doesn't really have HIR associated with it, or at least not HIR that corresponds to an associated type, we must compute many queries eagerly and [feed](https://github.com/rust-lang/rust/pull/104940) them, like `opt_def_kind`, `associated_item`, `visibility`, and`defaultness`.
54+
55+
The values for most of these queries is obvious, since the RPITIT conceptually inherits most of its information from the parent function (e.g. `visibility`), or because it's trivially knowable because it's an associated type (`opt_def_kind`).
56+
57+
Some other queries are more involved, or cannot be feeded, and we document the interesting ones of those below:
58+
59+
##### `generics_of` for the trait
60+
61+
The GAT for an RPITIT conceptually inherits the same generics as the RPIT it comes from. However, instead of having the method as the generics' parent, the trait is the parent.
62+
63+
Currently we get away with taking the RPIT's generics and method generics and flattening them both into a new generics list, preserving the def-id of each of the parameters. (This may cause issues with def-ids having the wrong parents, but in the worst case this will cause diagnostics issues. If this ends up being an issue, we can synthesize new def-ids for generic params whose parent is the GAT.)
64+
65+
<details>
66+
<summary> <b>An illustrated example</b> </summary>
67+
68+
```rust
69+
trait Foo {
70+
fn method<'early: 'early, 'late, T>() -> impl Sized + Captures<'early, 'late>;
71+
}
72+
```
73+
74+
Would desugar to...
75+
```rust
76+
trait Foo {
77+
// vvvvvvvvv method's generics
78+
// vvvvvvvvvvvvvvvvvvvvvvvv opaque's generics
79+
type Gat<'early, T, 'early_duplicated, 'late>: Sized + Captures<'early_duplicated, 'late>;
80+
81+
fn method<'early: 'early, 'late, T>() -> Self::Gat<'early, T, 'early, 'late>;
82+
}
83+
```
84+
</details>
85+
86+
##### `generics_of` for the impl
87+
88+
The generics for an impl's GAT are a bit more interesting. They are composed of RPITIT's own generics (from the trait definition), appended onto the impl's methods generics. This has the same issue as above, where the generics for the GAT have parameters whose def-ids have the wrong parent, but this should only cause issues in diagnostics.
89+
90+
We could fix this similarly if we were to synthesize new generics def-ids, but this can be done later in a forwards-compatible way, perhaps by a interested new contributor.
91+
92+
##### `opt_rpitit_info`
93+
94+
Some queries rely on computing information that would result in cycles if we were to feed them eagerly, like `explicit_predicates_of`. Therefore we defer to the `predicates_of` provider to return the right value for our RPITIT's GAT. We do this by detecting early on in the query if the associated type is synthetic by using [`opt_rpitit_info`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/context/struct.TyCtxt.html#method.opt_rpitit_info), which returns `Some` if the associated type is synthetic.
95+
96+
Then, during a query like `explicit_predicates_of`, we can detect if an associated type is synthetic like:
97+
98+
```rust
99+
fn explicit_predicates_of(tcx: TyCtxt<'_>, def_id: LocalDefId) -> ... {
100+
if let Some(rpitit_info) = tcx.opt_rpitit_info(def_id) {
101+
// Do something special for RPITITs...
102+
return ...;
103+
}
104+
105+
// The regular computation which relies on access to the HIR of `def_id`.
106+
}
107+
```
108+
109+
##### `explicit_predicates_of`
110+
111+
RPITITs begin by copying the predicates of the method that defined it, both on the trait and impl side.
112+
113+
Additionally, we install "bidirectional outlives" predicates. Specifically, we add region-outlives predicates in both directions for each captured early-bound lifetime that constrains it to be equal to the duplicated early-bound lifetime that results from lowering. This is best illustrated in an example:
114+
115+
```rust
116+
trait Foo<'a> {
117+
fn bar() -> impl Sized + 'a;
118+
}
119+
120+
// Desugars into...
121+
122+
trait Foo<'a> {
123+
type Gat<'a_duplicated>: Sized + 'a
124+
where
125+
'a: 'a_duplicated,
126+
'a_duplicated: 'a;
127+
//~^ Specifically, we should be able to assume that the
128+
// duplicated `'a_duplicated` lifetime always stays in
129+
// sync with the `'a` lifetime.
130+
131+
fn bar() -> Self::Gat<'a>;
132+
}
133+
```
134+
135+
##### `assumed_wf_types`
136+
137+
The GATs in both the trait and impl inherit the `assumed_wf_types` of the trait method that defines the RPITIT. This is to make sure that the following code is well formed when lowered.
138+
139+
```rust
140+
trait Foo {
141+
fn iter<'a, T>(x: &'a [T]) -> impl Iterator<Item = &'a T>;
142+
}
143+
144+
// which is lowered to...
145+
146+
trait FooDesugared {
147+
type Iter<'a, T>: Iterator<Item = &'a T>;
148+
//~^ assumed wf: `&'a [T]`
149+
// Without assumed wf types, the GAT would not be well-formed on its own.
150+
151+
fn iter<'a, T>(x: &'a [T]) -> Self::Iter<'a, T>;
152+
}
153+
```
154+
155+
Because `assumed_wf_types` is only defined for local def ids, in order to properly implement `assumed_wf_types` for impls of foreign traits with RPITs, we need to encode the assumed wf types of RPITITs in an extern query [`assumed_wf_types_for_rpitit`](https://github.com/rust-lang/rust/blob/a17c7968b727d8413801961fc4e89869b6ab00d3/compiler/rustc_ty_utils/src/implied_bounds.rs#L14).
156+
157+
### Typechecking, etc.
158+
159+
#### The RPITIT inference algorithm
160+
161+
The RPITIT inference algorithm is implemented in [`collect_return_position_impl_trait_in_trait_tys`](https://doc.rust-lang.org/nightly/nightly-rustc/rustc_hir_analysis/check/compare_impl_item/fn.collect_return_position_impl_trait_in_trait_tys.html).
162+
163+
**High-level:** Given a impl method and a trait method, we take the trait method and instantiate each RPITIT in the signature with an infer var. We then equate this trait method signature with the impl method signature, and process all obligations that fall out in order to infer the type of all of the RPITITs in the method.
164+
165+
The method is also responsible for making sure that the hidden types for each RPITIT actually satisfy the bounds of the `impl Trait`, i.e. that if we infer `impl Trait = Foo`, that `Foo: Trait` holds.
166+
167+
<details>
168+
<summary><b>An example...</b></summary>
169+
170+
```rust
171+
#![feature(return_position_impl_trait_in_trait)]
172+
173+
use std::ops::Deref;
174+
175+
trait Foo {
176+
fn bar() -> impl Deref<Target = impl Sized>;
177+
// ^- RPITIT ?0 ^- RPITIT ?1
178+
}
179+
180+
impl Foo for () {
181+
fn bar() -> Box<String> { Box::new(String::new()) }
182+
}
183+
```
184+
185+
We end up with the trait signature that looks like `fn() -> ?0`, and nested obligations `?0: Deref<Target = ?1>`, `?1: Sized`. The impl signature is `fn() -> Box<String>`.
186+
187+
Equating these signatures gives us `?0 = Box<String>`, which then after processing the obligation `Box<String>: Deref<Target = ?1>` gives us `?1 = String`, and the other obligation `String: Sized` evaluates to true.
188+
189+
By the end of the algorithm, we end up with a mapping between associated type def-ids to concrete types inferred from the signature. We can then use this mapping to implement `type_of` for the synthetic associated types in the impl, since this mapping describes the type that should come after the `=` in `type Assoc = ...` for each RPITIT.
190+
</details>
191+
192+
#### Default trait body
193+
194+
Type-checking a default trait body, like:
195+
196+
```rust
197+
trait Foo {
198+
fn bar() -> impl Sized {
199+
1i32
200+
}
201+
}
202+
```
203+
204+
requires one interesting hack. We need to install a projection predicate into the param-env of `Foo::bar` allowing us to assume that the RPITIT's GAT normalizes to the RPITIT's opaque type. This relies on the observation that a trait method and RPITIT's GAT will always be "in sync". That is, one will only ever be overridden if the other one is as well.
205+
206+
Compare this to a similar desugaring of the code above, which would fail because we cannot rely on this same assumption:
207+
208+
```rust
209+
#![feature(impl_trait_in_assoc_type)]
210+
#![feature(associated_type_defaults)]
211+
212+
trait Foo {
213+
type RPITIT = impl Sized;
214+
215+
fn bar() -> Self::RPITIT {
216+
01i32
217+
}
218+
}
219+
```
220+
221+
Failing because a down-stream impl could theoretically provide an implementation for `RPITIT` without providing an implementation of `foo`:
222+
223+
```text
224+
error[E0308]: mismatched types
225+
--> src/lib.rs:8:9
226+
|
227+
5 | type RPITIT = impl Sized;
228+
| ------------------------- associated type defaults can't be assumed inside the trait defining them
229+
6 |
230+
7 | fn bar() -> Self::RPITIT {
231+
| ------------ expected `<Self as Foo>::RPITIT` because of return type
232+
8 | 01i32
233+
| ^^^^^ expected associated type, found `i32`
234+
|
235+
= note: expected associated type `<Self as Foo>::RPITIT`
236+
found type `i32`
237+
```
238+
239+
#### Well-formedness checking
240+
241+
We check well-formedness of RPITITs just like regular associated types.
242+
243+
Since we added lifetime bounds in `predicates_of` that link the duplicated early-bound lifetimes to their original lifetimes, and we implemented `assumed_wf_types` which inherits the WF types of the method from which the RPITIT originates ([#113704]), we have no issues WF-checking the GAT as if it were a regular GAT.
244+
245+
### What's broken, what's weird, etc.
246+
247+
##### Specialization is super busted
248+
249+
The "default trait methods" described above does not interact well with specialization, because we only install those projection bounds in trait default methods, and not in impl methods. Given that specialization is already pretty busted, I won't go into detail, but it's currently a bug tracked in:
250+
* `tests/ui/impl-trait/in-trait/specialization-broken.rs`
251+
252+
##### Projections don't have variances
253+
254+
This code fails because projections don't have variances:
255+
```rust
256+
#![feature(return_position_impl_trait_in_trait)]
257+
258+
trait Foo {
259+
// Note that the RPITIT below does *not* capture `'lt`.
260+
fn bar<'lt: 'lt>() -> impl Eq;
261+
}
262+
263+
fn test<'a, 'b, T: Foo>() -> bool {
264+
<T as Foo>::bar::<'a>() == <T as Foo>::bar::<'b>()
265+
//~^ ERROR
266+
// (requires that `'a == 'b`)
267+
}
268+
```
269+
270+
This is because we can't relate `<T as Foo>::Rpitit<'a>` and `<T as Foo>::Rpitit<'b>`, even if they don't capture their lifetime. If we were using regular opaque types, this would work, because they would be bivariant in that lifetime parameter:
271+
```rust
272+
#![feature(return_position_impl_trait_in_trait)]
273+
274+
fn bar<'lt: 'lt>() -> impl Eq {
275+
()
276+
}
277+
278+
fn test<'a, 'b>() -> bool {
279+
bar::<'a>() == bar::<'b>()
280+
}
281+
```
282+
283+
This is probably okay though, since RPITITs will likely have their captures behavior changed to capture all in-scope lifetimes anyways. This could also be relaxed later in a forwards-compatible way if we were to consider variances of RPITITs when relating projections.
284+
285+
[#112988]: https://github.com/rust-lang/rust/pull/112988
286+
[RFC 3425]: https://github.com/rust-lang/rfcs/pull/3425
287+
[#101224]: https://github.com/rust-lang/rust/pull/101224
288+
[#113704]: https://github.com/rust-lang/rust/pull/113704

0 commit comments

Comments
 (0)