Arbitrary self types v2: use Receiver trait

In this new version of Arbitrary Self Types, we no longer use the Deref trait
exclusively when working out which self types are valid. Instead, we follow a
chain of Receiver traits. This enables methods to be called on smart pointer
types which fundamentally cannot support Deref (for instance because they are
wrappers for pointers that don't follow Rust's aliasing rules).

This includes:
* Changes to tests appropriately
* New tests for:
  * The basics of the feature
  * Ensuring lifetime elision works properly
  * Generic Receivers
  * A copy of the method subst test enhanced with Receiver

This is really the heart of the 'arbitrary self types v2' feature, and
is the most critical commit in the current PR.

Subsequent commits are focused on:
* Detecting "shadowing" problems, where a smart pointer type can hide
  methods in the pointee.
* Diagnostics and cleanup.

Naming: in this commit, the "Autoderef" type is modified so that it no
longer solely focuses on the "Deref" trait, but can now consider the
"Receiver" trait instead. Should it be renamed, to something like
"TraitFollower"? This was considered, but rejected, because
* even in the Receiver case, it still considers built-in derefs
* the name Autoderef is short and snappy.

Author: adetaylor

Diff for: compiler/rustc_error_codes/src/error_codes/E0307.md

@@ -65,8 +65,10 @@ impl Trait for Foo {
 ```
 
 The nightly feature [Arbitrary self types][AST] extends the accepted
-set of receiver types to also include any type that can dereference to
-`Self`:
+set of receiver types to also include any type that implements the
+`Receiver` trait and can follow its chain of `Target` types to `Self`.
+There's a blanket implementation of `Receiver` for `T: Deref`, so any
+type which dereferences to `Self` can be used.
 
 ```
 #![feature(arbitrary_self_types)]

Diff for: compiler/rustc_hir_analysis/messages.ftl

@@ -241,10 +241,10 @@ hir_analysis_invalid_generic_receiver_ty_help =
     use a concrete type such as `self`, `&self`, `&mut self`, `self: Box<Self>`, `self: Rc<Self>`, `self: Arc<Self>`, or `self: Pin<P>` (where P is one of the previous types except `Self`)
 
 hir_analysis_invalid_receiver_ty = invalid `self` parameter type: `{$receiver_ty}`
-    .note = type of `self` must be `Self` or a type that dereferences to it
+    .note = type of `self` must be `Self` or some type implementing `Receiver`
 
 hir_analysis_invalid_receiver_ty_help =
-    consider changing to `self`, `&self`, `&mut self`, `self: Box<Self>`, `self: Rc<Self>`, `self: Arc<Self>`, or `self: Pin<P>` (where P is one of the previous types except `Self`)
+    consider changing to `self`, `&self`, `&mut self`, or a type implementing `Receiver` such as `self: Box<Self>`, `self: Rc<Self>`, or `self: Arc<Self>`
 
 hir_analysis_invalid_union_field =
     field must implement `Copy` or be wrapped in `ManuallyDrop<...>` to be used in a union

Diff for: compiler/rustc_hir_analysis/src/autoderef.rs

@@ -18,7 +18,6 @@ pub enum AutoderefKind {
     /// A type which must dispatch to a `Deref` implementation.
     Overloaded,
 }
-
 struct AutoderefSnapshot<'tcx> {
     at_start: bool,
     reached_recursion_limit: bool,
@@ -27,6 +26,10 @@ struct AutoderefSnapshot<'tcx> {
     obligations: PredicateObligations<'tcx>,
 }
 
+/// Recursively dereference a type, considering both built-in
+/// dereferences (`*`) and the `Deref` trait.
+/// Although called `Autoderef` it can be configured to use the
+/// `Receiver` trait instead of the `Deref` trait.
 pub struct Autoderef<'a, 'tcx> {
     // Meta infos:
     infcx: &'a InferCtxt<'tcx>,
@@ -39,6 +42,7 @@ pub struct Autoderef<'a, 'tcx> {
 
     // Configurations:
     include_raw_pointers: bool,
+    use_receiver_trait: bool,
     silence_errors: bool,
 }
 
@@ -69,6 +73,10 @@ impl<'a, 'tcx> Iterator for Autoderef<'a, 'tcx> {
         }
 
         // Otherwise, deref if type is derefable:
+        // NOTE: in the case of self.use_receiver_trait = true, you might think it would
+        // be better to skip this clause and use the Overloaded case only, since &T
+        // and &mut T implement Receiver. But built-in derefs apply equally to Receiver
+        // and Deref, and this has benefits for const and the emitted MIR.
         let (kind, new_ty) =
             if let Some(ty) = self.state.cur_ty.builtin_deref(self.include_raw_pointers) {
                 debug_assert_eq!(ty, self.infcx.resolve_vars_if_possible(ty));
@@ -111,7 +119,7 @@ impl<'a, 'tcx> Autoderef<'a, 'tcx> {
         body_def_id: LocalDefId,
         span: Span,
         base_ty: Ty<'tcx>,
-    ) -> Autoderef<'a, 'tcx> {
+    ) -> Self {
         Autoderef {
             infcx,
             span,
@@ -125,6 +133,7 @@ impl<'a, 'tcx> Autoderef<'a, 'tcx> {
                 reached_recursion_limit: false,
             },
             include_raw_pointers: false,
+            use_receiver_trait: false,
             silence_errors: false,
         }
     }
@@ -137,8 +146,13 @@ impl<'a, 'tcx> Autoderef<'a, 'tcx> {
             return None;
         }
 
-        // <ty as Deref>
-        let trait_ref = ty::TraitRef::new(tcx, tcx.lang_items().deref_trait()?, [ty]);
+        // <ty as Deref>, or whatever the equivalent trait is that we've been asked to walk.
+        let (trait_def_id, trait_target_def_id) = if self.use_receiver_trait {
+            (tcx.lang_items().receiver_trait()?, tcx.lang_items().receiver_target()?)
+        } else {
+            (tcx.lang_items().deref_trait()?, tcx.lang_items().deref_target()?)
+        };
+        let trait_ref = ty::TraitRef::new(tcx, trait_def_id, [ty]);
         let cause = traits::ObligationCause::misc(self.span, self.body_id);
         let obligation = traits::Obligation::new(
             tcx,
@@ -151,11 +165,8 @@ impl<'a, 'tcx> Autoderef<'a, 'tcx> {
             return None;
         }
 
-        let (normalized_ty, obligations) = self.structurally_normalize(Ty::new_projection(
-            tcx,
-            tcx.lang_items().deref_target()?,
-            [ty],
-        ))?;
+        let (normalized_ty, obligations) =
+            self.structurally_normalize(Ty::new_projection(tcx, trait_target_def_id, [ty]))?;
         debug!("overloaded_deref_ty({:?}) = ({:?}, {:?})", ty, normalized_ty, obligations);
         self.state.obligations.extend(obligations);
 
@@ -234,6 +245,14 @@ impl<'a, 'tcx> Autoderef<'a, 'tcx> {
         self
     }
 
+    /// Use `core::ops::Receiver` and `core::ops::Receiver::Target` as
+    /// the trait and associated type to iterate, instead of
+    /// `core::ops::Deref` and `core::ops::Deref::Target`
+    pub fn use_receiver_trait(mut self) -> Self {
+        self.use_receiver_trait = true;
+        self
+    }
+
     pub fn silence_errors(mut self) -> Self {
         self.silence_errors = true;
         self

Diff for: compiler/rustc_hir_analysis/src/check/wfcheck.rs

@@ -1821,13 +1821,18 @@ fn receiver_is_valid<'tcx>(
 
     let mut autoderef = Autoderef::new(infcx, wfcx.param_env, wfcx.body_def_id, span, receiver_ty);
 
+    // The `arbitrary_self_types` feature allows custom smart pointer
+    // types to be method receivers, as identified by following the Receiver<Target=T>
+    // chain.
+    if arbitrary_self_types_enabled.is_some() {
+        autoderef = autoderef.use_receiver_trait();
+    }
+
     // The `arbitrary_self_types_pointers` feature allows raw pointer receivers like `self: *const Self`.
     if arbitrary_self_types_enabled == Some(ArbitrarySelfTypesLevel::WithPointers) {
         autoderef = autoderef.include_raw_pointers();
     }
 
-    let receiver_trait_def_id = tcx.require_lang_item(LangItem::LegacyReceiver, Some(span));
-
     // Keep dereferencing `receiver_ty` until we get to `self_ty`.
     while let Some((potential_self_ty, _)) = autoderef.next() {
         debug!(
@@ -1849,11 +1854,13 @@ fn receiver_is_valid<'tcx>(
         }
 
         // Without `feature(arbitrary_self_types)`, we require that each step in the
-        // deref chain implement `receiver`.
+        // deref chain implement `LegacyReceiver`.
         if arbitrary_self_types_enabled.is_none() {
-            if !receiver_is_implemented(
+            let legacy_receiver_trait_def_id =
+                tcx.require_lang_item(LangItem::LegacyReceiver, Some(span));
+            if !legacy_receiver_is_implemented(
                 wfcx,
-                receiver_trait_def_id,
+                legacy_receiver_trait_def_id,
                 cause.clone(),
                 potential_self_ty,
             ) {
@@ -1866,7 +1873,7 @@ fn receiver_is_valid<'tcx>(
                 cause.clone(),
                 wfcx.param_env,
                 potential_self_ty,
-                receiver_trait_def_id,
+                legacy_receiver_trait_def_id,
             );
         }
     }
@@ -1875,22 +1882,22 @@ fn receiver_is_valid<'tcx>(
     Err(ReceiverValidityError::DoesNotDeref)
 }
 
-fn receiver_is_implemented<'tcx>(
+fn legacy_receiver_is_implemented<'tcx>(
     wfcx: &WfCheckingCtxt<'_, 'tcx>,
-    receiver_trait_def_id: DefId,
+    legacy_receiver_trait_def_id: DefId,
     cause: ObligationCause<'tcx>,
     receiver_ty: Ty<'tcx>,
 ) -> bool {
     let tcx = wfcx.tcx();
-    let trait_ref = ty::TraitRef::new(tcx, receiver_trait_def_id, [receiver_ty]);
+    let trait_ref = ty::TraitRef::new(tcx, legacy_receiver_trait_def_id, [receiver_ty]);
 
     let obligation = Obligation::new(tcx, cause, wfcx.param_env, trait_ref);
 
     if wfcx.infcx.predicate_must_hold_modulo_regions(&obligation) {
         true
     } else {
         debug!(
-            "receiver_is_implemented: type `{:?}` does not implement `Receiver` trait",
+            "receiver_is_implemented: type `{:?}` does not implement `LegacyReceiver` trait",
             receiver_ty
         );
         false

Diff for: compiler/rustc_hir_typeck/src/method/probe.rs

@@ -366,6 +366,7 @@ impl<'a, 'tcx> FnCtxt<'a, 'tcx> {
                         autoderefs: 0,
                         from_unsafe_deref: false,
                         unsize: false,
+                        reachable_via_deref: true,
                     }]),
                     opt_bad_ty: None,
                     reached_recursion_limit: false,
@@ -516,47 +517,93 @@ fn method_autoderef_steps<'tcx>(
     let (ref infcx, goal, inference_vars) = tcx.infer_ctxt().build_with_canonical(DUMMY_SP, &goal);
     let ParamEnvAnd { param_env, value: self_ty } = goal;
 
-    let mut autoderef =
+    // If arbitrary self types is not enabled, we follow the chain of
+    // `Deref<Target=T>`. If arbitrary self types is enabled, we instead
+    // follow the chain of `Receiver<Target=T>`, but we also record whether
+    // such types are reachable by following the (potentially shorter)
+    // chain of `Deref<Target=T>`. We will use the first list when finding
+    // potentially relevant function implementations (e.g. relevant impl blocks)
+    // but the second list when determining types that the receiver may be
+    // converted to, in order to find out which of those methods might actually
+    // be callable.
+    let mut autoderef_via_deref =
         Autoderef::new(infcx, param_env, hir::def_id::CRATE_DEF_ID, DUMMY_SP, self_ty)
             .include_raw_pointers()
             .silence_errors();
-    let mut reached_raw_pointer = false;
-    let mut steps: Vec<_> = autoderef
-        .by_ref()
-        .map(|(ty, d)| {
-            let step = CandidateStep {
-                self_ty: infcx.make_query_response_ignoring_pending_obligations(inference_vars, ty),
-                autoderefs: d,
-                from_unsafe_deref: reached_raw_pointer,
-                unsize: false,
-            };
-            if let ty::RawPtr(_, _) = ty.kind() {
-                // all the subsequent steps will be from_unsafe_deref
-                reached_raw_pointer = true;
-            }
-            step
-        })
-        .collect();
 
-    let final_ty = autoderef.final_ty(true);
+    let mut reached_raw_pointer = false;
+    let arbitrary_self_types_enabled =
+        tcx.features().arbitrary_self_types() || tcx.features().arbitrary_self_types_pointers();
+    let (mut steps, reached_recursion_limit): (Vec<_>, bool) = if arbitrary_self_types_enabled {
+        let reachable_via_deref =
+            autoderef_via_deref.by_ref().map(|_| true).chain(std::iter::repeat(false));
+
+        let mut autoderef_via_receiver =
+            Autoderef::new(infcx, param_env, hir::def_id::CRATE_DEF_ID, DUMMY_SP, self_ty)
+                .include_raw_pointers()
+                .use_receiver_trait()
+                .silence_errors();
+        let steps = autoderef_via_receiver
+            .by_ref()
+            .zip(reachable_via_deref)
+            .map(|((ty, d), reachable_via_deref)| {
+                let step = CandidateStep {
+                    self_ty: infcx
+                        .make_query_response_ignoring_pending_obligations(inference_vars, ty),
+                    autoderefs: d,
+                    from_unsafe_deref: reached_raw_pointer,
+                    unsize: false,
+                    reachable_via_deref,
+                };
+                if ty.is_unsafe_ptr() {
+                    // all the subsequent steps will be from_unsafe_deref
+                    reached_raw_pointer = true;
+                }
+                step
+            })
+            .collect();
+        (steps, autoderef_via_receiver.reached_recursion_limit())
+    } else {
+        let steps = autoderef_via_deref
+            .by_ref()
+            .map(|(ty, d)| {
+                let step = CandidateStep {
+                    self_ty: infcx
+                        .make_query_response_ignoring_pending_obligations(inference_vars, ty),
+                    autoderefs: d,
+                    from_unsafe_deref: reached_raw_pointer,
+                    unsize: false,
+                    reachable_via_deref: true,
+                };
+                if ty.is_unsafe_ptr() {
+                    // all the subsequent steps will be from_unsafe_deref
+                    reached_raw_pointer = true;
+                }
+                step
+            })
+            .collect();
+        (steps, autoderef_via_deref.reached_recursion_limit())
+    };
+    let final_ty = autoderef_via_deref.final_ty(true);
     let opt_bad_ty = match final_ty.kind() {
         ty::Infer(ty::TyVar(_)) | ty::Error(_) => Some(MethodAutoderefBadTy {
             reached_raw_pointer,
             ty: infcx.make_query_response_ignoring_pending_obligations(inference_vars, final_ty),
         }),
         ty::Array(elem_ty, _) => {
-            let dereferences = steps.len() - 1;
-
+            let autoderefs = steps.iter().filter(|s| s.reachable_via_deref).count() - 1;
             steps.push(CandidateStep {
                 self_ty: infcx.make_query_response_ignoring_pending_obligations(
                     inference_vars,
                     Ty::new_slice(infcx.tcx, *elem_ty),
                 ),
-                autoderefs: dereferences,
+                autoderefs,
                 // this could be from an unsafe deref if we had
                 // a *mut/const [T; N]
                 from_unsafe_deref: reached_raw_pointer,
                 unsize: true,
+                reachable_via_deref: true, // this is always the final type from
+                                           // autoderef_via_deref
             });
 
             None
@@ -569,7 +616,7 @@ fn method_autoderef_steps<'tcx>(
     MethodAutoderefStepsResult {
         steps: tcx.arena.alloc_from_iter(steps),
         opt_bad_ty: opt_bad_ty.map(|ty| &*tcx.arena.alloc(ty)),
-        reached_recursion_limit: autoderef.reached_recursion_limit(),
+        reached_recursion_limit,
     }
 }
 
@@ -1065,6 +1112,10 @@ impl<'a, 'tcx> ProbeContext<'a, 'tcx> {
     ) -> Option<PickResult<'tcx>> {
         self.steps
             .iter()
+            // At this point we're considering the types to which the receiver can be converted,
+            // so we want to follow the `Deref` chain not the `Receiver` chain. Filter out
+            // steps which can only be reached by following the (longer) `Receiver` chain.
+            .filter(|step| step.reachable_via_deref)
             .filter(|step| {
                 debug!("pick_all_method: step={:?}", step);
                 // skip types that are from a type error or that would require dereferencing

Diff for: compiler/rustc_middle/src/traits/query.rs

@@ -149,11 +149,21 @@ pub struct CandidateStep<'tcx> {
     /// `foo.by_raw_ptr()` will work and `foo.by_ref()` won't.
     pub from_unsafe_deref: bool,
     pub unsize: bool,
+    /// We will generate CandidateSteps which are reachable via a chain
+    /// of following `Receiver`. The first 'n' of those will be reachable
+    /// by following a chain of 'Deref' instead (since there's a blanket
+    /// implementation of Receiver for Deref).
+    /// We use the entire set of steps when identifying method candidates
+    /// (e.g. identifying relevant `impl` blocks) but only those that are
+    /// reachable via Deref when examining what the receiver type can
+    /// be converted into by autodereffing.
+    pub reachable_via_deref: bool,
 }
 
 #[derive(Copy, Clone, Debug, HashStable)]
 pub struct MethodAutoderefStepsResult<'tcx> {
-    /// The valid autoderef steps that could be found.
+    /// The valid autoderef steps that could be found by following a chain
+    /// of `Receiver<Target=T>` or `Deref<Target=T>` trait implementations.
     pub steps: &'tcx [CandidateStep<'tcx>],
     /// If Some(T), a type autoderef reported an error on.
     pub opt_bad_ty: Option<&'tcx MethodAutoderefBadTy<'tcx>>,

Diff for: tests/ui/async-await/inference_var_self_argument.stderr

@@ -4,8 +4,8 @@ error[E0307]: invalid `self` parameter type: `&dyn Foo`
 LL |     async fn foo(self: &dyn Foo) {
    |                        ^^^^^^^^
    |
-   = note: type of `self` must be `Self` or a type that dereferences to it
-   = help: consider changing to `self`, `&self`, `&mut self`, `self: Box<Self>`, `self: Rc<Self>`, `self: Arc<Self>`, or `self: Pin<P>` (where P is one of the previous types except `Self`)
+   = note: type of `self` must be `Self` or some type implementing `Receiver`
+   = help: consider changing to `self`, `&self`, `&mut self`, or a type implementing `Receiver` such as `self: Box<Self>`, `self: Rc<Self>`, or `self: Arc<Self>`
 
 error[E0038]: the trait `Foo` cannot be made into an object
   --> $DIR/inference_var_self_argument.rs:5:5

Diff for: tests/ui/async-await/issue-66312.stderr

@@ -4,8 +4,8 @@ error[E0307]: invalid `self` parameter type: `T`
 LL |     fn is_some(self: T);
    |                      ^
    |
-   = note: type of `self` must be `Self` or a type that dereferences to it
-   = help: consider changing to `self`, `&self`, `&mut self`, `self: Box<Self>`, `self: Rc<Self>`, `self: Arc<Self>`, or `self: Pin<P>` (where P is one of the previous types except `Self`)
+   = note: type of `self` must be `Self` or some type implementing `Receiver`
+   = help: consider changing to `self`, `&self`, `&mut self`, or a type implementing `Receiver` such as `self: Box<Self>`, `self: Rc<Self>`, or `self: Arc<Self>`
 
 error[E0308]: mismatched types
   --> $DIR/issue-66312.rs:9:8