Auto merge of #142070 - matthiaskrgr:rollup-e7lxtuo, r=matthiaskrgr

Rollup of 6 pull requests

Successful merges:

 - #140638 (UnsafePinned: also include the effects of UnsafeCell)
 - #141777 (Do not run PGO/BOLT in x64 Linux alt builds)
 - #141938 (update rust offload bootstrap)
 - #141962 (rustc-dev-guide subtree update)
 - #141965 (`tests/ui`: A New Order [3/N])
 - #141970 (implement new `x` flag: `--skip-std-check-if-no-download-rustc`)

r? `@ghost`
`@rustbot` modify labels: rollup

Author: bors

library/core/src/pin/unsafe_pinned.rs

@@ -1,10 +1,13 @@
+use crate::cell::UnsafeCell;
 use crate::marker::{PointerLike, Unpin};
 use crate::ops::{CoerceUnsized, DispatchFromDyn};
 use crate::pin::Pin;
 use crate::{fmt, ptr};
 
-/// This type provides a way to opt-out of typical aliasing rules;
+/// This type provides a way to entirely opt-out of typical aliasing rules;
 /// specifically, `&mut UnsafePinned<T>` is not guaranteed to be a unique pointer.
+/// This also subsumes the effects of `UnsafeCell`, i.e., `&UnsafePinned<T>` may point to data
+/// that is being mutated.
 ///
 /// However, even if you define your type like `pub struct Wrapper(UnsafePinned<...>)`, it is still
 /// very risky to have an `&mut Wrapper` that aliases anything else. Many functions that work
@@ -17,38 +20,24 @@ use crate::{fmt, ptr};
 /// the public API of a library. It is an internal implementation detail of libraries that need to
 /// support aliasing mutable references.
 ///
-/// Further note that this does *not* lift the requirement that shared references must be read-only!
-/// Use `UnsafeCell` for that.
-///
 /// This type blocks niches the same way `UnsafeCell` does.
 #[lang = "unsafe_pinned"]
 #[repr(transparent)]
 #[unstable(feature = "unsafe_pinned", issue = "125735")]
 pub struct UnsafePinned<T: ?Sized> {
-    value: T,
+    value: UnsafeCell<T>,
 }
 
+// Override the manual `!Sync` in `UnsafeCell`.
+#[unstable(feature = "unsafe_pinned", issue = "125735")]
+unsafe impl<T: ?Sized + Sync> Sync for UnsafePinned<T> {}
+
 /// When this type is used, that almost certainly means safe APIs need to use pinning to avoid the
 /// aliases from becoming invalidated. Therefore let's mark this as `!Unpin`. You can always opt
 /// back in to `Unpin` with an `impl` block, provided your API is still sound while unpinned.
 #[unstable(feature = "unsafe_pinned", issue = "125735")]
 impl<T: ?Sized> !Unpin for UnsafePinned<T> {}
 
-/// The type is `Copy` when `T` is to avoid people assuming that `Copy` implies there is no
-/// `UnsafePinned` anywhere. (This is an issue with `UnsafeCell`: people use `Copy` bounds to mean
-/// `Freeze`.) Given that there is no `unsafe impl Copy for ...`, this is also the option that
-/// leaves the user more choices (as they can always wrap this in a `!Copy` type).
-// FIXME(unsafe_pinned): this may be unsound or a footgun?
-#[unstable(feature = "unsafe_pinned", issue = "125735")]
-impl<T: Copy> Copy for UnsafePinned<T> {}
-
-#[unstable(feature = "unsafe_pinned", issue = "125735")]
-impl<T: Copy> Clone for UnsafePinned<T> {
-    fn clone(&self) -> Self {
-        *self
-    }
-}
-
 // `Send` and `Sync` are inherited from `T`. This is similar to `SyncUnsafeCell`, since
 // we eventually concluded that `UnsafeCell` implicitly making things `!Sync` is sometimes
 // unergonomic. A type that needs to be `!Send`/`!Sync` should really have an explicit
@@ -63,7 +52,7 @@ impl<T> UnsafePinned<T> {
     #[must_use]
     #[unstable(feature = "unsafe_pinned", issue = "125735")]
     pub const fn new(value: T) -> Self {
-        UnsafePinned { value }
+        UnsafePinned { value: UnsafeCell::new(value) }
     }
 
     /// Unwraps the value, consuming this `UnsafePinned`.
@@ -72,7 +61,7 @@ impl<T> UnsafePinned<T> {
     #[unstable(feature = "unsafe_pinned", issue = "125735")]
     #[rustc_allow_const_fn_unstable(const_precise_live_drops)]
     pub const fn into_inner(self) -> T {
-        self.value
+        self.value.into_inner()
     }
 }
 

src/bootstrap/src/core/build_steps/check.rs

@@ -77,6 +77,13 @@ impl Step for Std {
     }
 
     fn run(self, builder: &Builder<'_>) {
+        if !builder.download_rustc() && builder.config.skip_std_check_if_no_download_rustc {
+            eprintln!(
+                "WARNING: `--skip-std-check-if-no-download-rustc` flag was passed and `rust.download-rustc` is not available. Skipping."
+            );
+            return;
+        }
+
         builder.require_submodule("library/stdarch", None);
 
         let stage = self.custom_stage.unwrap_or(builder.top_stage);

src/bootstrap/src/core/build_steps/llvm.rs

@@ -443,23 +443,26 @@ impl Step for Llvm {
         // See https://github.com/rust-lang/rust/pull/50104
         cfg.define("LLVM_ENABLE_LIBXML2", "OFF");
 
-        if !enabled_llvm_projects.is_empty() {
-            enabled_llvm_projects.sort();
-            enabled_llvm_projects.dedup();
-            cfg.define("LLVM_ENABLE_PROJECTS", enabled_llvm_projects.join(";"));
-        }
-
         let mut enabled_llvm_runtimes = Vec::new();
 
         if helpers::forcing_clang_based_tests() {
             enabled_llvm_runtimes.push("compiler-rt");
         }
 
+        // This is an experimental flag, which likely builds more than necessary.
+        // We will optimize it when we get closer to releasing it on nightly.
         if builder.config.llvm_offload {
             enabled_llvm_runtimes.push("offload");
             //FIXME(ZuseZ4): LLVM intends to drop the offload dependency on openmp.
             //Remove this line once they achieved it.
             enabled_llvm_runtimes.push("openmp");
+            enabled_llvm_projects.push("compiler-rt");
+        }
+
+        if !enabled_llvm_projects.is_empty() {
+            enabled_llvm_projects.sort();
+            enabled_llvm_projects.dedup();
+            cfg.define("LLVM_ENABLE_PROJECTS", enabled_llvm_projects.join(";"));
         }
 
         if !enabled_llvm_runtimes.is_empty() {

src/bootstrap/src/core/config/config.rs

@@ -423,6 +423,11 @@ pub struct Config {
 
     /// Cache for determining path modifications
     pub path_modification_cache: Arc<Mutex<HashMap<Vec<&'static str>, PathFreshness>>>,
+
+    /// Skip checking the standard library if `rust.download-rustc` isn't available.
+    /// This is mostly for RA as building the stage1 compiler to check the library tree
+    /// on each code change might be too much for some computers.
+    pub skip_std_check_if_no_download_rustc: bool,
 }
 
 #[derive(Debug, Default, Clone, Copy, PartialEq, Eq)]
@@ -1507,6 +1512,7 @@ impl Config {
         config.enable_bolt_settings = flags.enable_bolt_settings;
         config.bypass_bootstrap_lock = flags.bypass_bootstrap_lock;
         config.is_running_on_ci = flags.ci.unwrap_or(CiEnv::is_ci());
+        config.skip_std_check_if_no_download_rustc = flags.skip_std_check_if_no_download_rustc;
 
         // Infer the rest of the configuration.
 

src/bootstrap/src/core/config/flags.rs

@@ -182,6 +182,11 @@ pub struct Flags {
     /// Make bootstrap to behave as it's running on the CI environment or not.
     #[arg(global = true, long, value_name = "bool")]
     pub ci: Option<bool>,
+    /// Skip checking the standard library if `rust.download-rustc` isn't available.
+    /// This is mostly for RA as building the stage1 compiler to check the library tree
+    /// on each code change might be too much for some computers.
+    #[arg(global = true, long)]
+    pub skip_std_check_if_no_download_rustc: bool,
 }
 
 impl Flags {

src/bootstrap/src/utils/change_tracker.rs

@@ -416,4 +416,9 @@ pub const CONFIG_CHANGE_HISTORY: &[ChangeInfo] = &[
         severity: ChangeSeverity::Warning,
         summary: "Stage0 library no longer matches the in-tree library, which means stage1 compiler now uses the beta library.",
     },
+    ChangeInfo {
+        change_id: 141970,
+        severity: ChangeSeverity::Info,
+        summary: "Added new bootstrap flag `--skip-std-check-if-no-download-rustc` that skips std checks when download-rustc is unavailable. Mainly intended for developers to reduce RA overhead.",
+    },
 ];

src/ci/docker/host-x86_64/dist-x86_64-linux/Dockerfile

@@ -96,14 +96,13 @@ ENV RUST_CONFIGURE_ARGS \
       --set rust.lto=thin \
       --set rust.codegen-units=1
 
-# Note that `rust.debug` is set to true *only* for `opt-dist`
-ENV SCRIPT python3 ../x.py build --set rust.debug=true opt-dist && \
-    ./build/$HOSTS/stage0-tools-bin/opt-dist linux-ci -- python3 ../x.py dist \
-    --host $HOSTS --target $HOSTS \
-    --include-default-paths \
-    build-manifest bootstrap && \
-    # Use GCC for building GCC, as it seems to behave badly when built with Clang
-    CC=/rustroot/bin/cc CXX=/rustroot/bin/c++ python3 ../x.py dist gcc
+ARG SCRIPT_ARG
+
+COPY host-x86_64/dist-x86_64-linux/dist.sh /scripts/
+COPY host-x86_64/dist-x86_64-linux/dist-alt.sh /scripts/
+
+ENV SCRIPT /scripts/${SCRIPT_ARG}
+
 ENV CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_LINKER=clang
 
 # This is the only builder which will create source tarballs

src/ci/docker/host-x86_64/dist-x86_64-linux/dist-alt.sh

@@ -0,0 +1,8 @@
+#!/bin/bash
+
+set -eux
+
+python3 ../x.py dist \
+    --host $HOSTS --target $HOSTS \
+    --include-default-paths \
+    build-manifest bootstrap

src/ci/docker/host-x86_64/dist-x86_64-linux/dist.sh

@@ -0,0 +1,13 @@
+#!/bin/bash
+
+set -eux
+
+python3 ../x.py build --set rust.debug=true opt-dist
+
+./build/$HOSTS/stage0-tools-bin/opt-dist linux-ci -- python3 ../x.py dist \
+    --host $HOSTS --target $HOSTS \
+    --include-default-paths \
+    build-manifest bootstrap
+
+# Use GCC for building GCC, as it seems to behave badly when built with Clang
+CC=/rustroot/bin/cc CXX=/rustroot/bin/c++ python3 ../x.py dist gcc

src/ci/github-actions/jobs.yml

@@ -10,6 +10,10 @@ runners:
     free_disk: true
     <<: *base-job
 
+  - &job-linux-4c-largedisk
+    os: ubuntu-24.04-4core-16gb
+    <<: *base-job
+
   - &job-linux-8c
     os: ubuntu-24.04-8core-32gb
     <<: *base-job
@@ -105,6 +109,15 @@ envs:
   pr:
     PR_CI_JOB: 1
 
+jobs:
+  dist-x86_64-linux: &job-dist-x86_64-linux
+    name: dist-x86_64-linux
+    env:
+      CODEGEN_BACKENDS: llvm,cranelift
+      DOCKER_SCRIPT: dist.sh
+    <<: *job-linux-36c-codebuild
+
+
 # Jobs that run on each push to a pull request (PR)
 # These jobs automatically inherit envs.pr, to avoid repeating
 # it in each job definition.
@@ -138,10 +151,7 @@ pr:
 # These jobs automatically inherit envs.try, to avoid repeating
 # it in each job definition.
 try:
-  - name: dist-x86_64-linux
-    env:
-      CODEGEN_BACKENDS: llvm,cranelift
-    <<: *job-linux-36c-codebuild
+  - <<: *job-dist-x86_64-linux
 
 # Main CI jobs that have to be green to merge a commit into master
 # These jobs automatically inherit envs.auto, to avoid repeating
@@ -234,16 +244,14 @@ auto:
   - name: dist-x86_64-illumos
     <<: *job-linux-4c
 
-  - name: dist-x86_64-linux
-    env:
-      CODEGEN_BACKENDS: llvm,cranelift
-    <<: *job-linux-36c-codebuild
+  - <<: *job-dist-x86_64-linux
 
   - name: dist-x86_64-linux-alt
     env:
       IMAGE: dist-x86_64-linux
       CODEGEN_BACKENDS: llvm,cranelift
-    <<: *job-linux-16c
+      DOCKER_SCRIPT: dist-alt.sh
+    <<: *job-linux-4c-largedisk
 
   - name: dist-x86_64-musl
     env:

src/doc/rustc-dev-guide/rust-version

@@ -1 +1 @@
-99e7c15e81385b38a8186b51edc4577d5d7b5bdd
+c68032fd4c442d275f4daa571ba19c076106b490

src/doc/rustc-dev-guide/src/autodiff/flags.md

@@ -16,7 +16,9 @@ LooseTypes // Risk incorrect derivatives instead of aborting when missing Type I
 ```
 
 <div class="warning">
+
 `LooseTypes` is often helpful to get rid of Enzyme errors stating `Can not deduce type of <X>` and to be able to run some code. But please keep in mind that this flag absolutely has the chance to cause incorrect gradients. Even worse, the gradients might be correct for certain input values, but not for others. So please create issues about such bugs and only use this flag temporarily while you wait for your bug to be fixed.
+
 </div>
 
 ### Benchmark flags

src/doc/rustc-dev-guide/src/building/suggested.md

@@ -59,6 +59,14 @@ always overrides the inner ones.
 
 ## Configuring `rust-analyzer` for `rustc`
 
+### Checking the "library" tree
+
+Checking the "library" tree requires a stage1 compiler, which can be a heavy process on some computers.
+For this reason, bootstrap has a flag called `--skip-std-check-if-no-download-rustc` that skips checking the
+"library" tree if `rust.download-rustc` isn't available. If you want to avoid putting a heavy load on your computer
+with `rust-analyzer`, you can add the `--skip-std-check-if-no-download-rustc` flag to your `./x check` command in
+the `rust-analyzer` configuration.
+
 ### Project-local rust-analyzer setup
 
 `rust-analyzer` can help you check and format your code whenever you save a

src/doc/rustc-dev-guide/src/coroutine-closures.md

@@ -1,16 +1,22 @@
+# Async closures/"coroutine-closures"
+
+<!-- toc -->
+
 Please read [RFC 3668](https://rust-lang.github.io/rfcs/3668-async-closures.html) to understand the general motivation of the feature. This is a very technical and somewhat "vertical" chapter; ideally we'd split this and sprinkle it across all the relevant chapters, but for the purposes of understanding async closures *holistically*, I've put this together all here in one chapter.
 
-# Coroutine-closures -- a technical deep dive
+## Coroutine-closures -- a technical deep dive
 
 Coroutine-closures are a generalization of async closures, being special syntax for closure expressions which return a coroutine, notably one that is allowed to capture from the closure's upvars.
 
 For now, the only usable kind of coroutine-closure is the async closure, and supporting async closures is the extent of this PR. We may eventually support `gen || {}`, etc., and most of the problems and curiosities described in this document apply to all coroutine-closures in general.
 
 As a consequence of the code being somewhat general, this document may flip between calling them "async closures" and "coroutine-closures". The future that is returned by the async closure will generally be called the "coroutine" or the "child coroutine".
 
-## HIR
+### HIR
 
-Async closures (and in the future, other coroutine flavors such as `gen`) are represented in HIR as a `hir::Closure` whose closure-kind is `ClosureKind::CoroutineClosure(_)`[^k1], which wraps an async block, which is also represented in HIR as a `hir::Closure`) and whose closure-kind is `ClosureKind::Closure(CoroutineKind::Desugared(_, CoroutineSource::Closure))`[^k2].
+Async closures (and in the future, other coroutine flavors such as `gen`) are represented in HIR as a `hir::Closure`.
+The closure-kind of the `hir::Closure` is `ClosureKind::CoroutineClosure(_)`[^k1], which wraps an async block, which is also represented in HIR as a `hir::Closure`.
+The closure-kind of the async block is `ClosureKind::Closure(CoroutineKind::Desugared(_, CoroutineSource::Closure))`[^k2].
 
 [^k1]: <https://github.com/rust-lang/rust/blob/5ca0e9fa9b2f92b463a0a2b0b34315e09c0b7236/compiler/rustc_ast_lowering/src/expr.rs#L1147>
 
@@ -24,7 +30,7 @@ Like `async fn`, when lowering an async closure's body, we need to unconditional
 
 [^l3]: <https://github.com/rust-lang/rust/blob/5ca0e9fa9b2f92b463a0a2b0b34315e09c0b7236/compiler/rustc_hir_typeck/src/upvar.rs#L250-L256>
 
-## `rustc_middle::ty` Representation
+### `rustc_middle::ty` Representation
 
 For the purposes of keeping the implementation mostly future-compatible (i.e. with gen `|| {}` and `async gen || {}`), most of this section calls async closures "coroutine-closures".
 
@@ -72,7 +78,7 @@ To most easily construct the `Coroutine` that a coroutine-closure returns, you c
 
 Most of the args to that function will be components that you can get out of the `CoroutineArgs`, except for the `goal_kind: ClosureKind` which controls which flavor of coroutine to return based off of the `ClosureKind` passed in -- i.e. it will prepare the by-ref coroutine if `ClosureKind::Fn | ClosureKind::FnMut`, and the by-move coroutine if `ClosureKind::FnOnce`.
 
-## Trait Hierarchy
+### Trait Hierarchy
 
 We introduce a parallel hierarchy of `Fn*` traits that are implemented for . The motivation for the introduction was covered in a blog post: [Async Closures](https://hackmd.io/@compiler-errors/async-closures).
 
@@ -98,11 +104,11 @@ We mention above that "regular" callable types can implement `AsyncFn*`, but the
 
 See the "follow-up: when do..." section below for an elaborated answer. The full answer describes a pretty interesting and hopefully thorough heuristic that is used to ensure that most async closures "just work".
 
-## Tale of two bodies...
+### Tale of two bodies...
 
 When async closures are called with `AsyncFn`/`AsyncFnMut`, they return a coroutine that borrows from the closure. However, when they are called via `AsyncFnOnce`, we consume that closure, and cannot return a coroutine that borrows from data that is now dropped.
 
-To work around around this limitation, we synthesize a separate by-move MIR body for calling `AsyncFnOnce::call_once` on a coroutine-closure that can be called by-ref.
+To work around this limitation, we synthesize a separate by-move MIR body for calling `AsyncFnOnce::call_once` on a coroutine-closure that can be called by-ref.
 
 This body operates identically to the "normal" coroutine returned from calling the coroutine-closure, except for the fact that it has a different set of upvars, since we must *move* the captures from the parent coroutine-closure into the child coroutine.
 
@@ -120,7 +126,7 @@ Since we've synthesized a new def id, this query is also responsible for feeding
 
 [^b3]: <https://github.com/rust-lang/rust/blob/5ca0e9fa9b2f92b463a0a2b0b34315e09c0b7236/compiler/rustc_mir_transform/src/lib.rs#L339-L342>
 
-## Closure signature inference
+### Closure signature inference
 
 The closure signature inference algorithm for async closures is a bit more complicated than the inference algorithm for "traditional" closures. Like closures, we iterate through all of the clauses that may be relevant (for the expectation type passed in)[^deduce1].
 
@@ -173,7 +179,7 @@ s.as_bytes();
 
 So *instead*, we use this alias (in this case, a projection: `AsyncFnKindHelper::Upvars<'env, ...>`) to delay the computation of the *tupled upvars* and give us something to put in its place, while still allowing us to return a `TyKind::Coroutine` (which is a rigid type) and we may successfully confirm the built-in traits we need (in our case, `Future`), since the `Future` implementation doesn't depend on the upvars at all.
 
-## Upvar analysis
+### Upvar analysis
 
 By and large, the upvar analysis for coroutine-closures and their child coroutines proceeds like normal upvar analysis. However, there are several interesting bits that happen to account for async closures' special natures:
 
@@ -262,7 +268,7 @@ let c = async || {
 
 If either of these cases apply, then we should capture the borrow with the lifetime of the parent coroutine-closure's env. Luckily, if this function is not correct, then the program is not unsound, since we still borrowck and validate the choices made from this function -- the only side-effect is that the user may receive unnecessary borrowck errors.
 
-## Instance resolution
+### Instance resolution
 
 If a coroutine-closure has a closure-kind of `FnOnce`, then its `AsyncFnOnce::call_once` and `FnOnce::call_once` implementations resolve to the coroutine-closure's body[^res1], and the `Future::poll` of the coroutine that gets returned resolves to the body of the child closure.
 
@@ -282,7 +288,7 @@ This is represented by the `ConstructCoroutineInClosureShim`[^i1]. The `receiver
 
 [^i3]: <https://github.com/rust-lang/rust/blob/07cbbdd69363da97075650e9be24b78af0bcdd23/compiler/rustc_middle/src/ty/instance.rs#L841>
 
-## Borrow-checking
+### Borrow-checking
 
 It turns out that borrow-checking async closures is pretty straightforward. After adding a new `DefiningTy::CoroutineClosure`[^bck1] variant, and teaching borrowck how to generate the signature of the coroutine-closure[^bck2], borrowck proceeds totally fine.