Auto merge of rust-lang#126193 - RalfJung:miri-sync, r=RalfJung

Miri subtree update

r? `@ghost`

Author: bors

Diff for: src/tools/miri/CONTRIBUTING.md

@@ -223,15 +223,15 @@ will eventually sync those changes back into this repository.
 When working on Miri in the rustc tree, here's how you can run tests:
 
 ```
-./x.py test miri --stage 0
+./x.py test miri
 ```
 
 `--bless` will work, too.
 
 You can also directly run Miri on a Rust source file:
 
 ```
-./x.py run miri --stage 0 --args src/tools/miri/tests/pass/hello.rs
+./x.py run miri --stage 1 --args src/tools/miri/tests/pass/hello.rs
 ```
 
 ## Advanced topic: Syncing with the rustc repo
@@ -287,7 +287,22 @@ https. Add the following to your `.gitconfig`:
     pushInsteadOf = https://github.com/
 ```
 
-## Internal environment variables
+## Further environment variables
+
+The following environment variables are relevant to `./miri`:
+
+* `MIRI_AUTO_OPS` indicates whether the automatic execution of rustfmt, clippy and toolchain setup
+  (as controlled by the `./auto-*` files) should be skipped. If it is set to `no`, they are skipped.
+  This is used to allow automated IDE actions to avoid the auto ops.
+* `MIRI_LOG`, `MIRI_BACKTRACE` control logging and backtrace printing during Miri executions.
+* `MIRI_TEST_THREADS` (recognized by `./miri test`) sets the number of threads to use for running
+  tests. By default, the number of cores is used.
+* `MIRI_SKIP_UI_CHECKS` (recognized by `./miri test`) disables checking that the `stderr` or
+  `stdout` files match the actual output.
+
+Furthermore, the usual environment variables recognized by `cargo miri` also work for `./miri`, e.g.
+`MIRI_LIB_SRC`. Note that `MIRIFLAGS` is ignored by `./miri test` as each test controls the flags it
+is run with.
 
 The following environment variables are *internal* and must not be used by
 anyone but Miri itself. They are used to communicate between different Miri

Diff for: src/tools/miri/README.md

@@ -448,28 +448,19 @@ Some native rustc `-Z` flags are also very relevant for Miri:
 * `-Zmir-emit-retag` controls whether `Retag` statements are emitted. Miri
   enables this per default because it is needed for [Stacked Borrows] and [Tree Borrows].
 
-Moreover, Miri recognizes some environment variables (unless noted otherwise, these are supported
-by all intended entry points, i.e. `cargo miri` and `./miri {test,run}`):
-
-* `MIRI_AUTO_OPS` indicates whether the automatic execution of rustfmt, clippy and toolchain setup
-  should be skipped. If it is set to `no`, they are skipped. This is used to allow automated IDE
-  actions to avoid the auto ops.
-* `MIRI_LOG`, `MIRI_BACKTRACE` control logging and backtrace printing during
-  Miri executions, also [see "Testing the Miri driver" in `CONTRIBUTING.md`][testing-miri].
+Moreover, Miri recognizes some environment variables:
+
 * `MIRIFLAGS` defines extra flags to be passed to Miri.
 * `MIRI_LIB_SRC` defines the directory where Miri expects the sources of the standard library that
   it will build and use for interpretation. This directory must point to the `library` subdirectory
   of a `rust-lang/rust` repository checkout.
-* `MIRI_SYSROOT` indicates the sysroot to use. When using `cargo miri`, this skips the automatic
+* `MIRI_SYSROOT` indicates the sysroot to use. When using `cargo miri test`/`cargo miri run`, this skips the automatic
   setup -- only set this if you do not want to use the automatically created sysroot. When invoking
   `cargo miri setup`, this indicates where the sysroot will be put.
-* `MIRI_TEST_THREADS` (recognized by `./miri test`): set the number of threads to use for running tests.
-  By default, the number of cores is used.
 * `MIRI_NO_STD` makes sure that the target's sysroot is built without libstd. This allows testing
-  and running no_std programs. (Miri has a heuristic to detect no-std targets based on the target
-  name; this environment variable is only needed when that heuristic fails.)
-* `MIRI_SKIP_UI_CHECKS` (recognized by `./miri test`): don't check whether the
-  `stderr` or `stdout` files match the actual output.
+  and running no_std programs. This should *not usually be used*; Miri has a heuristic to detect
+  no-std targets based on the target name. Setting this on a target that does support libstd can
+  lead to confusing results.
 
 [testing-miri]: CONTRIBUTING.md#testing-the-miri-driver
 

Diff for: src/tools/miri/bench-cargo-miri/big-allocs/Cargo.lock

@@ -0,0 +1,7 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "big-allocs"
+version = "0.1.0"

Diff for: src/tools/miri/bench-cargo-miri/big-allocs/Cargo.toml

@@ -0,0 +1,8 @@
+[package]
+name = "big-allocs"
+version = "0.1.0"
+edition = "2021"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]

Diff for: src/tools/miri/bench-cargo-miri/big-allocs/src/main.rs

@@ -0,0 +1,13 @@
+//! This is a regression test for https://github.com/rust-lang/miri/issues/3637.
+//! `Allocation`s are backed by a `Box<[u8]>`, which we create using `alloc_zeroed`, which should
+//! make very large allocations cheap. But then we also need to not clone those `Allocation`s, or
+//! we end up slow anyway.
+
+fn main() {
+    // We can't use too big of an allocation or this code will encounter an allocation failure in
+    // CI. Since the allocation can't be huge, we need to do a few iterations so that the effect
+    // we're trying to measure is clearly visible above the interpreter's startup time.
+    for _ in 0..10 {
+        drop(Vec::<u8>::with_capacity(512 * 1024 * 1024));
+    }
+}

Diff for: src/tools/miri/cargo-miri/src/main.rs

@@ -1,15 +1,14 @@
 #![allow(clippy::useless_format, clippy::derive_partial_eq_without_eq, rustc::internal)]
 
-#[macro_use]
-mod util;
-
 mod arg;
 mod phases;
 mod setup;
+mod util;
 
 use std::{env, iter};
 
 use crate::phases::*;
+use crate::util::show_error;
 
 /// Returns `true` if our flags look like they may be for rustdoc, i.e., this is cargo calling us to
 /// be rustdoc. It's hard to be sure as cargo does not have a RUSTDOC_WRAPPER or an env var that

Diff for: src/tools/miri/cargo-miri/src/util.rs

@@ -1,3 +1,4 @@
+use std::collections::HashMap;
 use std::env;
 use std::ffi::OsString;
 use std::fs::File;
@@ -11,14 +12,15 @@ use serde::{Deserialize, Serialize};
 
 pub use crate::arg::*;
 
-pub fn show_error(msg: &impl std::fmt::Display) -> ! {
+pub fn show_error_(msg: &impl std::fmt::Display) -> ! {
     eprintln!("fatal error: {msg}");
     std::process::exit(1)
 }
 
 macro_rules! show_error {
-    ($($tt:tt)*) => { crate::util::show_error(&format_args!($($tt)*)) };
+    ($($tt:tt)*) => { crate::util::show_error_(&format_args!($($tt)*)) };
 }
+pub(crate) use show_error;
 
 /// The information to run a crate with the given environment.
 #[derive(Clone, Serialize, Deserialize)]
@@ -232,21 +234,18 @@ pub fn get_cargo_metadata() -> Metadata {
 }
 
 /// Pulls all the crates in this workspace from the cargo metadata.
-/// Workspace members are emitted like "miri 0.1.0 (path+file:///path/to/miri)"
 /// Additionally, somewhere between cargo metadata and TyCtxt, '-' gets replaced with '_' so we
 /// make that same transformation here.
 pub fn local_crates(metadata: &Metadata) -> String {
     assert!(!metadata.workspace_members.is_empty());
-    let mut local_crates = String::new();
-    for member in &metadata.workspace_members {
-        let name = member.repr.split(' ').next().unwrap();
-        let name = name.replace('-', "_");
-        local_crates.push_str(&name);
-        local_crates.push(',');
-    }
-    local_crates.pop(); // Remove the trailing ','
-
-    local_crates
+    let package_name_by_id: HashMap<_, _> =
+        metadata.packages.iter().map(|package| (&package.id, package.name.as_str())).collect();
+    metadata
+        .workspace_members
+        .iter()
+        .map(|id| package_name_by_id[id].replace('-', "_"))
+        .collect::<Vec<_>>()
+        .join(",")
 }
 
 /// Debug-print a command that is going to be run.

Diff for: src/tools/miri/ci/ci.sh

@@ -148,8 +148,8 @@ case $HOST_TARGET in
     UNIX="panic/panic panic/unwind concurrency/simple atomic libc-mem libc-misc libc-random env num_cpus" # the things that are very similar across all Unixes, and hence easily supported there
     TEST_TARGET=x86_64-unknown-freebsd run_tests_minimal $BASIC $UNIX threadname libc-time fs
     TEST_TARGET=i686-unknown-freebsd   run_tests_minimal $BASIC $UNIX threadname libc-time fs
-    TEST_TARGET=x86_64-unknown-illumos run_tests_minimal $BASIC $UNIX threadname pthread-sync libc-time
-    TEST_TARGET=x86_64-pc-solaris      run_tests_minimal $BASIC $UNIX threadname pthread-sync libc-time
+    TEST_TARGET=x86_64-unknown-illumos run_tests_minimal $BASIC $UNIX threadname pthread-sync available-parallelism libc-time
+    TEST_TARGET=x86_64-pc-solaris      run_tests_minimal $BASIC $UNIX threadname pthread-sync available-parallelism libc-time
     TEST_TARGET=aarch64-linux-android  run_tests_minimal $BASIC $UNIX
     TEST_TARGET=wasm32-wasip2          run_tests_minimal empty_main wasm heap_alloc libc-mem
     TEST_TARGET=wasm32-unknown-unknown run_tests_minimal empty_main wasm

Diff for: src/tools/miri/rust-version

@@ -1 +1 @@
-21e6de7eb64c09102de3f100420a09edc1a2a8d7
+565cadb514d35e7b851540edbc172af0f606014f

Diff for: src/tools/miri/src/alloc_addresses/mod.rs

@@ -257,7 +257,7 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
         Ok(())
     }
 
-    fn ptr_from_addr_cast(&self, addr: u64) -> InterpResult<'tcx, Pointer<Option<Provenance>>> {
+    fn ptr_from_addr_cast(&self, addr: u64) -> InterpResult<'tcx, Pointer> {
         trace!("Casting {:#x} to a pointer", addr);
 
         let ecx = self.eval_context_ref();
@@ -297,10 +297,10 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
     /// Convert a relative (tcx) pointer to a Miri pointer.
     fn adjust_alloc_root_pointer(
         &self,
-        ptr: Pointer<CtfeProvenance>,
+        ptr: interpret::Pointer<CtfeProvenance>,
         tag: BorTag,
         kind: MemoryKind,
-    ) -> InterpResult<'tcx, Pointer<Provenance>> {
+    ) -> InterpResult<'tcx, interpret::Pointer<Provenance>> {
         let ecx = self.eval_context_ref();
 
         let (prov, offset) = ptr.into_parts(); // offset is relative (AllocId provenance)
@@ -310,12 +310,15 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
         // Add offset with the right kind of pointer-overflowing arithmetic.
         let dl = ecx.data_layout();
         let absolute_addr = dl.overflowing_offset(base_addr, offset.bytes()).0;
-        Ok(Pointer::new(Provenance::Concrete { alloc_id, tag }, Size::from_bytes(absolute_addr)))
+        Ok(interpret::Pointer::new(
+            Provenance::Concrete { alloc_id, tag },
+            Size::from_bytes(absolute_addr),
+        ))
     }
 
     /// When a pointer is used for a memory access, this computes where in which allocation the
     /// access is going.
-    fn ptr_get_alloc(&self, ptr: Pointer<Provenance>) -> Option<(AllocId, Size)> {
+    fn ptr_get_alloc(&self, ptr: interpret::Pointer<Provenance>) -> Option<(AllocId, Size)> {
         let ecx = self.eval_context_ref();
 
         let (tag, addr) = ptr.into_parts(); // addr is absolute (Tag provenance)

Diff for: src/tools/miri/src/bin/miri.rs

@@ -8,7 +8,6 @@
 )]
 
 // Some "regular" crates we want to share with rustc
-#[macro_use]
 extern crate tracing;
 
 // The rustc crates we need
@@ -26,6 +25,8 @@ use std::num::NonZero;
 use std::path::PathBuf;
 use std::str::FromStr;
 
+use tracing::debug;
+
 use rustc_data_structures::sync::Lrc;
 use rustc_driver::Compilation;
 use rustc_hir::{self as hir, Node};

Diff for: src/tools/miri/src/borrow_tracker/mod.rs

@@ -281,8 +281,8 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
     fn retag_ptr_value(
         &mut self,
         kind: RetagKind,
-        val: &ImmTy<'tcx, Provenance>,
-    ) -> InterpResult<'tcx, ImmTy<'tcx, Provenance>> {
+        val: &ImmTy<'tcx>,
+    ) -> InterpResult<'tcx, ImmTy<'tcx>> {
         let this = self.eval_context_mut();
         let method = this.machine.borrow_tracker.as_ref().unwrap().borrow().borrow_tracker_method;
         match method {
@@ -294,7 +294,7 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
     fn retag_place_contents(
         &mut self,
         kind: RetagKind,
-        place: &PlaceTy<'tcx, Provenance>,
+        place: &PlaceTy<'tcx>,
     ) -> InterpResult<'tcx> {
         let this = self.eval_context_mut();
         let method = this.machine.borrow_tracker.as_ref().unwrap().borrow().borrow_tracker_method;
@@ -304,10 +304,7 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
         }
     }
 
-    fn protect_place(
-        &mut self,
-        place: &MPlaceTy<'tcx, Provenance>,
-    ) -> InterpResult<'tcx, MPlaceTy<'tcx, Provenance>> {
+    fn protect_place(&mut self, place: &MPlaceTy<'tcx>) -> InterpResult<'tcx, MPlaceTy<'tcx>> {
         let this = self.eval_context_mut();
         let method = this.machine.borrow_tracker.as_ref().unwrap().borrow().borrow_tracker_method;
         match method {
@@ -327,7 +324,7 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
 
     fn give_pointer_debug_name(
         &mut self,
-        ptr: Pointer<Option<Provenance>>,
+        ptr: Pointer,
         nth_parent: u8,
         name: &str,
     ) -> InterpResult<'tcx> {

Diff for: src/tools/miri/src/borrow_tracker/stacked_borrows/diagnostics.rs

@@ -149,11 +149,7 @@ impl<'ecx, 'tcx> DiagnosticCxBuilder<'ecx, 'tcx> {
         DiagnosticCxBuilder { machine, operation }
     }
 
-    pub fn read(
-        machine: &'ecx MiriMachine<'tcx>,
-        tag: ProvenanceExtra,
-        range: AllocRange,
-    ) -> Self {
+    pub fn read(machine: &'ecx MiriMachine<'tcx>, tag: ProvenanceExtra, range: AllocRange) -> Self {
         let operation = Operation::Access(AccessOp { kind: AccessKind::Read, tag, range });
         DiagnosticCxBuilder { machine, operation }
     }