Skip to content

Commit 3c511bb

Browse files
committed
Auto merge of #3067 - Vanille-N:spurious-incremental, r=RalfJung
Continuation of #3054: enable spurious reads in TB The last additions to the test suite of TB left some unresolved `#[should_panic]` that these new modifications solve. ## Problem Recall that the issues were arising from the interleavings that follow. ### A. `Reserved -> Frozen` has visible effects after function exit The transition `Reserved -> Frozen` irreversibly blocks write accesses to the tag, so in the interleaving below `y` initially `Reserved` becomes `Frozen` only in the target where a spurious read through `x` is inserted. This makes the later write through `y` UB only in the target and not in the source. ``` 1: retag x (&, protect) 2: retag y (&mut, protect) 1: spurious read x 1: ret x 2: ret y 2: write y ``` ### B. Protectors only announce their presence on retag There is a read-on-reborrow for protected locations, but if the retag of `x` occurs before that of `y` and there is no explicit access through `x`, then `y` is unaware of the existence of `x`. This is problematic because a spurious read inserted through `x` between the retag of `y` and the return of the function protecting `x` is a noalias violation in the target without UB in the source. ``` 1: retag x (&, protect) 2: retag y (&mut, protect) 1: spurious read x 1: ret x 2: write y 2: ret y ``` ## Step 1: Finer behavior for `Reserved` Since one problem is that `Reserved -> Frozen` has consequences beyond function exit, we decide to remove this transition entirely. To replace it we introduce a new subtype of `Reserved` with the extra boolean `aliased` set. `Reserved { aliased: true }` forbids child accesses, but only temporarily: it has no effect on activation once the tag is no longer protected. This makes the semantics of Tree Borrows slightly weaker in favor of being more similar to noalias. This solves interleaving **A.**, but **B.** is still a problem and the exhaustive tests do not pass yet. ## Step 2: Read on function exit Protected tags issue a "reminder" that they are protected until this instant inclusive, in the form of an implicit read (symmetrically to the implicit read on retag). This ensures that if the periods on which two tags `x` and `y` are protected overlap then no matter the interleaving of retags and returns, there is either a protector currently active or a read that has been emitted, both of which temporarily block activation. This makes the exhaustive test designed previously pass, but it has an effect on the ability to return an activated pointer that I had not foreseen before implementing it. ## Step 2': Do not propagate to children A naive implementation of **Step 2** makes the following code UB: ```rs fn reborrow(x: &mut u8) -> &mut u8 { let y = &mut *x; *y = *y; y // callee returns `y: Active`... } let x = &mut 0u8; let y = reborrow(x); // ... and caller receives `y: Frozen` *y = 1; // UB ``` This is unacceptable, and a simple fix is to make this implicit read visible only to foreign tags. We still lack hindsight on the ramifications of this decision, and the fact that the problematic pattern was only discovered because it occured in one completely unrelated test (with a cryptic error message) is worrying. We should be vigilant as to how this interacts with the rest of the model. ## TODO As of commit #281c30, the data race model has not been fully updated. We have removed the reborrow of mutable references counting as a write access, but we still need the implicit read of function exit to count as a read.
2 parents 4587c7c + bf1356e commit 3c511bb

37 files changed

+1097
-481
lines changed

src/tools/miri/src/borrow_tracker/mod.rs

+18-2
Original file line numberDiff line numberDiff line change
@@ -66,10 +66,13 @@ pub struct FrameState {
6666
/// `stacked_borrows::GlobalState` upon function return, and if we attempt to pop a protected
6767
/// tag, to identify which call is responsible for protecting the tag.
6868
/// See `Stack::item_popped` for more explanation.
69+
/// Tree Borrows also needs to know which allocation these tags
70+
/// belong to so that it can perform a read through them immediately before
71+
/// the frame gets popped.
6972
///
7073
/// This will contain one tag per reference passed to the function, so
7174
/// a size of 2 is enough for the vast majority of functions.
72-
pub protected_tags: SmallVec<[BorTag; 2]>,
75+
pub protected_tags: SmallVec<[(AllocId, BorTag); 2]>,
7376
}
7477

7578
impl VisitTags for FrameState {
@@ -208,7 +211,7 @@ impl GlobalStateInner {
208211
}
209212

210213
pub fn end_call(&mut self, frame: &machine::FrameExtra<'_>) {
211-
for tag in &frame
214+
for (_, tag) in &frame
212215
.borrow_tracker
213216
.as_ref()
214217
.expect("we should have borrow tracking data")
@@ -453,6 +456,19 @@ impl AllocState {
453456
AllocState::TreeBorrows(tb) => tb.borrow_mut().remove_unreachable_tags(tags),
454457
}
455458
}
459+
460+
/// Tree Borrows needs to be told when a tag stops being protected.
461+
pub fn release_protector<'tcx>(
462+
&self,
463+
machine: &MiriMachine<'_, 'tcx>,
464+
global: &GlobalState,
465+
tag: BorTag,
466+
) -> InterpResult<'tcx> {
467+
match self {
468+
AllocState::StackedBorrows(_sb) => Ok(()),
469+
AllocState::TreeBorrows(tb) => tb.borrow_mut().release_protector(machine, global, tag),
470+
}
471+
}
456472
}
457473

458474
impl VisitTags for AllocState {

src/tools/miri/src/borrow_tracker/stacked_borrows/diagnostics.rs

+2-1
Original file line numberDiff line numberDiff line change
@@ -436,6 +436,7 @@ impl<'history, 'ecx, 'mir, 'tcx> DiagnosticCx<'history, 'ecx, 'mir, 'tcx> {
436436
ProtectorKind::WeakProtector => "weakly protected",
437437
ProtectorKind::StrongProtector => "strongly protected",
438438
};
439+
let item_tag = item.tag();
439440
let call_id = self
440441
.machine
441442
.threads
@@ -444,7 +445,7 @@ impl<'history, 'ecx, 'mir, 'tcx> DiagnosticCx<'history, 'ecx, 'mir, 'tcx> {
444445
.map(|frame| {
445446
frame.extra.borrow_tracker.as_ref().expect("we should have borrow tracking data")
446447
})
447-
.find(|frame| frame.protected_tags.contains(&item.tag()))
448+
.find(|frame| frame.protected_tags.iter().any(|(_, tag)| tag == &item_tag))
448449
.map(|frame| frame.call_id)
449450
.unwrap(); // FIXME: Surely we should find something, but a panic seems wrong here?
450451
match self.operation {

src/tools/miri/src/borrow_tracker/stacked_borrows/mod.rs

+7-1
Original file line numberDiff line numberDiff line change
@@ -719,7 +719,13 @@ trait EvalContextPrivExt<'mir: 'ecx, 'tcx: 'mir, 'ecx>: crate::MiriInterpCxExt<'
719719

720720
if let Some(protect) = new_perm.protector() {
721721
// See comment in `Stack::item_invalidated` for why we store the tag twice.
722-
this.frame_mut().extra.borrow_tracker.as_mut().unwrap().protected_tags.push(new_tag);
722+
this.frame_mut()
723+
.extra
724+
.borrow_tracker
725+
.as_mut()
726+
.unwrap()
727+
.protected_tags
728+
.push((alloc_id, new_tag));
723729
this.machine
724730
.borrow_tracker
725731
.as_mut()

src/tools/miri/src/borrow_tracker/tree_borrows/diagnostics.rs

+18-3
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ pub enum AccessCause {
1919
Explicit(AccessKind),
2020
Reborrow,
2121
Dealloc,
22+
FnExit,
2223
}
2324

2425
impl fmt::Display for AccessCause {
@@ -27,6 +28,7 @@ impl fmt::Display for AccessCause {
2728
Self::Explicit(kind) => write!(f, "{kind}"),
2829
Self::Reborrow => write!(f, "reborrow"),
2930
Self::Dealloc => write!(f, "deallocation"),
31+
Self::FnExit => write!(f, "protector release"),
3032
}
3133
}
3234
}
@@ -38,6 +40,7 @@ impl AccessCause {
3840
Self::Explicit(kind) => format!("{rel} {kind}"),
3941
Self::Reborrow => format!("reborrow (acting as a {rel} read access)"),
4042
Self::Dealloc => format!("deallocation (acting as a {rel} write access)"),
43+
Self::FnExit => format!("protector release (acting as a {rel} read access)"),
4144
}
4245
}
4346
}
@@ -52,7 +55,9 @@ pub struct Event {
5255
/// Relative position of the tag to the one used for the access.
5356
pub is_foreign: bool,
5457
/// User-visible range of the access.
55-
pub access_range: AllocRange,
58+
/// `None` means that this is an implicit access to the entire allocation
59+
/// (used for the implicit read on protector release).
60+
pub access_range: Option<AllocRange>,
5661
/// The transition recorded by this event only occured on a subrange of
5762
/// `access_range`: a single access on `access_range` triggers several events,
5863
/// each with their own mutually disjoint `transition_range`. No-op transitions
@@ -123,7 +128,17 @@ impl HistoryData {
123128
// NOTE: `transition_range` is explicitly absent from the error message, it has no significance
124129
// to the user. The meaningful one is `access_range`.
125130
let access = access_cause.print_as_access(is_foreign);
126-
self.events.push((Some(span.data()), format!("{this} later transitioned to {endpoint} due to a {access} at offsets {access_range:?}", endpoint = transition.endpoint())));
131+
let access_range_text = match access_range {
132+
Some(r) => format!("at offsets {r:?}"),
133+
None => format!("on every location previously accessed by this tag"),
134+
};
135+
self.events.push((
136+
Some(span.data()),
137+
format!(
138+
"{this} later transitioned to {endpoint} due to a {access} {access_range_text}",
139+
endpoint = transition.endpoint()
140+
),
141+
));
127142
self.events
128143
.push((None, format!("this transition corresponds to {}", transition.summary())));
129144
}
@@ -745,7 +760,7 @@ const DEFAULT_FORMATTER: DisplayFmt = DisplayFmt {
745760
bot: '─',
746761
warning_text: "Warning: this tree is indicative only. Some tags may have been hidden.",
747762
},
748-
perm: DisplayFmtPermission { open: "|", sep: "|", close: "|", uninit: "---", range_sep: ".." },
763+
perm: DisplayFmtPermission { open: "|", sep: "|", close: "|", uninit: "----", range_sep: ".." },
749764
padding: DisplayFmtPadding {
750765
join_middle: "├",
751766
join_last: "└",

src/tools/miri/src/borrow_tracker/tree_borrows/mod.rs

+37-20
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,9 @@ use log::trace;
22

33
use rustc_target::abi::{Abi, Align, Size};
44

5-
use crate::borrow_tracker::{AccessKind, GlobalStateInner, ProtectorKind, RetagFields};
5+
use crate::borrow_tracker::{
6+
AccessKind, GlobalState, GlobalStateInner, ProtectorKind, RetagFields,
7+
};
68
use rustc_middle::{
79
mir::{Mutability, RetagKind},
810
ty::{
@@ -70,7 +72,7 @@ impl<'tcx> Tree {
7072
self.perform_access(
7173
access_kind,
7274
tag,
73-
range,
75+
Some(range),
7476
global,
7577
span,
7678
diagnostics::AccessCause::Explicit(access_kind),
@@ -99,6 +101,29 @@ impl<'tcx> Tree {
99101
pub fn expose_tag(&mut self, _tag: BorTag) {
100102
// TODO
101103
}
104+
105+
/// A tag just lost its protector.
106+
///
107+
/// This emits a special kind of access that is only applied
108+
/// to initialized locations, as a protection against other
109+
/// tags not having been made aware of the existence of this
110+
/// protector.
111+
pub fn release_protector(
112+
&mut self,
113+
machine: &MiriMachine<'_, 'tcx>,
114+
global: &GlobalState,
115+
tag: BorTag,
116+
) -> InterpResult<'tcx> {
117+
let span = machine.current_span();
118+
self.perform_access(
119+
AccessKind::Read,
120+
tag,
121+
None, // no specified range because it occurs on the entire allocation
122+
global,
123+
span,
124+
diagnostics::AccessCause::FnExit,
125+
)
126+
}
102127
}
103128

104129
/// Policy for a new borrow.
@@ -248,7 +273,13 @@ trait EvalContextPrivExt<'mir: 'ecx, 'tcx: 'mir, 'ecx>: crate::MiriInterpCxExt<'
248273
// We register the protection in two different places.
249274
// This makes creating a protector slower, but checking whether a tag
250275
// is protected faster.
251-
this.frame_mut().extra.borrow_tracker.as_mut().unwrap().protected_tags.push(new_tag);
276+
this.frame_mut()
277+
.extra
278+
.borrow_tracker
279+
.as_mut()
280+
.unwrap()
281+
.protected_tags
282+
.push((alloc_id, new_tag));
252283
this.machine
253284
.borrow_tracker
254285
.as_mut()
@@ -275,7 +306,7 @@ trait EvalContextPrivExt<'mir: 'ecx, 'tcx: 'mir, 'ecx>: crate::MiriInterpCxExt<'
275306
tree_borrows.perform_access(
276307
AccessKind::Read,
277308
orig_tag,
278-
range,
309+
Some(range),
279310
this.machine.borrow_tracker.as_ref().unwrap(),
280311
this.machine.current_span(),
281312
diagnostics::AccessCause::Reborrow,
@@ -287,21 +318,7 @@ trait EvalContextPrivExt<'mir: 'ecx, 'tcx: 'mir, 'ecx>: crate::MiriInterpCxExt<'
287318
// Also inform the data race model (but only if any bytes are actually affected).
288319
if range.size.bytes() > 0 {
289320
if let Some(data_race) = alloc_extra.data_race.as_ref() {
290-
// We sometimes need to make it a write, since not all retags commute with reads!
291-
// FIXME: Is that truly the semantics we want? Some optimizations are likely to be
292-
// very unhappy without this. We'd tsill ge some UB just by picking a suitable
293-
// interleaving, but wether UB happens can depend on whether a write occurs in the
294-
// future...
295-
let is_write = new_perm.initial_state.is_active()
296-
|| (new_perm.initial_state.is_reserved(None) && new_perm.protector.is_some());
297-
if is_write {
298-
// Need to get mutable access to alloc_extra.
299-
// (Cannot always do this as we can do read-only reborrowing on read-only allocations.)
300-
let (alloc_extra, machine) = this.get_alloc_extra_mut(alloc_id)?;
301-
alloc_extra.data_race.as_mut().unwrap().write(alloc_id, range, machine)?;
302-
} else {
303-
data_race.read(alloc_id, range, &this.machine)?;
304-
}
321+
data_race.read(alloc_id, range, &this.machine)?;
305322
}
306323
}
307324

@@ -532,7 +549,7 @@ pub trait EvalContextExt<'mir, 'tcx: 'mir>: crate::MiriInterpCxExt<'mir, 'tcx> {
532549
// if converting this alloc_id from a global to a local one
533550
// uncovers a non-supported `extern static`.
534551
let alloc_extra = this.get_alloc_extra(alloc_id)?;
535-
trace!("Stacked Borrows tag {tag:?} exposed in {alloc_id:?}");
552+
trace!("Tree Borrows tag {tag:?} exposed in {alloc_id:?}");
536553
alloc_extra.borrow_tracker_tb().borrow_mut().expose_tag(tag);
537554
}
538555
AllocKind::Function | AllocKind::VTable | AllocKind::Dead => {

0 commit comments

Comments
 (0)