Add mdbook-toc, markers, and documentation (#1028)

* Add mdbook-toc to travis, book.toml  and documentation

* Add toc markers

* Whitespace cleanup and some punctuation

* Addressed comments

Author: igaray

.travis.yml

@@ -13,6 +13,7 @@ install:
 - source ~/.cargo/env || true
 - cargo install mdbook --version '^0.4.5'
 - cargo install mdbook-linkcheck --version '^0.7.2'
+- cargo install mdbook-toc --version '^0.6.1'
 script:
 - git checkout -b ci
 - git rebase origin/master

README.md

@@ -40,7 +40,7 @@ rustdocs][rustdocs].
 To build a local static HTML site, install [`mdbook`](https://github.com/rust-lang/mdBook) with:
 
 ```
-> cargo install mdbook mdbook-linkcheck
+> cargo install mdbook mdbook-linkcheck mdbook-toc
 ```
 
 and execute the following command in the root of the repository:
@@ -56,6 +56,11 @@ The build files are found in the `book` directory.
 We use `mdbook-linkcheck` to validate URLs included in our documentation.
 `linkcheck` will be run automatically when you build with the instructions in the section above.
 
+### Table of Contents
+
+We use `mdbook-toc` to auto-generate TOCs for long sections. You can invoke the preprocessor by
+including the `<!-- toc -->` marker at the place where you want the TOC.
+
 ### Pre-commit script
 
 We also test that line lengths are less than 100 columns. To test this locally,
@@ -95,7 +100,7 @@ but we leave these instructions for when we do it again in the future.
 
 7. Click on the log and Ctrl-f to get a search box in the log
 
-8. Search for rustc-dev-guide. This gets you to the place where the links are checked. It is usually ~11K lines into the log
+8. Search for rustc-dev-guide. This gets you to the place where the links are checked. It is usually ~11K lines into the log.
 
 9. Look at the links in the log near that point in the log
 

book.toml

@@ -6,6 +6,10 @@ description = "A guide to developing rustc"
 [build]
 create-missing = false
 
+[preprocessor.toc]
+command = "mdbook-toc"
+renderer = ["html"]
+
 [output.html]
 git-repository-url = "https://github.com/rust-lang/rustc-dev-guide"
 

src/about-this-guide.md

@@ -47,7 +47,7 @@ a correction!
 If you do contribute to the guide, please see the corresponding
 [subsection on writing documentation in this guide].
 
-[subsection on writing documentation in this guide]: contributing.md#contributing-to-rustc-dev-guide.
+[subsection on writing documentation in this guide]: contributing.md#contributing-to-rustc-dev-guide
 
 > “‘All conditioned things are impermanent’ — when one sees this with wisdom, one turns away from
 > suffering.” _The Dhammapada, verse 277_

src/backend/backend-agnostic.md

@@ -1,5 +1,7 @@
 # Backend Agnostic Codegen
 
+<!-- toc -->
+
 As of January 2021, `rustc_codegen_ssa` provides an abstract interface for all backends to
 implement, to allow other codegen backends (e.g. [Cranelift]).
 

src/backend/implicit-caller-location.md

@@ -1,9 +1,11 @@
 # Implicit Caller Location
 
+<!-- toc -->
+
 Approved in [RFC 2091], this feature enables the accurate reporting of caller location during panics
-initiated from functions like `Option::unwrap`, `Result::expect`, and `Index::index`. This feature 
-adds the [`#[track_caller]`][attr-reference] attribute for functions, the 
-[`caller_location`][intrinsic] intrinsic, and the stabilization-friendly 
+initiated from functions like `Option::unwrap`, `Result::expect`, and `Index::index`. This feature
+adds the [`#[track_caller]`][attr-reference] attribute for functions, the
+[`caller_location`][intrinsic] intrinsic, and the stabilization-friendly
 [`core::panic::Location::caller`][wrapper] wrapper.
 
 ## Motivating Example
@@ -28,25 +30,25 @@ note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
 As of 1.42, we get a much more helpful message:
 
 ```
-$ rustc +1.42.0 example.rs; example.exe 
+$ rustc +1.42.0 example.rs; example.exe
 thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', example.rs:3:5
 note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
 ```
 
 These error messages are achieved through a combination of changes to `panic!` internals to make use
-of `core::panic::Location::caller` and a number of `#[track_caller]` annotations in the standard 
+of `core::panic::Location::caller` and a number of `#[track_caller]` annotations in the standard
 library which propagate caller information.
 
 ## Reading Caller Location
 
 Previously, `panic!` made use of the `file!()`, `line!()`, and `column!()` macros to construct a
 [`Location`] pointing to where the panic occurred. These macros couldn't be given an overridden
-location, so functions which intentionally invoked `panic!` couldn't provide their own location, 
+location, so functions which intentionally invoked `panic!` couldn't provide their own location,
 hiding the actual source of error.
 
-Internally, `panic!()` now calls [`core::panic::Location::caller()`][wrapper] to find out where it 
-was expanded. This function is itself annotated with `#[track_caller]` and wraps the 
-[`caller_location`][intrinsic] compiler intrinsic implemented by rustc. This intrinsic is easiest 
+Internally, `panic!()` now calls [`core::panic::Location::caller()`][wrapper] to find out where it
+was expanded. This function is itself annotated with `#[track_caller]` and wraps the
+[`caller_location`][intrinsic] compiler intrinsic implemented by rustc. This intrinsic is easiest
 explained in terms of how it works in a `const` context.
 
 ## Caller Location in `const`
@@ -56,20 +58,20 @@ to find the right location and allocating a const value to return.
 
 ### Finding the right `Location`
 
-In a const context we "walk up the stack" from where the intrinsic is invoked, stopping when we 
+In a const context we "walk up the stack" from where the intrinsic is invoked, stopping when we
 reach the first function call in the stack which does *not* have the attribute. This walk is in
 [`InterpCx::find_closest_untracked_caller_location()`][const-find-closest].
 
-Starting at the bottom, we iterate up over stack [`Frame`][const-frame]s in the 
+Starting at the bottom, we iterate up over stack [`Frame`][const-frame]s in the
 [`InterpCx::stack`][const-stack], calling
-[`InstanceDef::requires_caller_location`][requires-location] on the 
+[`InstanceDef::requires_caller_location`][requires-location] on the
 [`Instance`s from each `Frame`][frame-instance]. We stop once we find one that returns `false` and
 return the span of the *previous* frame which was the "topmost" tracked function.
 
 ### Allocating a static `Location`
 
-Once we have a `Span`, we need to allocate static memory for the `Location`, which is performed by 
-the [`TyCtxt::const_caller_location()`][const-location-query] query. Internally this calls 
+Once we have a `Span`, we need to allocate static memory for the `Location`, which is performed by
+the [`TyCtxt::const_caller_location()`][const-location-query] query. Internally this calls
 [`InterpCx::alloc_caller_location()`][alloc-location] and results in a unique
 [memory kind][location-memory-kind] (`MemoryKind::CallerLocation`). The SSA codegen backend is able
 to emit code for these same values, and we use this code there as well.
@@ -78,14 +80,14 @@ Once our `Location` has been allocated in static memory, our intrinsic returns a
 
 ## Generating code for `#[track_caller]` callees
 
-To generate efficient code for a tracked function and its callers, we need to provide the same 
+To generate efficient code for a tracked function and its callers, we need to provide the same
 behavior from the intrinsic's point of view without having a stack to walk up at runtime. We invert
 the approach: as we grow the stack down we pass an additional argument to calls of tracked functions
 rather than walking up the stack when the intrinsic is called. That additional argument can be
 returned wherever the caller location is queried.
 
 The argument we append is of type `&'static core::panic::Location<'static>`. A reference was chosen
-to avoid unnecessary copying because a pointer is a third the size of 
+to avoid unnecessary copying because a pointer is a third the size of
 `std::mem::size_of::<core::panic::Location>() == 24` at time of writing.
 
 When generating a call to a function which is tracked, we pass the location argument the value of
@@ -151,12 +153,12 @@ probably the best we can do without modifying fully-stabilized type signatures.
 
 > *Note:* We always emit a [`ReifyShim`] when taking a pointer to a tracked function. While the
 > constraint here is imposed by codegen contexts, we don't know during MIR construction of the shim
-> whether we'll be called in a const context (safe to ignore shim) or in a codegen context (unsafe 
+> whether we'll be called in a const context (safe to ignore shim) or in a codegen context (unsafe
 > to ignore shim). Even if we did know, the results from const and codegen contexts must agree.
 
 ## The Attribute
 
-The `#[track_caller]` attribute is checked alongside other codegen attributes to ensure the 
+The `#[track_caller]` attribute is checked alongside other codegen attributes to ensure the
 function:
 
 * has the `"Rust"` ABI (as opposed to e.g., `"C"`)
@@ -171,7 +173,7 @@ used in both const and codegen contexts to ensure correct propagation.
 
 When applied to trait method implementations, the attribute works as it does for regular functions.
 
-When applied to a trait method prototype, the attribute applies to all implementations of the 
+When applied to a trait method prototype, the attribute applies to all implementations of the
 method. When applied to a default trait method implementation, the attribute takes effect on
 that implementation *and* any overrides.
 
@@ -203,14 +205,14 @@ trait TrackedFourWays {
         assert_tracked!();
     }
 
-    /// Overrides of this implementation are tracked (it is too). 
+    /// Overrides of this implementation are tracked (it is too).
     #[track_caller]
     fn default_tracked_to_override() {
         assert_tracked!();
     }
 }
 
-/// This impl uses the default impl for `default_tracked` and provides its own for 
+/// This impl uses the default impl for `default_tracked` and provides its own for
 /// `default_tracked_to_override`.
 impl TrackedFourWays for () {
     fn blanket_tracked() {
@@ -253,7 +255,7 @@ up on the tracking issue. During the course of implementing that, it was also di
 implementation was possible without modifying the number of arguments in a function's MIR, which
 would simplify later stages and unlock use in traits.
 
-Because the RFC's implementation strategy could not readily support traits, the semantics were not 
+Because the RFC's implementation strategy could not readily support traits, the semantics were not
 originally specified. They have since been implemented following the path which seemed most correct
 to the author and reviewers.
 

src/backend/monomorph.md

@@ -1,5 +1,7 @@
 # Monomorphization
 
+<!-- toc -->
+
 As you probably know, rust has a very expressive type system that has extensive
 support for generic types. But of course, assembly is not generic, so we need
 to figure out the concrete types of all the generics before the code can
@@ -57,12 +59,12 @@ units](../appendix/glossary.md#codegen-unit).
 
 ## Codegen Unit (CGU) partitioning
 
-For better incremental build times, the CGU partitioner creates two CGU for each source level 
-modules. One is for "stable" i.e. non-generic code and the other is more volatile code i.e. 
+For better incremental build times, the CGU partitioner creates two CGU for each source level
+modules. One is for "stable" i.e. non-generic code and the other is more volatile code i.e.
 monoporphized/specialized instances.
 
 For depenencies, consider Crate A and Crate B, such that Crate B depends on Crate A.
-The following table lists different scenarios for a function in Crate A that might be used by one 
+The following table lists different scenarios for a function in Crate A that might be used by one
 or more modules in Crate B.
 
 | Crate A function | Behavior |

src/backend/updating-llvm.md

@@ -1,5 +1,7 @@
 # Updating LLVM
 
+<!-- toc -->
+
 The Rust compiler uses LLVM as its primary codegen backend today, and naturally
 we want to at least occasionally update this dependency! Currently we do not
 have a strict policy about when to update LLVM or what it can be updated to, but

src/borrow_check/moves_and_initialization/move_paths.md

@@ -1,17 +1,19 @@
 # Move paths
 
+<!-- toc -->
+
 In reality, it's not enough to track initialization at the granularity
 of local variables. Rust also allows us to do moves and initialization
 at the field granularity:
 
 ```rust,ignore
 fn foo() {
     let a: (Vec<u32>, Vec<u32>) = (vec![22], vec![44]);
-    
+
     // a.0 and a.1 are both initialized
-    
+
     let b = a.0; // moves a.0
-    
+
     // a.0 is not initialized, but a.1 still is
 
     let c = a.0; // ERROR
@@ -73,7 +75,7 @@ there is no need for a [`MovePathIndex`]. Some examples:
   there would be no move-path for `foo[1]`.
 - You cannot move from inside of a borrowed reference, so if we have e.g. `foo: &String`,
   there would be no move-path for `*foo`.
-  
+
 These rules are enforced by the [`move_path_for`] function, which
 converts a [`Place`] into a [`MovePathIndex`] -- in error cases like
 those just discussed, the function returns an `Err`. This in turn
@@ -102,7 +104,7 @@ of [`MoveData`]. There are two different methods:
   [`LookupResult`] indicating the closest path it was able to find
   that exists (e.g., for `foo[1]`, it might return just the path for
   `foo`).
-  
+
 [`find`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/struct.MovePathLookup.html#method.find
 [`find_local`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_mir/dataflow/move_paths/struct.MovePathLookup.html#method.find_local
 [`mir::Local`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/mir/struct.Local.html
@@ -120,7 +122,7 @@ just over the paths that are **actually referenced** in the source,
 not all **possible** paths that could have been referenced). These
 references are used for example in the
 [`find_in_move_path_or_its_descendants`] function, which determines
-whether a move-path (e.g., `a.b`) or any child of that move-path 
+whether a move-path (e.g., `a.b`) or any child of that move-path
 (e.g.,`a.b.c`) matches a given predicate.
 
 [`Place`]: https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/mir/struct.Place.html

src/borrow_check/region_inference.md

@@ -1,5 +1,7 @@
 # Region inference (NLL)
 
+<!-- toc -->
+
 The MIR-based region checking code is located in [the `rustc_mir::borrow_check`
 module][nll].
 

src/borrow_check/region_inference/constraint_propagation.md

@@ -1,5 +1,7 @@
 # Constraint propagation
 
+<!-- toc -->
+
 The main work of the region inference is **constraint propagation**,
 which is done in the [`propagate_constraints`] function.  There are
 three sorts of constraints that are used in NLL, and we'll explain how

src/borrow_check/region_inference/lifetime_parameters.md

@@ -1,5 +1,7 @@
 # Universal regions
 
+<!-- toc -->
+
 "Universal regions" is the name that the code uses to refer to "named
 lifetimes" -- e.g., lifetime parameters and `'static`. The name
 derives from the fact that such lifetimes are "universally quantified"

src/borrow_check/region_inference/member_constraints.md

@@ -1,5 +1,7 @@
 # Member constraints
 
+<!-- toc -->
+
 A member constraint `'m member of ['c_1..'c_N]` expresses that the
 region `'m` must be *equal* to some **choice regions** `'c_i` (for
 some `i`). These constraints cannot be expressed by users, but they

src/borrow_check/region_inference/placeholders_and_universes.md

@@ -1,5 +1,7 @@
 # Placeholders and universes
 
+<!-- toc -->
+
 From time to time we have to reason about regions that we can't
 concretely know. For example, consider this program:
 

src/bug-fix-procedure.md

@@ -1,4 +1,7 @@
 # Rustc Bug Fix Procedure
+
+<!-- toc -->
+
 This page defines the best practices procedure for making bug fixes or soundness
 corrections in the compiler that can cause existing code to stop compiling. This
 text is based on

src/building/bootstrapping.md

@@ -1,5 +1,7 @@
 # Bootstrapping the Compiler
 
+<!-- toc -->
+
 This subchapter is about the bootstrapping process.
 
 ## What is bootstrapping? How does it work?

src/compiler-debugging.md

@@ -1,6 +1,8 @@
 # Debugging the compiler
 [debugging]: #debugging
 
+<!-- toc -->
+
 This chapter contains a few tips to debug the compiler. These tips aim to be
 useful no matter what you are working on.  Some of the other chapters have
 advice about specific parts of the compiler (e.g. the [Queries Debugging and
@@ -9,14 +11,14 @@ chapter](./backend/debugging.md)).
 
 ## Configuring the compiler
 
-By default, rustc is built without most debug information. To enable debug info, 
-set `debug = true` in your config.toml. 
+By default, rustc is built without most debug information. To enable debug info,
+set `debug = true` in your config.toml.
 
-Setting `debug = true` turns on many different debug options (e.g., `debug-assertions`, 
-`debug-logging`, etc.) which can be individually tweaked if you want to, but many people 
+Setting `debug = true` turns on many different debug options (e.g., `debug-assertions`,
+`debug-logging`, etc.) which can be individually tweaked if you want to, but many people
 simply set `debug = true`. Check out the comments in config.toml.example for more info.
 
-You will need to rebuild the compiler once you've changed any configuration options. 
+You will need to rebuild the compiler once you've changed any configuration options.
 
 ## `-Z` flags
 
@@ -36,7 +38,7 @@ normal Rust programs.  IIRC backtraces **don't work** on MinGW,
 sorry. If you have trouble or the backtraces are full of `unknown`,
 you might want to find some way to use Linux, Mac, or MSVC on Windows.
 
-In the default configuration (without `debug` set to `true`), you don't have line numbers 
+In the default configuration (without `debug` set to `true`), you don't have line numbers
 enabled, so the backtrace looks like this:
 
 ```text
@@ -56,7 +58,7 @@ stack backtrace:
   37: rustc_driver::run_compiler
 ```
 
-If you set `debug = true`, you will get line numbers for the stack trace. 
+If you set `debug = true`, you will get line numbers for the stack trace.
 Then the backtrace will look like this:
 
 ```text