From bc2c599e86cc999ee761702a6ed61d10f938b4cd Mon Sep 17 00:00:00 2001 From: Camelid Date: Thu, 4 Feb 2021 12:32:45 -0800 Subject: [PATCH 1/5] Update docs from date triage for 2021-02 The biggest change was updating the list of rustdoc passes. Several new ones have been added since that documentation was written, so I added those, and I also sorted the list so it is roughly alphabetical (except for the part for the `strip-*` passes, which I left in the same order since one of the list items has two passes so there's no "correct" order). --- src/compiler-src.md | 4 ---- src/overview.md | 2 +- src/parallel-rustc.md | 6 ++--- src/rustdoc-internals.md | 49 +++++++++++++++++++++++++++++++--------- src/tests/adding.md | 4 ++-- src/type-inference.md | 2 +- 6 files changed, 45 insertions(+), 22 deletions(-) diff --git a/src/compiler-src.md b/src/compiler-src.md index 96e91fd73..c4f415441 100644 --- a/src/compiler-src.md +++ b/src/compiler-src.md @@ -21,10 +21,6 @@ containing the compiler, the standard libraries (`core`, `alloc`, `std`, `proc_macro`, etc), and `rustdoc`, along with the build system and a bunch of tools and submodules for building a full Rust distribution. -As of August 2020 , this structure is gradually undergoing -some transformation to make it a bit less monolithic and more approachable, -especially to newcomers. - The repository consists of three main directories: - `compiler/` contains the source code for `rustc`. It consists of many crates diff --git a/src/overview.md b/src/overview.md index 45ae0ab2e..c96321f39 100644 --- a/src/overview.md +++ b/src/overview.md @@ -250,7 +250,7 @@ Moreover, the compiler wasn't originally built to use a query system; the query system has been retrofitted into the compiler, so parts of it are not query-fied yet. Also, LLVM isn't our code, so that isn't querified either. The plan is to eventually query-fy all of the steps listed in the previous section, but as of -April 2020 , only the steps between HIR and LLVM-IR are +February 2021 , only the steps between HIR and LLVM IR are query-fied. That is, lexing and parsing are done all at once for the whole program. diff --git a/src/parallel-rustc.md b/src/parallel-rustc.md index 6f7f6f54b..c47a46e78 100644 --- a/src/parallel-rustc.md +++ b/src/parallel-rustc.md @@ -22,9 +22,9 @@ There are a few basic ideas in this effort: [`rayon`]: https://crates.io/crates/rayon -As of May 2020 , much of this effort is on hold due to -lack of manpower. We have a working prototype with promising performance gains -in many cases. However, there are two blockers: +As of February 2021 , much of this effort is on hold due +to lack of manpower. We have a working prototype with promising performance +gains in many cases. However, there are two blockers: - It's not clear what invariants need to be upheld that might not hold in the face of concurrency. An auditing effort was underway, but seems to have diff --git a/src/rustdoc-internals.md b/src/rustdoc-internals.md index 7460c012c..90f4426a1 100644 --- a/src/rustdoc-internals.md +++ b/src/rustdoc-internals.md @@ -51,14 +51,14 @@ which describe the publicly-documentable items in the target crate. ### Hot potato Before moving on to the next major step, a few important "passes" occur over -the documentation. These do things like combine the separate "attributes" into +the documentation. These do things like combine the separate "attributes" into a single string and strip leading whitespace to make the document easier on the markdown parser, or drop items that are not public or deliberately hidden with `#[doc(hidden)]`. These are all implemented in the `passes/` directory, one file per pass. By default, all of these passes are run on a crate, but the ones regarding dropping private/hidden items can be bypassed by passing `--document-private-items` to rustdoc. Note that unlike the previous set of AST -transformations, the passes happen on the _cleaned_ crate. +transformations, the passes are run on the _cleaned_ crate. (Strictly speaking, you can fine-tune the passes run and even add your own, but [we're trying to deprecate that][44136]. If you need finer-grain control over @@ -66,29 +66,56 @@ these passes, please let us know!) [44136]: https://github.com/rust-lang/rust/issues/44136 -Here is the list of passes as of March 2018 : +Here is the list of passes as of February 2021 : + +- `calculate-doc-coverage` calculates information used for the `--show-coverage` + flag. + +- `check-code-block-syntax` validates syntax inside Rust code blocks + (`` ```rust ``) + +- `check-invalid-html-tags` detects invalid HTML (like an unclosed `\`) + in doc comments. + +- `check-non-autolinks` detects links that could or should be written using + angle brackets (the code behind the nightly-only + `non_autolinks` lint). -- `propagate-doc-cfg` - propagates `#[doc(cfg(...))]` to child items. - `collapse-docs` concatenates all document attributes into one document attribute. This is necessary because each line of a doc comment is given as a separate doc attribute, and this will combine them into a single string with line breaks between each attribute. -- `unindent-comments` removes excess indentation on comments in order for - markdown to like it. This is necessary because the convention for writing - documentation is to provide a space between the `///` or `//!` marker and the - text, and stripping that leading space will make the text easier to parse by - the Markdown parser. (In the past, the markdown parser used was not - Commonmark- compliant, which caused annoyances with extra whitespace but this - seems to be less of an issue today.) + +- `collect-intra-doc-links` resolves intra-doc links. + +- `collect-trait-impls` collects trait impls for each item in the crate. For + example, if we define a struct that implements a trait, this pass will note + that the struct implements that trait. + +- `doc-test-lints` runs various lints on the doctests. + +- `propagate-doc-cfg` propagates `#[doc(cfg(...))]` to child items. + - `strip-priv-imports` strips all private import statements (`use`, `extern crate`) from a crate. This is necessary because rustdoc will handle *public* imports by either inlining the item's documentation to the module or creating a "Reexports" section with the import in it. The pass ensures that all of these imports are actually relevant to documentation. + - `strip-hidden` and `strip-private` strip all `doc(hidden)` and private items from the output. `strip-private` implies `strip-priv-imports`. Basically, the goal is to remove items that are not relevant for public documentation. +- `stripper` provides utilities for the `strip-*` passes. + +- `unindent-comments` removes excess indentation on comments in order for + markdown to like it. This is necessary because the convention for writing + documentation is to provide a space between the `///` or `//!` marker and the + text, and stripping that leading space will make the text easier to parse by + the Markdown parser. (In the past, the Markdown parser used was not + CommonMark-compliant, which caused annoyances with extra whitespace but this + seems to be less of an issue today.) + ## From clean to crate This is where the "second phase" in rustdoc begins. This phase primarily lives diff --git a/src/tests/adding.md b/src/tests/adding.md index c19271d2c..465317672 100644 --- a/src/tests/adding.md +++ b/src/tests/adding.md @@ -339,8 +339,8 @@ The error levels that you can have are: ## Revisions -Certain classes of tests support "revisions" (as of February 2018 , this includes compile-fail, run-fail, and incremental, though +Certain classes of tests support "revisions" (as of February 2021 , this includes compile-fail, run-fail, and incremental, though incremental tests are somewhat different). Revisions allow a single test file to be used for multiple tests. This is done by adding a special header at the top of the file: diff --git a/src/type-inference.md b/src/type-inference.md index 6496b48ff..ea6f6eaff 100644 --- a/src/type-inference.md +++ b/src/type-inference.md @@ -72,7 +72,7 @@ inference works, or perhaps this blog post on [Unification in the Chalk project]: http://smallcultfollowing.com/babysteps/blog/2017/03/25/unification-in-chalk-part-1/ All told, the inference context stores four kinds of inference variables (as of -January 2018 ): +February 2021 ): - Type variables, which come in three varieties: - General type variables (the most common). These can be unified with any From c6c49a1bacd78a2eb72c4ffb4c95d8dc5ef77f5a Mon Sep 17 00:00:00 2001 From: Camelid Date: Mon, 15 Feb 2021 22:06:00 -0800 Subject: [PATCH 2/5] Update based on reviews --- src/rustdoc-internals.md | 30 ++++++++++++++++++++---------- 1 file changed, 20 insertions(+), 10 deletions(-) diff --git a/src/rustdoc-internals.md b/src/rustdoc-internals.md index 90f4426a1..6835d4c3f 100644 --- a/src/rustdoc-internals.md +++ b/src/rustdoc-internals.md @@ -74,7 +74,7 @@ Here is the list of passes as of February 2021 : - `check-code-block-syntax` validates syntax inside Rust code blocks (`` ```rust ``) -- `check-invalid-html-tags` detects invalid HTML (like an unclosed `\`) +- `check-invalid-html-tags` detects invalid HTML (like an unclosed ``) in doc comments. - `check-non-autolinks` detects links that could or should be written using @@ -106,15 +106,25 @@ Here is the list of passes as of February 2021 : from the output. `strip-private` implies `strip-priv-imports`. Basically, the goal is to remove items that are not relevant for public documentation. -- `stripper` provides utilities for the `strip-*` passes. - -- `unindent-comments` removes excess indentation on comments in order for - markdown to like it. This is necessary because the convention for writing - documentation is to provide a space between the `///` or `//!` marker and the - text, and stripping that leading space will make the text easier to parse by - the Markdown parser. (In the past, the Markdown parser used was not - CommonMark-compliant, which caused annoyances with extra whitespace but this - seems to be less of an issue today.) +- `unindent-comments` removes excess indentation on comments in order for the + Markdown to be parsed correctly. This is necessary because the convention for + writing documentation is to provide a space between the `///` or `//!` marker + and the doc text, but Markdown is whitespace-sensitive. For example, a block + of text with four-space indentation is parsed as a code block, so if we didn't + unindent comments, these list items + + ```rust,ignore + /// A list: + /// + /// - Foo + /// - Bar + ``` + + would be parsed as if they were in a code block, which is likely not what the + user intended. + +There is also a `stripper` module in `passes/`, but it is a collection of +utility functions for the `strip-*` passes and is not a pass itself. ## From clean to crate From 58ac87567c7bb5d110925159c51794faf2431c6e Mon Sep 17 00:00:00 2001 From: Camelid Date: Tue, 16 Feb 2021 11:08:03 -0800 Subject: [PATCH 3/5] Update --- src/overview.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/src/overview.md b/src/overview.md index c96321f39..c34d01dfd 100644 --- a/src/overview.md +++ b/src/overview.md @@ -251,8 +251,8 @@ system has been retrofitted into the compiler, so parts of it are not query-fied yet. Also, LLVM isn't our code, so that isn't querified either. The plan is to eventually query-fy all of the steps listed in the previous section, but as of February 2021 , only the steps between HIR and LLVM IR are -query-fied. That is, lexing and parsing are done all at once for the whole -program. +query-fied. That is, lexing, parsing, name resolution, and macro expansion are +done all at once for the whole program. One other thing to mention here is the all-important "typing context", [`TyCtxt`], which is a giant struct that is at the center of all things. From db75996c0a9a13aa4890c685d77a00ab7947d25f Mon Sep 17 00:00:00 2001 From: Camelid Date: Sat, 20 Feb 2021 16:28:08 -0800 Subject: [PATCH 4/5] Revert date update for inference Didn't get a response on Zulip, so saving this for another time. --- src/type-inference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/type-inference.md b/src/type-inference.md index ea6f6eaff..6496b48ff 100644 --- a/src/type-inference.md +++ b/src/type-inference.md @@ -72,7 +72,7 @@ inference works, or perhaps this blog post on [Unification in the Chalk project]: http://smallcultfollowing.com/babysteps/blog/2017/03/25/unification-in-chalk-part-1/ All told, the inference context stores four kinds of inference variables (as of -February 2021 ): +January 2018 ): - Type variables, which come in three varieties: - General type variables (the most common). These can be unified with any From b8c9b3979c39e8f26476882487831f49c42f5daa Mon Sep 17 00:00:00 2001 From: Camelid Date: Sat, 20 Feb 2021 18:50:22 -0800 Subject: [PATCH 5/5] Link to intra-doc links documentation Co-authored-by: Joshua Nelson --- src/rustdoc-internals.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/src/rustdoc-internals.md b/src/rustdoc-internals.md index 6835d4c3f..f7799f967 100644 --- a/src/rustdoc-internals.md +++ b/src/rustdoc-internals.md @@ -86,7 +86,7 @@ Here is the list of passes as of February 2021 : separate doc attribute, and this will combine them into a single string with line breaks between each attribute. -- `collect-intra-doc-links` resolves intra-doc links. +- `collect-intra-doc-links` resolves [intra-doc links](https://doc.rust-lang.org/rustdoc/linking-to-items-by-name.html). - `collect-trait-impls` collects trait impls for each item in the crate. For example, if we define a struct that implements a trait, this pass will note @@ -261,4 +261,3 @@ on the internet. For example, the url for `std` will be `/std/". [`rustdoc` api docs]: https://doc.rust-lang.org/nightly/nightly-rustc/rustdoc/ [The rustdoc user guide]: https://doc.rust-lang.org/nightly/rustdoc/ -