diff --git a/rust-version b/rust-version index 651db7864..9693bfd63 100644 --- a/rust-version +++ b/rust-version @@ -1 +1 @@ -9c87288a7d2f03625a813df6d3bfe43c09ad4f5a +ecda83b30f0f68cf5692855dddc0bc38ee8863fc diff --git a/src/SUMMARY.md b/src/SUMMARY.md index a8fddbc75..91c4aeacb 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -75,6 +75,7 @@ - [Prologue](./building/bootstrapping/intro.md) - [What Bootstrapping does](./building/bootstrapping/what-bootstrapping-does.md) - [How Bootstrap does it](./building/bootstrapping/how-bootstrap-does-it.md) +- [Debugging bootstrap](./building/bootstrapping/debugging-bootstrap.md) # High-level Compiler Architecture diff --git a/src/backend/debugging.md b/src/backend/debugging.md index 275a1777a..805291017 100644 --- a/src/backend/debugging.md +++ b/src/backend/debugging.md @@ -56,7 +56,7 @@ These tools include: By default, the Rust build system does not check for changes to the LLVM source code or its build configuration settings. So, if you need to rebuild the LLVM that is linked -into `rustc`, first delete the file `llvm-finished-building`, which should be located +into `rustc`, first delete the file `.llvm-stamp`, which should be located in `build//llvm/`. The default rustc compilation pipeline has multiple codegen units, which is diff --git a/src/building/bootstrapping/debugging-bootstrap.md b/src/building/bootstrapping/debugging-bootstrap.md new file mode 100644 index 000000000..972b4a8fb --- /dev/null +++ b/src/building/bootstrapping/debugging-bootstrap.md @@ -0,0 +1,100 @@ +# Debugging bootstrap + +> FIXME: this page could be expanded + +## `tracing` in bootstrap + +Bootstrap has conditional [`tracing`][tracing] setup to provide structured logging. + +[tracing]: https://docs.rs/tracing/0.1.41/tracing/index.html + +### Enabling `tracing` output + +Bootstrap will conditionally enable `tracing` output if the `BOOTSTRAP_TRACING` env var is set. + +Example usage: + +```bash +$ BOOTSTRAP_TRACING=TRACE ./x build library --stage 1 +``` + +Example output[^experimental]: + +![Example bootstrap tracing output](./debugging-bootstrap/tracing-output-example.png) + +[^experimental]: This shows what's *possible* with the infra in an experimental implementation. + +The env var `BOOTSTRAP_TRACING` accepts a [`tracing` env-filter][tracing-env-filter]. The `TRACE` filter will enable *all* `trace` level or less verbose level tracing output. + +[tracing-env-filter]: https://docs.rs/tracing-subscriber/0.3.19/tracing_subscriber/filter/struct.EnvFilter.html + +### Using `tracing` in bootstrap + +Both `tracing::*` macros and the `tracing::instrument` proc-macro attribute need to be gated behind `tracing` feature. Examples: + +```rs +#[cfg(feature = "tracing")] +use tracing::{instrument, trace}; + +struct Foo; + +impl Step for Foo { + type Output = (); + + #[cfg_attr(feature = "tracing", instrument(level = "trace", name = "Foo::should_run", skip_all))] + fn should_run(run: ShouldRun<'_>) -> ShouldRun<'_> { + #[cfg(feature = "tracing")] + trace!(?run, "entered Foo::should_run"); + + todo!() + } + + #[cfg_attr( + feature = "tracing", + instrument( + level = "trace", + name = "Foo::run", + skip_all, + fields(compiler = ?builder.compiler), + ), + )] + fn run(self, builder: &Builder<'_>) -> Self::Output { + #[cfg(feature = "tracing")] + trace!(?run, "entered Foo::run"); + + todo!() + } +} +``` + +For `#[instrument]`, it's recommended to: + +- Gate it behind `trace` level for fine-granularity, possibly `debug` level for core functions. +- Explicitly pick an instrumentation name via `name = ".."` to distinguish between e.g. `run` of different steps. +- Take care to not cause diverging behavior via tracing, e.g. building extra things only when tracing infra is enabled. + +### Enabling `tracing` bootstrap feature in rust-analyzer + +You can adjust your `settings.json`'s `rust-analyzer.check.overrideCommand` and `rust-analyzer.cargo.buildScripts.overrideCommand` if you want to also enable `logging` cargo feature by default in your editor. This is mostly useful if you want proper r-a completions and such when working on bootstrap itself. + +```json +"rust-analyzer.check.overrideCommand": [ + "BOOTSTRAP_TRACING=1", // <- BOOTSTRAP_TRACING=1 won't enable tracing filter, but it will activate bootstrap's `tracing` feature + "python3", + "x.py", + "check", + "--json-output", + "--build-dir=build-rust-analyzer" +], +``` + +```json +"rust-analyzer.cargo.buildScripts.overrideCommand": [ + "BOOTSTRAP_TRACING=1", // <- note this + "python3", + "x.py", + "check", + "--json-output", + "--build-dir=build-rust-analyzer" +], +``` diff --git a/src/building/bootstrapping/debugging-bootstrap/tracing-output-example.png b/src/building/bootstrapping/debugging-bootstrap/tracing-output-example.png new file mode 100644 index 000000000..745aec50d Binary files /dev/null and b/src/building/bootstrapping/debugging-bootstrap/tracing-output-example.png differ diff --git a/src/building/bootstrapping/intro.md b/src/building/bootstrapping/intro.md index f829884fb..f72918c83 100644 --- a/src/building/bootstrapping/intro.md +++ b/src/building/bootstrapping/intro.md @@ -17,5 +17,8 @@ In this section, we give a high-level overview of [what Bootstrap does](./what-bootstrapping-does.md), followed by a high-level introduction to [how Bootstrap does it](./how-bootstrap-does-it.md). +Additionally, see [debugging bootstrap](./debugging-bootstrap.md) to learn +about debugging methods. + [boot]: https://en.wikipedia.org/wiki/Bootstrapping_(compilers) [ocaml-compiler]: https://github.com/rust-lang/rust/tree/ef75860a0a72f79f97216f8aaa5b388d98da6480/src/boot diff --git a/src/building/optimized-build.md b/src/building/optimized-build.md index 3080dc6bf..8feda5982 100644 --- a/src/building/optimized-build.md +++ b/src/building/optimized-build.md @@ -126,4 +126,4 @@ Here is an example of how can `opt-dist` be used locally (outside of CI): [`Environment`]: https://github.com/rust-lang/rust/blob/ee451f8faccf3050c76cdcd82543c917b40c7962/src/tools/opt-dist/src/environment.rs#L5 > Note: if you want to run the actual CI pipeline, instead of running `opt-dist` locally, -> you can execute `DEPLOY=1 src/ci/docker/run.sh dist-x86_64-linux`. +> you can execute `python3 src/ci/github-actions/ci.py run-local dist-x86_64-linux`. diff --git a/src/tests/ci.md b/src/tests/ci.md index cf7c7b79c..9dde40789 100644 --- a/src/tests/ci.md +++ b/src/tests/ci.md @@ -299,8 +299,7 @@ platform’s custom [Docker container]. This has a lot of advantages for us: - We can avoid reinstalling tools (like QEMU or the Android emulator) every time thanks to Docker image caching. - Users can run the same tests in the same environment locally by just running - `src/ci/docker/run.sh image-name`, which is awesome to debug failures. Note - that there are only linux docker images available locally due to licensing and + `python3 src/ci/github-actions/ci.py run-local `, which is awesome to debug failures. Note that there are only linux docker images available locally due to licensing and other restrictions. The docker images prefixed with `dist-` are used for building artifacts while diff --git a/src/tests/directives.md b/src/tests/directives.md index ea9caf8d0..33304962a 100644 --- a/src/tests/directives.md +++ b/src/tests/directives.md @@ -94,7 +94,8 @@ for more details. | Directive | Explanation | Supported test suites | Possible values | |-----------------------------------|--------------------------------------------------------------------------------------------------------------------------|----------------------------------------------|-----------------------------------------------------------------------------------------| | `check-run-results` | Check run test binary `run-{pass,fail}` output snapshot | `ui`, `crashes`, `incremental` if `run-pass` | N/A | -| `error-pattern` | Check that output contains a regex pattern | `ui`, `crashes`, `incremental` if `run-pass` | Regex | +| `error-pattern` | Check that output contains a specific string | `ui`, `crashes`, `incremental` if `run-pass` | String | +| `regex-error-pattern` | Check that output contains a regex pattern | `ui`, `crashes`, `incremental` if `run-pass` | Regex | | `check-stdout` | Check `stdout` against `error-pattern`s from running test binary[^check_stdout] | `ui`, `crashes`, `incremental` | N/A | | `normalize-stderr-32bit` | Normalize actual stderr (for 32-bit platforms) with a rule `"" -> ""` before comparing against snapshot | `ui`, `incremental` | `"" -> ""`, ``/`` is regex capture and replace syntax | | `normalize-stderr-64bit` | Normalize actual stderr (for 64-bit platforms) with a rule `"" -> ""` before comparing against snapshot | `ui`, `incremental` | `"" -> ""`, ``/`` is regex capture and replace syntax | diff --git a/src/tests/docker.md b/src/tests/docker.md index 31e3825f5..a0aa8bd3e 100644 --- a/src/tests/docker.md +++ b/src/tests/docker.md @@ -45,6 +45,15 @@ Some additional notes about using the Docker images: containers. With the container name, run `docker exec -it /bin/bash` where `` is the container name like `4ba195e95cef`. +The approach described above is a relatively low-level interface for running the Docker images +directly. If you want to run a full CI Linux job locally with Docker, in a way that is as close to CI as possible, you can use the following command: + +```bash +python3 src/ci/github-actions/ci.py run-local +# For example: +python3 src/ci/github-actions/ci.py run-local dist-x86_64-linux-alt +``` + [Docker]: https://www.docker.com/ [`src/ci/docker`]: https://github.com/rust-lang/rust/tree/master/src/ci/docker [`src/ci/docker/run.sh`]: https://github.com/rust-lang/rust/blob/master/src/ci/docker/run.sh