|
1 |
| -# Using the compiler testing framework |
| 1 | +# The compiler testing framework |
2 | 2 |
|
3 |
| -The compiler has an extensive testing framework, masterminded by the |
4 |
| -compiletest tool (sources in the [`src/tools/compiletest`]). This |
5 |
| -section gives a brief overview of how the testing framework is setup, |
6 |
| -and then gets into some of the details on |
7 |
| -[how to run tests](./tests/running.html#ui) as well as |
8 |
| -[how to add new tests](./tests/adding.html). |
| 3 | +The Rust project runs a wide variety of different tests, orchestrated by the |
| 4 | +build system (`x.py test`). The main test harness for testing the compiler |
| 5 | +itself is a tool called compiletest (sources in the |
| 6 | +[`src/tools/compiletest`]). This section gives a brief overview of how the |
| 7 | +testing framework is setup, and then gets into some of the details on [how to |
| 8 | +run tests](./tests/running.html#ui) as well as [how to add new |
| 9 | +tests](./tests/adding.html). |
9 | 10 |
|
10 | 11 | [`src/tools/compiletest`]: https://github.com/rust-lang/rust/tree/master/src/tools/compiletest
|
11 | 12 |
|
12 |
| -## Test suites |
| 13 | +## Compiletest test suites |
13 | 14 |
|
14 |
| -The tests are located in the tree in the [`src/test`] |
| 15 | +The compiletest tests are located in the tree in the [`src/test`] |
15 | 16 | directory. Immediately within you will see a series of subdirectories
|
16 | 17 | (e.g. `ui`, `run-make`, and so forth). Each of those directories is
|
17 | 18 | called a **test suite** -- they house a group of tests that are run in
|
@@ -48,6 +49,61 @@ that give more details.
|
48 | 49 | - `*-fulldeps` -- same as above, but indicates that the test depends on things other
|
49 | 50 | than `libstd` (and hence those things must be built)
|
50 | 51 |
|
| 52 | +## Other Tests |
| 53 | + |
| 54 | +The Rust build system handles running tests for various other things, |
| 55 | +including: |
| 56 | + |
| 57 | +- **Tidy** -- This is a custom tool used for validating source code style and |
| 58 | + formatting conventions, such as rejecting long lines. There is more |
| 59 | + information in the [section on coding conventions](./conventions.html#formatting). |
| 60 | + |
| 61 | + Example: `./x.py test src/tools/tidy` |
| 62 | + |
| 63 | +- **Unittests** -- The Rust standard library and many of the Rust packages |
| 64 | + include typical Rust `#[test]` unittests. Under the hood, `x.py` will run |
| 65 | + `cargo test` on each package to run all the tests. |
| 66 | + |
| 67 | + Example: `./x.py test src/libstd` |
| 68 | + |
| 69 | +- **Doctests** -- Example code embedded within Rust documentation is executed |
| 70 | + via `rustdoc --test`. Examples: |
| 71 | + |
| 72 | + `./x.py test src/doc` -- Runs `rustdoc --test` for all documentation in |
| 73 | + `src/doc`. |
| 74 | + |
| 75 | + `./x.py test --doc src/libstd` -- Runs `rustdoc --test` on the standard |
| 76 | + library. |
| 77 | + |
| 78 | +- **Linkchecker** -- A small tool for verifying `href` links within |
| 79 | + documentation. |
| 80 | + |
| 81 | + Example: `./x.py test src/tools/linkchecker` |
| 82 | + |
| 83 | +- **Distcheck** -- This verifies that the source distribution tarball created |
| 84 | + by the build system will unpack, build, and run all tests. |
| 85 | + |
| 86 | + Example: `./x.py test distcheck` |
| 87 | + |
| 88 | +- **Tool tests** -- Packages that are included with Rust have all of their |
| 89 | + tests run as well (typically by running `cargo test` within their |
| 90 | + directory). This includes things such as cargo, clippy, rustfmt, rls, miri, |
| 91 | + bootstrap (testing the Rust build system itself), etc. |
| 92 | + |
| 93 | +- **Cargotest** -- This is a small tool which runs `cargo test` on a few |
| 94 | + significant projects (such as `servo`, `ripgrep`, `tokei`, etc.) just to |
| 95 | + ensure there aren't any significant regressions. |
| 96 | + |
| 97 | + Example: `./x.py test src/tools/cargotest` |
| 98 | + |
| 99 | +## Testing infrastructure |
| 100 | + |
| 101 | +TODO - bors, platforms, etc. |
| 102 | + |
| 103 | +## Crater |
| 104 | + |
| 105 | +TODO |
| 106 | + |
51 | 107 | ## Further reading
|
52 | 108 |
|
53 | 109 | The following blog posts may also be of interest:
|
|
0 commit comments