Skip to content

Commit bfcdce6

Browse files
ehussmark-i-m
authored andcommitted
Add some more information about the various tests that get run.
CC #10
1 parent 6e1eccd commit bfcdce6

File tree

2 files changed

+65
-16
lines changed

2 files changed

+65
-16
lines changed

Diff for: src/tests/intro.md

+65-9
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,18 @@
1-
# Using the compiler testing framework
1+
# The compiler testing framework
22

3-
The compiler has an extensive testing framework, masterminded by the
4-
compiletest tool (sources in the [`src/tools/compiletest`]). This
5-
section gives a brief overview of how the testing framework is setup,
6-
and then gets into some of the details on
7-
[how to run tests](./tests/running.html#ui) as well as
8-
[how to add new tests](./tests/adding.html).
3+
The Rust project runs a wide variety of different tests, orchestrated by the
4+
build system (`x.py test`). The main test harness for testing the compiler
5+
itself is a tool called compiletest (sources in the
6+
[`src/tools/compiletest`]). This section gives a brief overview of how the
7+
testing framework is setup, and then gets into some of the details on [how to
8+
run tests](./tests/running.html#ui) as well as [how to add new
9+
tests](./tests/adding.html).
910

1011
[`src/tools/compiletest`]: https://github.com/rust-lang/rust/tree/master/src/tools/compiletest
1112

12-
## Test suites
13+
## Compiletest test suites
1314

14-
The tests are located in the tree in the [`src/test`]
15+
The compiletest tests are located in the tree in the [`src/test`]
1516
directory. Immediately within you will see a series of subdirectories
1617
(e.g. `ui`, `run-make`, and so forth). Each of those directories is
1718
called a **test suite** -- they house a group of tests that are run in
@@ -48,6 +49,61 @@ that give more details.
4849
- `*-fulldeps` -- same as above, but indicates that the test depends on things other
4950
than `libstd` (and hence those things must be built)
5051

52+
## Other Tests
53+
54+
The Rust build system handles running tests for various other things,
55+
including:
56+
57+
- **Tidy** -- This is a custom tool used for validating source code style and
58+
formatting conventions, such as rejecting long lines. There is more
59+
information in the [section on coding conventions](./conventions.html#formatting).
60+
61+
Example: `./x.py test src/tools/tidy`
62+
63+
- **Unittests** -- The Rust standard library and many of the Rust packages
64+
include typical Rust `#[test]` unittests. Under the hood, `x.py` will run
65+
`cargo test` on each package to run all the tests.
66+
67+
Example: `./x.py test src/libstd`
68+
69+
- **Doctests** -- Example code embedded within Rust documentation is executed
70+
via `rustdoc --test`. Examples:
71+
72+
`./x.py test src/doc` -- Runs `rustdoc --test` for all documentation in
73+
`src/doc`.
74+
75+
`./x.py test --doc src/libstd` -- Runs `rustdoc --test` on the standard
76+
library.
77+
78+
- **Linkchecker** -- A small tool for verifying `href` links within
79+
documentation.
80+
81+
Example: `./x.py test src/tools/linkchecker`
82+
83+
- **Distcheck** -- This verifies that the source distribution tarball created
84+
by the build system will unpack, build, and run all tests.
85+
86+
Example: `./x.py test distcheck`
87+
88+
- **Tool tests** -- Packages that are included with Rust have all of their
89+
tests run as well (typically by running `cargo test` within their
90+
directory). This includes things such as cargo, clippy, rustfmt, rls, miri,
91+
bootstrap (testing the Rust build system itself), etc.
92+
93+
- **Cargotest** -- This is a small tool which runs `cargo test` on a few
94+
significant projects (such as `servo`, `ripgrep`, `tokei`, etc.) just to
95+
ensure there aren't any significant regressions.
96+
97+
Example: `./x.py test src/tools/cargotest`
98+
99+
## Testing infrastructure
100+
101+
TODO - bors, platforms, etc.
102+
103+
## Crater
104+
105+
TODO
106+
51107
## Further reading
52108

53109
The following blog posts may also be of interest:

Diff for: src/tests/running.md

-7
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,6 @@ a very long time, and anyway bors / travis will do it for you. (Often,
1313
I will run this command in the background after opening a PR that I
1414
think is done, but rarely otherwise. -nmatsakis)
1515

16-
## Tidy
17-
18-
When you run the full suite of tests via `./x.py test`, the first
19-
thing that executes is a "tidy suite" that checks for long lines and
20-
other formatting conventions. There is more information in the
21-
[section on coding conventions](./conventions.html#formatting).
22-
2316
## Running a subset of the test suites
2417

2518
When working on a specific PR, you will usually want to run a smaller

0 commit comments

Comments
 (0)