Skip to content

Add a bit about crater #151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jun 5, 2018
Merged

Add a bit about crater #151

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3f69ddd
add a bit about crater
mark-i-m Jun 4, 2018
4543f7c
add more on crater + perf runs
Jun 4, 2018
d12d0e9
Address review feedback
Jun 4, 2018
dcdb6d9
long line
Jun 4, 2018
a2bc876
tests dont need to pass
mark-i-m Jun 5, 2018
641a06f
fix typo
mark-i-m Jun 5, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 32 additions & 1 deletion src/tests/intro.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -170,7 +170,38 @@ communicate with the server to coordinate running tests (see

## Crater

TODO
[Crater](https://github.com/rust-lang-nursery/crater) is a tool for compiling
and running tests for _every_ crate on [crates.io](https://crates.io/). It is
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and also many crates from GitHub... I'm not exactly sure what the search parameters are, but I know we at least try.

mainly used for checking for extent of breakage when implementing potentially
breaking changes.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and ensuring lack of breakage by running beta vs. stable "builds."


### When to run Crater

You should request a crater run if your PR makes large changes to the compiler
or could cause breakage. If you are unsure, feel free to ask your PR's reviewer.

### Requesting Crater Runs

The rust team maintains a few machines that can be used for running crater runs
on the changes introduced by a PR. If your PR needs a crater run, leave a
comment for the triage team in the PR thread. Your will be enqueued by the
triage team and the results will be posted when they are ready. A crater run
usually takes a few days (as of this writing).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please inform the team whether you require a "check-only" crater run, a "build only" crater run, or a "build-and-test" crater run. The difference is primarily in time; the conservative (if you're not sure) option is to go for the build-and-test run. If making changes that will only have an effect at compile-time (e.g., implementing a new trait) then you only need a check run.

Check runs will take around ~3-4 days, with the other two taking 5-6 days on average.


While crater is really useful, it is also important to be aware of a few caveats:

- Not all code is on crates.io! There is a lot of code in repos on GitHub and
elsewhere. Also, companies may not wish to publish their code. Thus, a
successful crater run is not a magically green light that there will be no
breakage; you still need to be careful.

- Crater only runs Linux builds (on x86_64, I believe). Thus, other
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we only test / compile for x86_64-unknown-linux-gnu.

architectures and platforms are not tested. Critically, this includes
Windows.

- Many crates are not tested. This could be for a lot of reasons, including
that the crate doesn't compile any more (e.g. used old nightly features),
has broken or flaky tests, requires network access, or other reasons.

## Further reading

Expand Down