libtest support for env variables

[epage]

With rust-lang/rust#133073, we decided to rename --nocapture but overlooked RUST_TEST_NOCAPTURE. I suggested we defer how to handle RUST_TEST_NOCAPTURE to not block --no-capture on bigger decisions.

Options

• Add RUST_TEST_NO_CAPTURE
• Switch to positive logic with RUST_TEST_CAPTURE
  • Maybe re-evaluate the true/false condition
• Deprecate it

[epage]

From rust-lang/rust#133073 (comment)

Thinking more on this, I'm leaning towards "defer".

--no-capture is a nice quality of life improvement that I'd rather not block on "bigger questions". For me, the big question I'm starting to wonder about is if we should deprecate the env variables. My suspicion is that they are a low-effort config system that is meant to make up for a lacking in the common test runner (cargo test). In that case, we should instead focus on the common test runner. These could also cause problems as Cargo tries to coordinate the behavior of test binaries. If cargo test is trying to read json output and the user bypasses cargo to tell the test to report other content, that could become problematic.

[epage]

Looking into the history of env variable support

1. RUST_THREADS was respected by the Rust runtime and libextra/test got this for free
2. Decouple number of concurrent tests from RUST_THREADS env var rust#7335 suggested decoupling libextra/test from the runtime by adding RUST_TASK_THREADS (implementation: Make the unit-test framework check RUST_TEST_TASKS over RUST_THREADS. rust#8823)
3. Optionally allow writing to stdout in test runs rust#13374 suggested the ability to not capture output. test: Add an option to not capture output rust#13706 added support for --nocapture / RUST_TEST_NOCAPTURE.
4. A --test-threads was finally requested in Add a --test-threads option for rustc test binaries rust#25636 (implementation: Add --test-threads option to test binaries rust#35414)

So this feels like its subject to path dependence. libtest inherited env variable support and then tweaked it. Without further information, I'm guessing RUST_TEST_NOCAPTURE just then copied that pattern.

[epage]

As this point, libtest supports

• RUST_TEST_SHUFFLE (unstable)
• RUST_TEST_SHUFFLE_SEED (unstable)
• RUST_TEST_NOCAPTURE
• RUST_TEST_THREADS