support config extensions #138934

onur-ozkan · 2025-03-25T17:21:44Z

Copied from the rustc-dev-guide addition:

When working on different tasks, you might need to switch between different bootstrap >configurations.
Sometimes you may want to keep an old configuration for future use. But saving raw config >values in
random files and manually copying and pasting them can quickly become messy, especially if >you have a
long history of different configurations.

To simplify managing multiple configurations, you can create config extensions.

For example, you can create a simple config file named cross.toml:
[build]
build = "x86_64-unknown-linux-gnu"
host = ["i686-unknown-linux-gnu"]
target = ["i686-unknown-linux-gnu"]


[llvm]
download-ci-llvm = false

[target.x86_64-unknown-linux-gnu]
llvm-config = "/path/to/llvm-19/bin/llvm-config"
Then, include this in your bootstrap.toml:
include = ["cross.toml"]
You can also include extensions within extensions recursively.

Note: In the include field, the overriding logic follows a right-to-left order. For example,
in include = ["a.toml", "b.toml"], extension b.toml overrides a.toml. Also, parent extensions
always overrides the inner ones.

try-job: x86_64-mingw-2

rustbot · 2025-03-25T17:21:52Z

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2025-03-25T17:21:54Z

The rustc-dev-guide subtree was changed. If this PR only touches the dev guide consider submitting a PR directly to rust-lang/rustc-dev-guide otherwise thank you for updating the dev guide with your changes.

cc @BoxyUwU, @jieyouxu, @Kobzol

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

TimNN · 2025-03-25T17:57:21Z

Some thoughts (aside from the fact that I like this feature):

~~Consider being explicit about the fact that the additional files must be in the root repository directory, both in the PR description and the docs.~~ edit: I didn't consider the case of bootstrap.toml not being located in the root repository directory.
Consider changing that, I'd personally prefer having all custom profiles in a single directory (so they are grouped in the file list, and to make it easier to .gitignore them all).
As an alternative, switch from custom profiles to "includes", i.e. referencing files instead of profile names.
- This would clearly differentiate built-in profiles from custom includes.
- This would make it easy to use a custom directory for custom profiles, possibly even shared between multiple rust checkouts.
- We could differentiate profiles and includes by checking for the .toml extension, or by having profile and include be separate fields.
- Relative includes would need somewhat careful handling (should probably be relative to the current file).
- Includes could open up the possibility of including multiple files (e.g. one to apply my "general" default configuration, and one to switch between different LLVM configurations). Though this may be better left for a follow-up.

jieyouxu · 2025-03-26T15:54:28Z

Rerolling review due to bandwidth because I'm still slowly testing #119899.
r? bootstrap

Kobzol · 2025-03-26T16:11:21Z

r? kobzol

Will take a look tomorrow.

jieyouxu · 2025-03-26T16:17:23Z

I haven't looked too closely, but based on description

Create some-config1 profile in some-config1.toml.
Create some-config2 profile in some-config2.toml and inherit some-config1.
Create some-config3 profile in some-config3.toml and inherit some-config2.

can this infinite loop? I guess even if it does, just don't do that 😆

onur-ozkan · 2025-03-26T17:20:00Z

I'm planning to rework on this next week.

@rustbot author

onur-ozkan · 2025-04-01T07:49:25Z

Consider being explicit about the fact that the additional files must be in the root repository directory, both in the PR description and the docs.

That's not actually true. I keep my config file outside the rust repository and invoke x from there to safely work on different checkouts. Bootstrap reads config files from the current working directory where you call it, so it doesn't have to be the project root.

TimNN · 2025-04-01T07:57:11Z

Consider being explicit about the fact that the additional files must be in the root repository directory, both in the PR description and the docs.

That's not actually true. I keep my config file outside the rust repository and invoke x from there to safely work on different checkouts. Bootstrap reads config files from the current working directory where you call it, so it doesn't have to be the project root.

Ah, sorry, I didn't consider that case (I didn't know that was even an option).

rustbot · 2025-04-01T08:53:52Z

This PR modifies bootstrap.example.toml.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

onur-ozkan · 2025-04-01T08:54:40Z

@rustbot ready

src/bootstrap/src/core/config/config.rs

Kobzol

Looks simple enough. I think that we should also mark profile as deprecated and just replace it with include = ["src/bootstrap/defaults/<profile>"], and then remove it sometime in the future. Both to avoid having two ways of achieving a similar thing, and also to make the contents of the profiles more discoverable, before it was not at all obvious what does profile = "compiler" mean and from where it is taken.

src/doc/rustc-dev-guide/src/building/suggested.md

onur-ozkan · 2025-04-18T09:24:45Z

@bors try

… r=<try> support config extensions _Copied from the `rustc-dev-guide` addition:_ >When working on different tasks, you might need to switch between different bootstrap >configurations. >Sometimes you may want to keep an old configuration for future use. But saving raw config >values in >random files and manually copying and pasting them can quickly become messy, especially if >you have a >long history of different configurations. > >To simplify managing multiple configurations, you can create config extensions. > >For example, you can create a simple config file named `cross.toml`: > >```toml >[build] >build = "x86_64-unknown-linux-gnu" >host = ["i686-unknown-linux-gnu"] >target = ["i686-unknown-linux-gnu"] > > >[llvm] >download-ci-llvm = false > >[target.x86_64-unknown-linux-gnu] >llvm-config = "/path/to/llvm-19/bin/llvm-config" >``` > >Then, include this in your `bootstrap.toml`: > >```toml >include = ["cross.toml"] >``` > >You can also include extensions within extensions recursively. > >**Note:** In the `include` field, the overriding logic follows a right-to-left order. For example, in `include = ["a.toml", "b.toml"]`, extension `b.toml` overrides `a.toml`. Also, parent extensions always overrides the inner ones. try-job: x86_64-mingw-2

bors · 2025-04-18T09:25:56Z

⌛ Trying commit 12de52b with merge 5497f4a...

bors · 2025-04-18T11:33:18Z

💔 Test failed - checks-actions

Signed-off-by: onur-ozkan <[email protected]>

onur-ozkan · 2025-04-18T13:11:12Z

@bors try

… r=<try> support config extensions _Copied from the `rustc-dev-guide` addition:_ >When working on different tasks, you might need to switch between different bootstrap >configurations. >Sometimes you may want to keep an old configuration for future use. But saving raw config >values in >random files and manually copying and pasting them can quickly become messy, especially if >you have a >long history of different configurations. > >To simplify managing multiple configurations, you can create config extensions. > >For example, you can create a simple config file named `cross.toml`: > >```toml >[build] >build = "x86_64-unknown-linux-gnu" >host = ["i686-unknown-linux-gnu"] >target = ["i686-unknown-linux-gnu"] > > >[llvm] >download-ci-llvm = false > >[target.x86_64-unknown-linux-gnu] >llvm-config = "/path/to/llvm-19/bin/llvm-config" >``` > >Then, include this in your `bootstrap.toml`: > >```toml >include = ["cross.toml"] >``` > >You can also include extensions within extensions recursively. > >**Note:** In the `include` field, the overriding logic follows a right-to-left order. For example, in `include = ["a.toml", "b.toml"]`, extension `b.toml` overrides `a.toml`. Also, parent extensions always overrides the inner ones. try-job: x86_64-mingw-2

bors · 2025-04-18T13:12:24Z

⌛ Trying commit ac7d1be with merge 63434d0f5bba383ab30a3adf000c2d82fdb288b5...

onur-ozkan · 2025-04-18T14:51:52Z

@rustbot ready

bors · 2025-04-18T15:22:34Z

☀️ Try build successful - checks-actions
Build commit: 63434d0 (63434d0f5bba383ab30a3adf000c2d82fdb288b5)

Kobzol · 2025-04-18T15:24:25Z

@bors r+

bors · 2025-04-18T15:24:28Z

📌 Commit ac7d1be has been approved by Kobzol

It is now in the queue for this repository.

Kobzol · 2025-04-18T15:24:33Z

Thanks for adding the test helper function!

…s, r=Kobzol support config extensions _Copied from the `rustc-dev-guide` addition:_ >When working on different tasks, you might need to switch between different bootstrap >configurations. >Sometimes you may want to keep an old configuration for future use. But saving raw config >values in >random files and manually copying and pasting them can quickly become messy, especially if >you have a >long history of different configurations. > >To simplify managing multiple configurations, you can create config extensions. > >For example, you can create a simple config file named `cross.toml`: > >```toml >[build] >build = "x86_64-unknown-linux-gnu" >host = ["i686-unknown-linux-gnu"] >target = ["i686-unknown-linux-gnu"] > > >[llvm] >download-ci-llvm = false > >[target.x86_64-unknown-linux-gnu] >llvm-config = "/path/to/llvm-19/bin/llvm-config" >``` > >Then, include this in your `bootstrap.toml`: > >```toml >include = ["cross.toml"] >``` > >You can also include extensions within extensions recursively. > >**Note:** In the `include` field, the overriding logic follows a right-to-left order. For example, in `include = ["a.toml", "b.toml"]`, extension `b.toml` overrides `a.toml`. Also, parent extensions always overrides the inner ones. try-job: x86_64-mingw-2

…iaskrgr Rollup of 8 pull requests Successful merges: - rust-lang#137454 (not lint break with label and unsafe block) - rust-lang#138934 (support config extensions) - rust-lang#139297 (Deduplicate & clean up Nix shell) - rust-lang#139834 (Don't canonicalize crate paths) - rust-lang#139868 (Move `pal::env` into `std::sys::env_consts`) - rust-lang#139978 (Add citool command for generating a test dashboard) - rust-lang#140000 (skip llvm-config in autodiff check builds, when its unavailable) - rust-lang#140007 (Disable has_thread_local on i686-win7-windows-msvc) r? `@ghost` `@rustbot` modify labels: rollup

…s, r=Kobzol support config extensions _Copied from the `rustc-dev-guide` addition:_ >When working on different tasks, you might need to switch between different bootstrap >configurations. >Sometimes you may want to keep an old configuration for future use. But saving raw config >values in >random files and manually copying and pasting them can quickly become messy, especially if >you have a >long history of different configurations. > >To simplify managing multiple configurations, you can create config extensions. > >For example, you can create a simple config file named `cross.toml`: > >```toml >[build] >build = "x86_64-unknown-linux-gnu" >host = ["i686-unknown-linux-gnu"] >target = ["i686-unknown-linux-gnu"] > > >[llvm] >download-ci-llvm = false > >[target.x86_64-unknown-linux-gnu] >llvm-config = "/path/to/llvm-19/bin/llvm-config" >``` > >Then, include this in your `bootstrap.toml`: > >```toml >include = ["cross.toml"] >``` > >You can also include extensions within extensions recursively. > >**Note:** In the `include` field, the overriding logic follows a right-to-left order. For example, in `include = ["a.toml", "b.toml"]`, extension `b.toml` overrides `a.toml`. Also, parent extensions always overrides the inner ones. try-job: x86_64-mingw-2

…iaskrgr Rollup of 7 pull requests Successful merges: - rust-lang#137454 (not lint break with label and unsafe block) - rust-lang#138934 (support config extensions) - rust-lang#139297 (Deduplicate & clean up Nix shell) - rust-lang#139834 (Don't canonicalize crate paths) - rust-lang#139978 (Add citool command for generating a test dashboard) - rust-lang#140000 (skip llvm-config in autodiff check builds, when its unavailable) - rust-lang#140007 (Disable has_thread_local on i686-win7-windows-msvc) r? `@ghost` `@rustbot` modify labels: rollup

…s, r=Kobzol support config extensions _Copied from the `rustc-dev-guide` addition:_ >When working on different tasks, you might need to switch between different bootstrap >configurations. >Sometimes you may want to keep an old configuration for future use. But saving raw config >values in >random files and manually copying and pasting them can quickly become messy, especially if >you have a >long history of different configurations. > >To simplify managing multiple configurations, you can create config extensions. > >For example, you can create a simple config file named `cross.toml`: > >```toml >[build] >build = "x86_64-unknown-linux-gnu" >host = ["i686-unknown-linux-gnu"] >target = ["i686-unknown-linux-gnu"] > > >[llvm] >download-ci-llvm = false > >[target.x86_64-unknown-linux-gnu] >llvm-config = "/path/to/llvm-19/bin/llvm-config" >``` > >Then, include this in your `bootstrap.toml`: > >```toml >include = ["cross.toml"] >``` > >You can also include extensions within extensions recursively. > >**Note:** In the `include` field, the overriding logic follows a right-to-left order. For example, in `include = ["a.toml", "b.toml"]`, extension `b.toml` overrides `a.toml`. Also, parent extensions always overrides the inner ones. try-job: x86_64-mingw-2

rustbot assigned jieyouxu Mar 25, 2025

rustbot added A-rustc-dev-guide Area: rustc-dev-guide S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Mar 25, 2025

rustbot assigned albertlarsan68 and unassigned jieyouxu Mar 26, 2025

rustbot assigned Kobzol and unassigned albertlarsan68 Mar 26, 2025

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 26, 2025

onur-ozkan force-pushed the extended-config-profiles branch from 6504079 to c7b6606 Compare April 1, 2025 08:44

onur-ozkan changed the title ~~support local bootstrap profiles~~ support config extensions Apr 1, 2025

onur-ozkan force-pushed the extended-config-profiles branch from c7b6606 to 13fcf1c Compare April 1, 2025 08:47

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 1, 2025

onur-ozkan force-pushed the extended-config-profiles branch from dd1536d to 586efa1 Compare April 1, 2025 09:00

onur-ozkan commented Apr 1, 2025

View reviewed changes

src/bootstrap/src/core/config/config.rs Outdated Show resolved Hide resolved

Kobzol reviewed Apr 1, 2025

View reviewed changes

src/doc/rustc-dev-guide/src/building/suggested.md Outdated Show resolved Hide resolved

onur-ozkan force-pushed the extended-config-profiles branch from 586efa1 to ad2e4a4 Compare April 1, 2025 14:30

onur-ozkan force-pushed the extended-config-profiles branch from f9ebd36 to 12de52b Compare April 18, 2025 09:17

This comment has been minimized.

Sign in to view

add coverage on config include logic

ac7d1be

Signed-off-by: onur-ozkan <[email protected]>

onur-ozkan force-pushed the extended-config-profiles branch from 12de52b to ac7d1be Compare April 18, 2025 12:29

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Apr 18, 2025

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 18, 2025

matthiaskrgr mentioned this pull request Apr 18, 2025

Rollup of 8 pull requests #140012

Closed

matthiaskrgr mentioned this pull request Apr 18, 2025

Rollup of 7 pull requests #140017

Open

ChrisDenton mentioned this pull request Apr 19, 2025

Rollup of 8 pull requests #140043

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

support config extensions #138934

support config extensions #138934

onur-ozkan commented Mar 25, 2025 •

edited

Loading

rustbot commented Mar 25, 2025

rustbot commented Mar 25, 2025

TimNN commented Mar 25, 2025 •

edited

Loading

jieyouxu commented Mar 26, 2025

Kobzol commented Mar 26, 2025

jieyouxu commented Mar 26, 2025 •

edited

Loading

onur-ozkan commented Mar 26, 2025

onur-ozkan commented Apr 1, 2025

TimNN commented Apr 1, 2025

rustbot commented Apr 1, 2025

onur-ozkan commented Apr 1, 2025

Kobzol left a comment

onur-ozkan commented Apr 18, 2025

bors commented Apr 18, 2025

This comment has been minimized.

bors commented Apr 18, 2025

onur-ozkan commented Apr 18, 2025

bors commented Apr 18, 2025

onur-ozkan commented Apr 18, 2025

bors commented Apr 18, 2025

Kobzol commented Apr 18, 2025

bors commented Apr 18, 2025

Kobzol commented Apr 18, 2025

support config extensions #138934

Are you sure you want to change the base?

support config extensions #138934

Conversation

onur-ozkan commented Mar 25, 2025 • edited Loading

rustbot commented Mar 25, 2025

rustbot commented Mar 25, 2025

TimNN commented Mar 25, 2025 • edited Loading

jieyouxu commented Mar 26, 2025

Kobzol commented Mar 26, 2025

jieyouxu commented Mar 26, 2025 • edited Loading

onur-ozkan commented Mar 26, 2025

onur-ozkan commented Apr 1, 2025

TimNN commented Apr 1, 2025

rustbot commented Apr 1, 2025

onur-ozkan commented Apr 1, 2025

Kobzol left a comment

Choose a reason for hiding this comment

onur-ozkan commented Apr 18, 2025

bors commented Apr 18, 2025

This comment has been minimized.

bors commented Apr 18, 2025

onur-ozkan commented Apr 18, 2025

bors commented Apr 18, 2025

onur-ozkan commented Apr 18, 2025

bors commented Apr 18, 2025

Kobzol commented Apr 18, 2025

bors commented Apr 18, 2025

Kobzol commented Apr 18, 2025

onur-ozkan commented Mar 25, 2025 •

edited

Loading

TimNN commented Mar 25, 2025 •

edited

Loading

jieyouxu commented Mar 26, 2025 •

edited

Loading