Skip to content

Improve docs #444

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 17, 2024
Merged

Improve docs #444

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6bdcc3c
Move subtree part of Readme into its own doc file
GuillaumeGomez Feb 17, 2024
cb14f43
Improve instructions to start working on the project
GuillaumeGomez Feb 17, 2024
79316d4
Split Readme even further
GuillaumeGomez Feb 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
252 changes: 29 additions & 223 deletions Readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,18 @@ A secondary goal is to check if using the gcc backend will provide any run-time
**This requires a patched libgccjit in order to work.
You need to use my [fork of gcc](https://github.com/antoyo/gcc) which already includes these patches.**

```bash
$ cp config.example.toml config.toml
```

If don't need to test GCC patches you wrote in our GCC fork, then the default configuration should
be all you need. You can update the `rustc_codegen_gcc` without worrying about GCC.

### Building with your own GCC version

If you wrote a patch for GCC and want to test it without this backend, you will need
to do a few more things.

To build it (most of these instructions come from [here](https://gcc.gnu.org/onlinedocs/jit/internals/index.html), so don't hesitate to take a look there if you encounter an issue):

```bash
Expand Down Expand Up @@ -51,18 +63,17 @@ $ make check-jit RUNTESTFLAGS="-v -v -v jit.exp=jit.dg/test-asm.cc"

**Put the path to your custom build of libgccjit in the file `config.toml`.**

If you followed the instructions exactly as written (ie, you have created a `gcc-build` folder
where gcc is built), the only thing you need to do is:
You now need to set the `gcc-path` value in `config.toml` with the result of this command:

```bash
$ cp config.example.toml config.toml
$ dirname $(readlink -f `find . -name libgccjit.so`)
```

But if you did something different, you also need to set the `gcc-path` value in `config.toml` with
the result of this command:
and to comment the `download-gccjit` setting:

```bash
$ dirname $(readlink -f `find . -name libgccjit.so`)
```toml
gcc-path = "[MY PATH]"
# download-gccjit = true
```

Then you can run commands like this:
Expand Down Expand Up @@ -128,224 +139,19 @@ $ LIBRARY_PATH="[gcc-path value]" LD_LIBRARY_PATH="[gcc-path value]" rustc +$(ca
<dd>Dump a C-like representation to /tmp/gccjit_dumps and enable debug info in order to debug this C-like representation.</dd>
</dl>

## Licensing

While this crate is licensed under a dual Apache/MIT license, it links to `libgccjit` which is under the GPLv3+ and thus, the resulting toolchain (rustc + GCC codegen) will need to be released under the GPL license.

However, programs compiled with `rustc_codegen_gcc` do not need to be released under a GPL license.

## Debugging

Sometimes, libgccjit will crash and output an error like this:

```
during RTL pass: expand
libgccjit.so: error: in expmed_mode_index, at expmed.h:249
0x7f0da2e61a35 expmed_mode_index
../../../gcc/gcc/expmed.h:249
0x7f0da2e61aa4 expmed_op_cost_ptr
../../../gcc/gcc/expmed.h:271
0x7f0da2e620dc sdiv_cost_ptr
../../../gcc/gcc/expmed.h:540
0x7f0da2e62129 sdiv_cost
../../../gcc/gcc/expmed.h:558
0x7f0da2e73c12 expand_divmod(int, tree_code, machine_mode, rtx_def*, rtx_def*, rtx_def*, int)
../../../gcc/gcc/expmed.c:4335
0x7f0da2ea1423 expand_expr_real_2(separate_ops*, rtx_def*, machine_mode, expand_modifier)
../../../gcc/gcc/expr.c:9240
0x7f0da2cd1a1e expand_gimple_stmt_1
../../../gcc/gcc/cfgexpand.c:3796
0x7f0da2cd1c30 expand_gimple_stmt
../../../gcc/gcc/cfgexpand.c:3857
0x7f0da2cd90a9 expand_gimple_basic_block
../../../gcc/gcc/cfgexpand.c:5898
0x7f0da2cdade8 execute
../../../gcc/gcc/cfgexpand.c:6582
```

To see the code which causes this error, call the following function:

```c
gcc_jit_context_dump_to_file(ctxt, "/tmp/output.c", 1 /* update_locations */)
```

This will create a C-like file and add the locations into the IR pointing to this C file.
Then, rerun the program and it will output the location in the second line:

```
libgccjit.so: /tmp/something.c:61322:0: error: in expmed_mode_index, at expmed.h:249
```

Or add a breakpoint to `add_error` in gdb and print the line number using:

```
p loc->m_line
p loc->m_filename->m_buffer
```

To print a debug representation of a tree:

```c
debug_tree(expr);
```

(defined in print-tree.h)

To print a debug representation of a gimple struct:

```c
debug_gimple_stmt(gimple_struct)
```

To get the `rustc` command to run in `gdb`, add the `--verbose` flag to `cargo build`.

To have the correct file paths in `gdb` instead of `/usr/src/debug/gcc/libstdc++-v3/libsupc++/eh_personality.cc`:

Maybe by calling the following at the beginning of gdb:

```
set substitute-path /usr/src/debug/gcc /path/to/gcc-repo/gcc
```

TODO(antoyo): but that's not what I remember I was doing.

### `failed to build archive` error

When you get this error:

```
error: failed to build archive: failed to open object file: No such file or directory (os error 2)
```

That can be caused by the fact that you try to compile with `lto = "fat"`, but you didn't compile the sysroot with LTO.
(Not sure if that's the reason since I cannot reproduce anymore. Maybe it happened when forgetting setting `FAT_LTO`.)

### ld: cannot find crtbegin.o

When compiling an executable with libgccijt, if setting the `*LIBRARY_PATH` variables to the install directory, you will get the following errors:
## Extra documentation

```
ld: cannot find crtbegin.o: No such file or directory
ld: cannot find -lgcc: No such file or directory
ld: cannot find -lgcc: No such file or directory
libgccjit.so: error: error invoking gcc driver
```

To fix this, set the variables to `gcc-build/build/gcc`.

### How to debug GCC LTO

Run do the command with `-v -save-temps` and then extract the `lto1` line from the output and run that under the debugger.

### How to send arguments to the GCC linker

```
CG_RUSTFLAGS="-Clink-args=-save-temps -v" ../y.sh cargo build
```

### How to see the personality functions in the asm dump

```
CG_RUSTFLAGS="-Clink-arg=-save-temps -v -Clink-arg=-dA" ../y.sh cargo build
```

### How to see the LLVM IR for a sysroot crate

```
cargo build -v --target x86_64-unknown-linux-gnu -Zbuild-std
# Take the command from the output and add --emit=llvm-ir
```

### To prevent the linker from unmangling symbols

Run with:

```
COLLECT_NO_DEMANGLE=1
```

### How to use a custom-build rustc

* Build the stage2 compiler (`rustup toolchain link debug-current build/x86_64-unknown-linux-gnu/stage2`).
* Clean and rebuild the codegen with `debug-current` in the file `rust-toolchain`.

### How to install a forked git-subtree

Using git-subtree with `rustc` requires a patched git to make it work.
The PR that is needed is [here](https://github.com/gitgitgadget/git/pull/493).
Use the following instructions to install it:

```bash
git clone [email protected]:tqc/git.git
cd git
git checkout tqc/subtree
make
make install
cd contrib/subtree
make
cp git-subtree ~/bin
```

Then, do a sync with this command:

```bash
PATH="$HOME/bin:$PATH" ~/bin/git-subtree push -P compiler/rustc_codegen_gcc/ ../rustc_codegen_gcc/ sync_branch_name
cd ../rustc_codegen_gcc
git checkout master
git pull
git checkout sync_branch_name
git merge master
```

To send the changes to the rust repo:

```bash
cd ../rust
git pull origin master
git checkout -b subtree-update_cg_gcc_YYYY-MM-DD
PATH="$HOME/bin:$PATH" ~/bin/git-subtree pull --prefix=compiler/rustc_codegen_gcc/ https://github.com/rust-lang/rustc_codegen_gcc.git master
git push

# Immediately merge the merge commit into cg_gcc to prevent merge conflicts when syncing from rust-lang/rust later.
PATH="$HOME/bin:$PATH" ~/bin/git-subtree push -P compiler/rustc_codegen_gcc/ ../rustc_codegen_gcc/ sync_branch_name
```

TODO: write a script that does the above.

https://rust-lang.zulipchat.com/#narrow/stream/301329-t-devtools/topic/subtree.20madness/near/258877725

### How to use [mem-trace](https://github.com/antoyo/mem-trace)

`rustc` needs to be built without `jemalloc` so that `mem-trace` can overload `malloc` since `jemalloc` is linked statically, so a `LD_PRELOAD`-ed library won't a chance to intercept the calls to `malloc`.
More specific documentation is available in the [`doc`](./doc) folder:

### How to generate GIMPLE
* [Common errors](./doc/errors.md)
* [Debugging GCC LTO](./doc/debugging-gcc-lto.md)
* [Debugging libgccjit](./doc/debugging-libgccjit.md)
* [Git subtree sync](./doc/subtree.md)
* [List of useful commands](./doc/tips.md)
* [Send a patch to GCC](./doc/sending-gcc-patch.md)

If you need to check what gccjit is generating (GIMPLE), then take a look at how to
generate it in [gimple.md](./doc/gimple.md).

### How to build a cross-compiling libgccjit

#### Building libgccjit

* Follow the instructions on [this repo](https://github.com/cross-cg-gcc-tools/cross-gcc).

#### Configuring rustc_codegen_gcc

* Run `./y.sh prepare --cross` so that the sysroot is patched for the cross-compiling case.
* Set the path to the cross-compiling libgccjit in `gcc-path` (in `config.toml`).
* Make sure you have the linker for your target (for instance `m68k-unknown-linux-gnu-gcc`) in your `$PATH`. Currently, the linker name is hardcoded as being `$TARGET-gcc`. Specify the target when building the sysroot: `./y.sh build --target-triple m68k-unknown-linux-gnu`.
* Build your project by specifying the target: `OVERWRITE_TARGET_TRIPLE=m68k-unknown-linux-gnu ../y.sh cargo build --target m68k-unknown-linux-gnu`.

If the target is not yet supported by the Rust compiler, create a [target specification file](https://docs.rust-embedded.org/embedonomicon/custom-target.html) (note that the `arch` specified in this file must be supported by the rust compiler).
Then, you can use it the following way:

* Add the target specification file using `--target` as an **absolute** path to build the sysroot: `./y.sh build --target-triple m68k-unknown-linux-gnu --target $(pwd)/m68k-unknown-linux-gnu.json`
* Build your project by specifying the target specification file: `OVERWRITE_TARGET_TRIPLE=m68k-unknown-linux-gnu ../y.sh cargo build --target path/to/m68k-unknown-linux-gnu.json`.

If you get the following error:
## Licensing

```
/usr/bin/ld: unrecognised emulation mode: m68kelf
```
While this crate is licensed under a dual Apache/MIT license, it links to `libgccjit` which is under the GPLv3+ and thus, the resulting toolchain (rustc + GCC codegen) will need to be released under the GPL license.

Make sure you set `gcc-path` (in `config.toml`) to the install directory.
However, programs compiled with `rustc_codegen_gcc` do not need to be released under a GPL license.
3 changes: 3 additions & 0 deletions doc/debugging-gcc-lto.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# How to debug GCC LTO

Run do the command with `-v -save-temps` and then extract the `lto1` line from the output and run that under the debugger.
74 changes: 74 additions & 0 deletions doc/debugging-libgccjit.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Debugging libgccjit

Sometimes, libgccjit will crash and output an error like this:

```
during RTL pass: expand
libgccjit.so: error: in expmed_mode_index, at expmed.h:249
0x7f0da2e61a35 expmed_mode_index
../../../gcc/gcc/expmed.h:249
0x7f0da2e61aa4 expmed_op_cost_ptr
../../../gcc/gcc/expmed.h:271
0x7f0da2e620dc sdiv_cost_ptr
../../../gcc/gcc/expmed.h:540
0x7f0da2e62129 sdiv_cost
../../../gcc/gcc/expmed.h:558
0x7f0da2e73c12 expand_divmod(int, tree_code, machine_mode, rtx_def*, rtx_def*, rtx_def*, int)
../../../gcc/gcc/expmed.c:4335
0x7f0da2ea1423 expand_expr_real_2(separate_ops*, rtx_def*, machine_mode, expand_modifier)
../../../gcc/gcc/expr.c:9240
0x7f0da2cd1a1e expand_gimple_stmt_1
../../../gcc/gcc/cfgexpand.c:3796
0x7f0da2cd1c30 expand_gimple_stmt
../../../gcc/gcc/cfgexpand.c:3857
0x7f0da2cd90a9 expand_gimple_basic_block
../../../gcc/gcc/cfgexpand.c:5898
0x7f0da2cdade8 execute
../../../gcc/gcc/cfgexpand.c:6582
```

To see the code which causes this error, call the following function:

```c
gcc_jit_context_dump_to_file(ctxt, "/tmp/output.c", 1 /* update_locations */)
```

This will create a C-like file and add the locations into the IR pointing to this C file.
Then, rerun the program and it will output the location in the second line:

```
libgccjit.so: /tmp/something.c:61322:0: error: in expmed_mode_index, at expmed.h:249
```

Or add a breakpoint to `add_error` in gdb and print the line number using:

```
p loc->m_line
p loc->m_filename->m_buffer
```

To print a debug representation of a tree:

```c
debug_tree(expr);
```

(defined in print-tree.h)

To print a debug representation of a gimple struct:

```c
debug_gimple_stmt(gimple_struct)
```

To get the `rustc` command to run in `gdb`, add the `--verbose` flag to `cargo build`.

To have the correct file paths in `gdb` instead of `/usr/src/debug/gcc/libstdc++-v3/libsupc++/eh_personality.cc`:

Maybe by calling the following at the beginning of gdb:

```
set substitute-path /usr/src/debug/gcc /path/to/gcc-repo/gcc
```

TODO(antoyo): but that's not what I remember I was doing.
27 changes: 27 additions & 0 deletions doc/errors.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Common errors

This file lists errors that were encountered and how to fix them.

### `failed to build archive` error

When you get this error:

```
error: failed to build archive: failed to open object file: No such file or directory (os error 2)
```

That can be caused by the fact that you try to compile with `lto = "fat"`, but you didn't compile the sysroot with LTO.
(Not sure if that's the reason since I cannot reproduce anymore. Maybe it happened when forgetting setting `FAT_LTO`.)

### ld: cannot find crtbegin.o

When compiling an executable with libgccijt, if setting the `*LIBRARY_PATH` variables to the install directory, you will get the following errors:

```
ld: cannot find crtbegin.o: No such file or directory
ld: cannot find -lgcc: No such file or directory
ld: cannot find -lgcc: No such file or directory
libgccjit.so: error: error invoking gcc driver
```

To fix this, set the variables to `gcc-build/build/gcc`.
Loading