Clarification on i128 and u128 atomic operation linkage

[jbaublitz]

I've been working on an example kernel module in Rust for educational purposes, and I bumped into some interesting behavior at link time. The working process is codified here, but prior to this I had some problems with linking that may be of interest to you.

Current state

• Using kbuild
• Extracting object files from generated static library to use in kbuild (kbuild doesn't seem to support .a files)
• Use xargo to cross compile to the x86_64-linux-kernel target to make sure libcore doesn't have invalid relocation table entries
• Not using compiler-builtins

This all works very nicely and there are no longer unresolved symbols even without this crate.

Previous state:

• Using kbuild
• Emit .o file for the x86_64-unknown-linux-gnu target using some specific flags: cargo rustc --release -- -C relocation-model=static -C code-model=kernel -Z plt=y --emit=obj=rust.o

This resolved a couple previous problems I bumped into by compiling to the kernel address space and avoiding GOT relocation entries for the .o file itself. However, I ran into a really strange issue when I tweaked this process slightly and started emitting a .a file so that I could link to and use libcore in my kernel module. I extracted the bundled .o files, and without compiler-builtins, I got unresolved symbol errors when linking them together and attempting to insert the kernel module. After I included compiler-builtins in Cargo.toml, all of the unresolved symbol errors went away, but two new ones came up. They were both __sync_*_16 symbols which I ultimately determined were builtins for u128 and i128 atomic operations that seem to be styled after the gcc-specific builtins such as __sync_lock_test_and_set.

I ultimately have two questions:

1. While the simplest solution here is obviously what I ended up doing above after discovering there's an x86_64-linux-kernel target, I'm a little confused as to why these symbols are not getting resolved through this crate at link time. Do you have any input on why that might be? Could this problem ever pop up when using cargo to generate executables or do you think this may just be a problem with the way that I was going about the build process due to my requirements?
2. Why does the compiler-builtins dependency seem to be unnecessary when building for the kernel target?

[alexcrichton]

Thanks for the questions! Unfortunately I probably don't have great answers for you :(

In general compiler-builtins is all about "whatever hacks are necessary to get everything working". Its existence is supposed to be entirely hidden from everyone, and is 100% just an implementation detail of the compiler. In that sense all of the symbols are just there for the compiler to call out to when it's otherwise lowering normal language operations. In that sense when an executable references a symbol in compiler-builtins is a compiler implementation detail, and it varies widely based on platform, codegen options, target, etc.

In terms of resolving these symbols, they're typically compiled and linked as such:

• Typically all symbols are compiled to their own object file and have hidden visibility. For us in Rust though we have multiple symbols per object, but they still have hidden visibility.
• Otherwise the linker is invoked with the archive file typically listed at the very end of the linker command line, so any unresolved symbols are caught by compiler-builtins. (in Rust's case it's an rlib)

So I think a lot of this boils down to kbuild/build systems/etc. It may also depend on different compilers. For example Rust code linked with gcc may not be finding the right compiler builtins. Alternatively the extraction of dealing with a staticlib may be iffy. Rust staticlibs should include all of compiler-builtins's object files, so the should be "just resolved" but it sounds like there's shenanigans which may prevent this.

As for why they may not be necessary for the kernel target, it sounds like that may also be related to the build system perhaps? Do you have some build system scripts/commands I could take a look at to see if they match the expected usage of compiler-builtins?

[jbaublitz]

Interestingly enough, I went to reproduce the issue in a gist and when I tested building it, I got a warning from xargo that indicated compiler_builtins was being included implicitly.

warning: dependency (compiler_builtins) specified without providing a local path, Git repository, or version to use. This will be considered an error in future versions

After that, I changed back to cargo as that's what I was originally using, and I seem to have tracked down the problem thanks to your response. It appears that xargo will pull in compiler_builtins regardless of the target. cargo on the other hand requires an explicit inclusion of compiler_builtins and even then, I get these two errors at kbuild link time:

WARNING: "__sync_lock_test_and_set_16" undefined!
WARNING: "__sync_val_compare_and_swap_16" undefined!

Ultimately, I think using xargo is the simple solution here, even more so because even if the above symbols were to get resolved, the kernel will fail to load the module if it's not linked with a libcore that's been compiled with a different relocation table generation strategy from what is typically used (it should use PLT instead of GOT). I'm still a little confused as to why those two symbols are not resolved in cargo with compiler_builtins and here's a link to a Makefile/Cargo.toml combination in a github gist to use with my kernel module in case you want to try to reproduce it. I'm uncertain if this is maybe an unrelated but relevant error affecting compiler_builtins. I can also imagine that this might never show up in any correctly configured situation if these symbols get optimized out based on various compilation flags that xargo sets properly, so this may also be a non-issue.

Regardless, thanks for the input! I'll leave this open for now, but feel free to close it if you determine that the errors above are simply a product of the misuse of cargo here combined with kbuild.