Object file (lse.o) with __aarch64_have_lse_atomics symbol included several times in static library

[fxcoudert]

Hi, I'm a Homebrew maintainer, trying to build librsvg on macOS with the latest rust. The issue encountered is reported at librsvg (https://gitlab.gnome.org/GNOME/librsvg/-/issues/787) but my investigation makes me think it's a compiler-builtins issue.

The issue in short is this: a static library file, created by cargo in this way:

	PKG_CONFIG_ALLOW_CROSS=1						\
	PKG_CONFIG='/opt/homebrew/Library/Homebrew/shims/mac/super/pkg-config'						\
	CARGO_TARGET_DIR=/private/tmp/librsvg-20211127-15889-8ebs99/librsvg-2.50.7/target					\
	cargo --locked build   --release \
	&& cd /private/tmp/librsvg-20211127-15889-8ebs99/librsvg-2.50.7 && /bin/sh ./libtool --silent --tag=CC   --mode=link clang  -g -O2   -o librsvg_c_api.la _rsvg_dummy.lo && mv /private/tmp/librsvg-20211127-15889-8ebs99/librsvg-2.50.7/target/release/librsvg_c_api.a .libs/librsvg_c_api.a

The librsvg_c_api.a file that is produced contains many times the same object file (lse.o), which I think comes from this project, because of:

compiler-builtins/build.rs

Line 510 in ec8b848

sources.extend(&[("__aarch64_have_lse_atomics", "cpu_model.c")]);

This, in turn, leads to build failure at the next step, because a static library is supposed to have object files with unique names (libtool enforces that).

I'm not a Rust user myself, just trying to help debug this issue and move forward so we can ship this new version. So I hope my deductions are valid…

[Amanieu]

The actual source is here:

compiler-builtins/build.rs

Line 526 in ec8b848

fn build_aarch64_out_of_line_atomics_libraries(builtins_dir: &Path, cfg: &cc::Build) {

Essentially we compile the same lse.S assembly source file with different combinations of pre-processor options to produce many different versions of lse.o. If you look at each lse.o then you will see that they each define a different symbol, so there should be no conflict.

[fxcoudert]

ar allows you to create archives where multiple members have the same name, indeed, but several tools reject those. libtool, in particular, will error out (by design) whenever it encounters such a case: the static library that is created from these multiple lse.o files is therefore unusable:

libtool:   error: object name conflicts in archive: .libs/librsvg-2.lax/librsvg_c_api.a//private/tmp/librsvg-20210901-98292-1of4fdi/librsvg-2.50.7/./.libs/librsvg_c_api.a

Given that libtool has been around for decades, is very widely used, enforces this requirement without issue in all the other software I know… maybe it would be better to avoid this situation altogether.

[fxcoudert]

I don't know rust, as I stated, but there is clearly a deliberate effort made here to avoid having all of these object files in the same library:

compiler-builtins/build.rs

Line 550 in ec8b848

"liboutline_atomic_helper_{}{}_{}.a",

It should be possible to similarly name the object files lse_{instruction_type}_{size}_{model_name}.o, getting rid of the issue.

[bjorn3]

I think it uses different static library names due to the cc crate requiring the same configuration for the entire static library. In this case it is necessary to compile the same source file with multiple configuration options, so it needs to use multiple static libraries to let cc do this.

[bjorn3]

I think the best fix would be to make rustc rename object files when bundelling static libraries to avoid conflicts. This is the only way to ensure that object file names are always unique inside the produced static library even if multiple crates include a static library from unrelated projects that use the same object file name like for example util.o.

[Amanieu]

The cc crate that we use for invoking a C compiler doesn't allow specifying the object file name, it's derived from the source file name. I think the best way to solve this would be to create copies of lse.S in OUT_DIR with different names and then compiling those. I am preparing a PR for this.

[carlocab]

We applied the patch from #444 to our build of Rust (see Homebrew/homebrew-core#90103), but we're still seeing the exact same error:

libtool:   error: object name conflicts in archive: .libs/librsvg-2.lax/librsvg_c_api.a//private/tmp/librsvg-20210901-98292-1of4fdi/librsvg-2.50.7/./.libs/librsvg_c_api.a

Did I do this wrong, or...?

[Amanieu]

This patch is included in compiler-builtins 0.1.55. Make sure that the Cargo.lock from rust-lang/rust that you are using has at least that version for the compiler-builtins crate.

[carlocab]

I see. What I did was patch the source in vendor/compiler_builtins in the release tarball. Are the sources there not actually used during the build?

[Amanieu]

Are you passing --enable-vendor to ./configure?

[carlocab]

We are not. But it seems passing it is also complicated, as we need to update the checksums of the vendored sources. Patching the Cargo.lock is simpler.

We probably will be using --enable-vendor when we're no longer carrying the patch though. Is there any downside to this? It seems like it should be the default in a source distribution, but I assume there is a reason why it isn't.

[Amanieu]

--enable-vendor is mostly there to support build environments that don't allow fetching crates from the internet. Otherwise the end results should be practically identical.