static binaries failed on macOS from version 15~18

[shenxianpeng]

I tested the new pre-release master-a13caef5, the static binaries on macOS still has the problem from version 15~18.

dyld[3039]: Library not loaded: /usr/local/opt/zstd/lib/libzstd.1.dylib
  Referenced from: <54D844F3-D2D4-323E-9B2B-6EC303AF4B79> /Users/runner/work/clang-tools-static-binaries/clang-tools-static-binaries/clang-format-16_macosx-amd64
  Reason: tried: '/usr/local/opt/zstd/lib/libzstd.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/zstd/lib/libzstd.1.dylib' (no such file), '/usr/local/opt/zstd/lib/libzstd.1.dylib' (no such file), '/usr/local/lib/libzstd.1.dylib' (no such file), '/usr/lib/libzstd.1.dylib' (no such file, not in dyld cache)
/Users/runner/work/_temp/31e31703-4e2c-489d-84ab-c0645130b52c.sh: line 5:  3039 Abort trap: 6           ./clang-format-16_macosx-amd64 --version

More details please see the test job https://github.com/cpp-linter/clang-tools-static-binaries/actions/runs/9353719255/job/25744898037#step:3:40

Originally posted by @shenxianpeng in #16 (comment)

[2bndy5]

I was looking at what recently changed in the MacOS runners, and I found that XCode v15.4.0 was updated to a stable release (see this PR). Maybe there was some other change in the std lib that is included with XCode v15.4.0?

EDIT: See XCode changelog here

[shenxianpeng]

I don't understand why only versions 15~18 were affected not all versions on MacOS. maybe it is also related to the llvm-project source code from version 15 to 18?

[2bndy5]

I'm not sure why the binaries are linked to a zstd library. Recently, the macos-14 image had bumped zstd from v1.5.5 to v1.5.6.

[shenxianpeng]

maybe I can try to create a link for zstd lib

sudo ln -s /opt/homebrew/Cellar/zstd/1.5.6/lib/libzstd.1.5.6.dylib /usr/lib/libzstd.1.dylib

# or just

sudo ln -s /opt/homebrew/Cellar/zstd/1.*/lib/libzstd.1.*.dylib /usr/lib/libzstd.1.dylib

[2bndy5]

The wildcard might be better, but you'd have to recreate that symbol link on every Mac machine that uses the static binary.

[2bndy5]

I skimmed the zstd lib's manual, and I found

Advanced experimental functions can be accessed using
#define ZSTD_STATIC_LINKING_ONLY before including zstd.h.

Advanced experimental APIs should never be used with a dynamically-linked
library. They are not "stable"; their definitions or signatures may change in
the future. Only static linking is allowed.

I haven't checked the workflow to see if mac builds use -DZSTD_STATIC_LINKING_ONLY=1

[shenxianpeng]

use -DZSTD_STATIC_LINKING_ONLY=1 works 👍

[shenxianpeng]

From the test the following issue still occurs when running clang-format --version command, so I reopened this ticket.

== Output clang-format version
dyld[3452]: Library not loaded: /usr/local/opt/zstd/lib/libzstd.1.dylib
  Referenced from: <182FA4B3-05C4-32D1-B379-F93C1FA28DC0> /Users/runner/work/clang-tools-static-binaries/clang-tools-static-binaries/clang-format-15_macosx-amd64
  Reason: tried: '/usr/local/opt/zstd/lib/libzstd.1.dylib' (no such file), '/System/Volumes/Preboot/Cryptexes/OS/usr/local/opt/zstd/lib/libzstd.1.dylib' (no such file), '/usr/local/opt/zstd/lib/libzstd.1.dylib' (no such file), '/usr/local/lib/libzstd.1.dylib' (no such file), '/usr/lib/libzstd.1.dylib' (no such file, not in dyld cache)
/Users/runner/work/_temp/d23ec023-0cb7-410c-9fcd-8c17b55eae17.sh: line 5:  3452 Abort trap: 6           ./clang-format-15_macosx-amd64 --version
Error: Process completed with exit code 134.

It's my bad, use -DZSTD_STATIC_LINKING_ONLY=1 does not make the build fail, my brain just assumed the problem was solved 😕

[2bndy5]

Well, we haven't released the new builds yet. I don't think the current test CI uses pre-releases. So I think that error is just from running the last (erroneous) release's builds, specifically the release that github thinks is the "latest": https://github.com/cpp-linter/clang-tools-static-binaries/releases/latest.

clang-tools-static-binaries/.github/workflows/test.yml

Line 33 in 2e4f8c1

gh release download --pattern 'clang-format-${{ env.bin_suffix }}'

gh release download -h

Download assets from a GitHub release.

Without an explicit tag name argument, assets are downloaded from the
latest release in the project. In this case, `--pattern` or `--archive`
is required.


USAGE
  gh release download [<tag>] [flags]

FLAGS
  -A, --archive format        Download the source code archive in the specified format (zip or tar.gz)
      --clobber               Overwrite existing files of the same name
  -D, --dir directory         The directory to download files into (default ".")
  -O, --output file           The file to write a single asset to (use "-" to write to standard output)
  -p, --pattern stringArray   Download only assets that match a glob pattern
      --skip-existing         Skip downloading when files of the same name exist

INHERITED FLAGS
      --help                     Show help for command
  -R, --repo [HOST/]OWNER/REPO   Select another repository using the [HOST/]OWNER/REPO format

EXAMPLES
  # download all assets from a specific release
  $ gh release download v1.2.3

  # download only Debian packages for the latest release
  $ gh release download --pattern '*.deb'

  # specify multiple file patterns
  $ gh release download -p '*.deb' -p '*.rpm'

  # download the archive of the source code for a release
  $ gh release download v1.2.3 --archive=zip

LEARN MORE
  Use `gh <command> <subcommand> --help` for more information about a command.
  Read the manual at https://cli.github.com/manual

We could add a tag named latest and keep moving the tag when we publish a release (or pre-release). That way we can have the test CI use gh release download latest <tool-name>.

---

For proof that the seg fault was fixed see this CI run for cpp-linter/clang-tools-pip#99

[shenxianpeng]

Thank you for pointing out @2bndy5

I have added tag as input for the test CI, and tested the pre-release build successfully.

clang-tools-static-binaries/.github/workflows/test.yml

Line 39 in 1bcb98a

gh release download ${{ inputs.tag }} --pattern 'clang-format-${{ env.bin_suffix }}'