Fix nondeterminism in macOS packages

[zombiezen]

While creating the macOS userspace (#6), I observed that most of the macOS package outputs are non-deterministic. Some of this was 256lights/zb#134, but even for non-self-referencing packages, I believe the difference in compile paths triggers different UUIDs. There is a probably a way of addressing that with linker flags. There may be other sources of nondeterminism.

This issue tracks going through and removing these sources of nondeterminism for macOS specifically.

[winterqt]

This should be a pretty simple fix with modern versions of macOS’ linker - I’d give away the “secret” but I want an excuse to contribute. 😉

I’ll send a PR as soon as I can get a dev env up. :)

(Super cool project btw!)

[zombiezen]

Sounds great, thank you! 💜 Standard library on macOS is currently very-not-hermetic and is using the host's Xcode CLI tools, so you should have a recent linker to work with. 👍

[winterqt]

Do you have a reproducible (ironic) example of this nondeterminism? Cannot reproduce with:

zb build hello.lua#hello
zb store object delete /Users/winter/zb/store/kb5x9p7s3gap7c628sbn50aa6phwcmyx-hello
zb build hello.lua#hello

[zombiezen]

I would have expected it to reproduce with those steps, but perhaps it has something to do with the build process? Here's an example I was seeing:

url='https://github.com/256lights/zb-stdlib/releases/download/v0.1.1/zb-stdlib-v0.1.1.tar.gz#packages/bash/bash.lua:bootstrap' &&
path1="$(zb build "$url" 2> /dev/null)" &&
echo "path1=$path1" &&
sudo zb store object rm "$path1" &&
path2="$(zb build "$url" 2> /dev/null)" &&
echo "path2=$path2"

Version info:

% zb version
zb version 0.1.0
System:       aarch64-apple-macos
CPUs:         12
OS:           macOS 15.5 (build 24F74)

[winterqt]

Bash specifically is due to build dir references in debug symbols.

diff --git a/packages/bash/build.sh b/packages/bash/build.sh
index 6e344eac87..1a8bb1da11 100644
--- a/packages/bash/build.sh
+++ b/packages/bash/build.sh
@@ -31,3 +31,4 @@
 esac
 make "-j${ZB_BUILD_CORES:-1}"
 make install
+command -v strip && echo stripping && strip "$out/bin/bash"

This diff, while not ideal, does fix it. #18 (/an equivalent to the above) would likely fix the others.

Have you considered adding a build dir ref checker to zb?

[zombiezen]

Bash specifically is due to build dir references in debug symbols.

Interesting. Yeah, adding a strip phase to the builder seems like a good idea.

Have you considered adding a build dir ref checker to zb?

I hadn't considered. I'd imagine that would require some thought: the Linux sandbox always uses /build as the build directory. If we want to go in that direction, we'd have to introduce some randomness to reduce false positives.