`-L` and `-I` are interpreted relative to the `$SDKROOT` by the
Darwin toolchain, so we have to avoid filtering out such paths in the
purity filter hacks in order to not break e.g. the .NET Core build
system. It’s also just the correct thing to do for the platform.
This reduces code duplication, makes the SDK variables available
earlier in the wrappers, and makes the behaviour between the two
wrappers more consistent.
This changes ld-wrapper to use a temporary file for the response file
passed to ld instead of using process substitution.
ld64 does not handle long command-lines when reading from the response
file, which defeats the point of using a response file to handle long
command-lines. cctools-port was patched to work around this, but nixpkgs
is now using Apple’s source release directly instead of the port.
Since it’s preferable not to patch Apple’s release heavily (to reduce
the difficulty of updating to new versions and to match upstream’s
behavior), use the approach that was adopted in cc-wrapper to work
around issues with response files in newer versions of clang.
Related PRs (cctools-port):
- https://github.com/NixOS/nixpkgs/pull/213831
- https://github.com/tpoechtrager/cctools-port/pull/132
Related PRs (cc-wrapper):
- https://github.com/NixOS/nixpkgs/pull/245282
- https://github.com/NixOS/nixpkgs/pull/258608
LLD supports Windows-style linker arguments, but these previously
triggered purity check false positives, because it saw that they
started with a '/' and assumed they were paths.
This tweaks the path detection to allow through certain values that
could be paths, but are much more likely to be LINK.EXE-style flags.
The risk of false negatives here is low — the only things we'd now
fail to catch would be attempts to link with libraries in the root
directory, which doesn't happen in practice.
We also teach the wrapper how to apply its purity checks to library
paths specified with the /LIBPATH: option.
Tested that paths we expect to be rejected (like /lib/libfoo.so) still
are.
NixOS/nixpkgs#146275 has more discussion on this; the abridged version
is that `lld` defaults to using `--build-id=fast` while GNU `ld` defaults
to `--build-id=sha1`. These differ in length and so
`separate-debug-info.sh`, as of this writing, errors on `lld`'s shorter
`--build-id=fast`-generated hashes.
`lld` offers the following `build-id` styles:
- UUID (random; fast but bad for reproducibility)
- fast (xxhash; fast but shorter hashes)
- user provided hexstring
- SHA1
- MD5
GNU `ld` supports the latter three options, `mold` supports all of these
plus SHA256.
UUID is out because it's not reproducible, fast isn't supported by GNU
`ld`
Using a nix provided (sourced from the output base hash) hash as the
`build-id` seems tempting but would require a little extra work
(we have to include some characteristic of the binary being hashed
so that binaries within a derivation still have unique hashes; it
seems easy to get this wrong; i.e. a path based approach would make
two otherwise identical binaries that reside at different paths have
different `build-id` hashes)
That leaves SHA1 and MD5 and GNU `ld` already defaults to the former.
This commit adds `$NIX_BUILD_ID_STYLE` as an escape hatch, in case any
packages have strong opinions about which hash to use.
----
Note that if/when NixOS/nixpkgs#146275 goes through, this change can be
reverted if linker speed is a priority.
The motivation behind this is to alleviate the problem
described in https://github.com/NixOS/nixpkgs/issues/41340.
I'm not sure if this completely fixes the problem, but it
eliminates one more area where we can exceed command line
length limits.
This is essentially the same change as in #112449,
except for `ld-wrapper.sh` instead of `cc-wrapper.sh`.
However, that change alone was not enough; on macOS the
`ld` provided by `darwin.cctools` fails if you use process
substitution to generate the response file, so I put up a
PR to fix that:
https://github.com/tpoechtrager/cctools-port/pull/131
… and I included a patch referencing that fix so that the
new `ld-wrapper` still works on macOS.
Without this change cross-built gcc fails to detect stack protector style:
$ nix log -f pkgs/stdenv/linux/make-bootstrap-tools-cross.nix powerpc64le.bootGCC | fgrep __stack_chk_fail
checking __stack_chk_fail in target C library... no
checking __stack_chk_fail in target C library... no
It happens because gcc treats search paths differently:
https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/configure.ac;h=446747311a6aec3c810ad6aa4190f7bd383b94f7;hb=HEAD#l2458
if test x$host != x$target || test "x$TARGET_SYSTEM_ROOT" != x ||
test x$build != x$host || test "x$with_build_sysroot" != x; then
...
if test "x$with_build_sysroot" != "x"; then
target_header_dir="${with_build_sysroot}${native_system_header_dir}"
elif test "x$with_sysroot" = x; then
target_header_dir="${test_exec_prefix}/${target_noncanonical}/sys-include"
elif test "x$with_sysroot" = xyes; then
target_header_dir="${test_exec_prefix}/${target_noncanonical}/sys-root${native_system_header_dir}"
else
target_header_dir="${with_sysroot}${native_system_header_dir}"
fi
else
target_header_dir=${native_system_header_dir}
fi
By passing --with-build-sysroot=/ we trick cross-case to use
`target_header_dir="${with_sysroot}${native_system_header_dir}"`
which makes it equivalent to non-cross
`target_header_dir="${with_build_sysroot}${native_system_header_dir}"`
Tested the following setups:
- cross-compiler without libc headers (powerpc64le-static)
- cross-compiler with libc headers (powerpc64le-debug)
- cross-build compiler with libc headers (powerpc64le bootstrapTools)
Before the change only 2 of 3 compilers detected libc headers.
After the change all 3 compilers detected libc headers.
For darwin we silently ignore '-syslibroot //' argument as it does not
introduce impurities.
While at it dropped mingw special case for no-libc build. Before the change
we passed both '--without-headers --with-native-system-headers-dir' for
no-libc gcc-static builds. This tricked darwin builds to find sys/sdt.h
and fail inhibid_libc builds. Now all targets avoid passing native headers
for gcc-static builds.
While at it fixed correct headers passing to
--with-native-system-headers-dir= in host != target case: we were passing
host's headers where intention was to pass target's headers.
Noticed the mismatch as a build failure on pkgsCross.powernv.stdenv.cc
on darwin where `sys/sdt.h` is present in host's headers (libSystem)
but not target's headers (`glibc`).
Co-authored-by: Adam Joseph <54836058+amjoseph-nixpkgs@users.noreply.github.com>
As far as I can tell, this has never actually done anything, as
LDEMULATION is not exported. I tried exporting it and builds broke,
and as it doesn't seem to have caused any problems as a noop all these
years it didn't seem worth investigating further.
So far we've ignored response files in arguments, and did not
check linkType against expanded parameters. This means if
we have `-static` in a @reponse-file, linkType will not be
set to `-static` as we never check against the expanded arguments
from response files.
Instead of always supplying flags, apply the flags as defaults. Use
clang's native flags instead of lifting the linker flags from binutils
with `-Wl,`.
If a project is using clang to drive linking, make clang do the right
thing with MACOSX_DEPLOYMENT_TARGET. This can be overridden by command
line arguments. This will cause modern clang to pass
`-platform_version 10.12 0.0.0`, since it doesn't know about the SDK
settings. Older versions of clang will pass down `-macos_version_min`
flags with no sdk version.
At the linker layer, apply a default value for anything left
ambiguous. If nothing is specified, pass a full
`-platform_version`. If only `-macos_version_min` is specified, then
lock down the sdk_version explicitly with `-sdk_version`. If a min
version and sdk version is passed, do nothing.
The new skip for the dynamic linker introduced in
ccfd26ef14 ("bintools-wrapper: skip dynamic linker for static
binaries") includes a change in behaviour, previously the $link32
variable was checked using an arithmetic expression via (( )). This
returns zero if the output of the arithmetic expression is nonzero, i.e.
link32=1 would return zero and the if condition would be executed.
The code refactored this to use "$link32" = "0" which is incorrect in
this case, since we want this if conditional to run if $link32 is 1.
Small shell excerpt for clarity:
$ export link32=1
$ export cookie=1
$ if [[ "$cookie" = "1" ]] && (( "$link32" )); then echo "do some stuff"; fi;
do some stuff
$ if [[ "$cookie" = "1" && "$link32" = "0" ]]; then echo "do some stuff"; fi;
$
Change it to check for $link32 = "1", as the previous code did.
Currently we set dynamic-linker unconditionally. This breaks
however some static binaries i.e. rust binaries linked against musl.
There is no reason we should set an elf interpreter for static binaries
hence this is skipped if `-static` or `-static-pie` is either passed to
our cc or ld wrapper.
I hate the thing too even though I made it, and rather just get rid of
it. But we can't do that yet. In the meantime, this brings us more
inline with autoconf and will make it slightly easier for me to write a
pkg-config wrapper, which we need.
Factor a bintools (i.e. binutils / cctools) wrapper out of cc-wrapper. While
only LD is wrapped, the setup hook defines environment variables on behalf of
other utilites.
