02c09e0171 (NixOS/nixpkgs#44558) was reverted in
c981787db9 but, as it turns out, it fixed an issue
I didn't know about at the time: the values of `propagateDoc` options were
(and now again are) inconsistent with the underlying things those wrappers wrap
(see NixOS/nixpkgs#46119), which was (and now is) likely to produce more instances
of NixOS/nixpkgs#43547, if not now, then eventually as stdenv changes.
This patch (which is a simplified version of the original reverted patch) is the
simplest solution to this whole thing: it forces wrappers to directly inspect the
outputs of the things they are wrapping instead of making stdenv guess the correct
values.
In particular, this contains Firefox-related and libgcrypt updates.
Other larger rebuilds would apparently need lots of time to catch up
on Hydra, due to nontrivial rebuilds in other branches than staging.
The hack of using `crossConfig` to enforce stricter handling of
dependencies is replaced with a dedicated `strictDeps` for that purpose.
(Experience has shown that my punning was a terrible idea that made more
difficult and embarrising to teach teach.)
Now that is is clear, a few packages now use `strictDeps`, to fix
various bugs:
- bintools-wrapper and cc-wrapper
... binutils and gcc add it already anyway.
Without this it's easy to get cross-toolchain paths longer than 256
chars and nix-daemon will then fail to commit them to /nix/store on XFS.
Per @Ericson2314's suggestion [1], make it more clear that the active
hardenings are decided via whitelist; the blacklist is merely for the
debug messages.
1: 36d5ce41d4 (r133279731)
Before the code would fail silently for zero values and with some output for
empties. We now currently handle both via defaulting value to zero and making
`let` return success error code when there's no syntax error.
- All deps go on the PATH
- CC and Bintools wrappers with their host != depender's host still get their
setup hooks run.
- Environment hooks get applied to all packages
This isn't so elegent, but eases the transition on a very significant
PR.
We now have the information to properly determine the role the
cc-wrapper dependency has, by taking advantage of `offset`. No longer
use the soon-to-be-deprecated crossConfig environment variable, the
temp hack used before this change.
Factor a bintools (i.e. binutils / cctools) wrapper out of cc-wrapper. While
only LD is wrapped, the setup hook defines environment variables on behalf of
other utilites.
On non-GNU (gcc) compilers, there is no "/lib/gcc/..."
so when this is eventually expanded this is empty
resulting in an incomplete "-idirafter " that
eats the next argument:
-idirafter -B/nix/store/wamjwwdvkmhbf4f2902nhw8jxxzv0hy3-clang-wrapper-4.0.1/bin/
Certain tools, e.g. compilers, are customarily prefixed with the name of
their target platform so that multiple builds can be used at once
without clobbering each other on the PATH. I was using identifiers named
`prefix` for this purpose, but that conflicts with the standard use of
`prefix` to mean the directory where something is installed. To avoid
conflict and confusion, I renamed those to `targetPrefix`.
If a dynamic linker for target is not found the generated script fails
due to unbound variable error (due to "set -u"). Correct by specifying
default value with dynamicLinker:- and not generating ldflagsBefore if
no linker is found.
This problem was found when cross compiling to mingw32 targets
This requires some small changes in the stdenv, then working around the
weird choice LLVM made to hardcode @rpath in its install name, and then
lets us remove a ton of annoying workaround hacks in many of our Go
packages. With any luck this will mean less hackery going forward.
cc-wrapper may wrap a cc-compiler, but it doesn't need one to build
itself. (c.f. expand-response-params is a separate derivation.) This
helps avoid cycles on the cross stuff, in addition to removing a
useless dependency edge.
I could have been super careful with overrides in the stdenv to avoid
the mass rebuild, but I don't think it's worth it.
1. `crossDrv` is now the default so we don't need to worry about that in
build != host builds.
2. shell is the build time shell, so `wrapCCCross` doesn't need to
worry, as build == host.
3. `shell.shellPath` will always be appended where useful.
4. Complicated `shell == ""` logic served no purpose.
This reverts commit 0a944b345e, reversing
changes made to 61733ed6cc.
I dislike these massive stdenv changes with unclear motivation,
especially when they involve gratuitous mass renames like NIX_CC ->
NIX_BINUTILS. The previous such rename (NIX_GCC -> NIX_CC) caused
months of pain, so let's not do that again.
This becomes necessary if more wrappers besides cc-wrapper start
supporting hardening flags. Also good to make the warning into an
error.
Also ensure interface is being used right: Not as a string, not just in
bash.
ccPath is only defined below, so this condition would never be true.
Worse, that's not quite true: what if somebody happend to have `/clang`
and no sandboxing. Boy, wouldn't that be annoying to debug!
Having multiple compilers in the build environment would result in an
invalid LD_DYLD_PATH like /usr/lib/dyld/usr/lib/dyld.
Since the path is hardcoded in XNU it can't be anything but
/usr/lib/dyld anyway.
This fixes a bug introduced in #27831: `for path in "$dir"/lib*.so` assumed that
all libs match `lib*.so`, but 07674788d6 started
adding libs that match `*.so` and `*.so.*`.
Now is an opportune time to do this, as the infixSalt conversion in
`add-flags.sh` ensures that all the relevant `NIX_*` vars will be
defined even if empty.
This is basically a sed job, in preparation of the next commit. The
rules are more or less:
- s"NIX_(.._WRAPPER_)?([a-zA-Z0-9@]*)"NIX_\1@infixSalt@_\2"g
- except for non-cc-wrapper-specific vars like `NIX_DEBUG`
This is an ugly temp hack for cross compilation, but now we have something better on the way.
Bind `infixSalt` as an environment variable as it will be used in it.
Unified processing of command line arguments in ld-wrapper broke support for
`NIX_DONT_SET_RPATH` and revealed that ld-wrapper adds the directory of its
`-plugin` argument to runpath. This pull request fixes that. It treats
`dir/libname.so` as `-L dir -l name`, because this is how `ld.so` interprets
resulting binary: with `dir` in `RUNPATH` and the bare `libname.so` (without
`dir`) in `NEEDED`, it looks for `libname.so` in each `RUNPATH` and chooses the
first, even when the linker was invoked with an absolute path to `.so`.
As described in https://github.com/NixOS/nixpkgs/issues/18461, MacOS no
longer accepts dylibs which only reexport other dylibs, because their
symbol tables are empty. To get around this, we define an object file
with a single "private extern" symbol, which hopefully won't clobber
anything.
The time to expand rpath was proportional to the number of -L flags times the
number of -l flags. Now it is proportional to their sum (assuming constant
number of files in each directory in an -L flag).
Issue reported by @nh2 at https://github.com/NixOS/nixpkgs/issues/27609#issuecomment-317916623
As @oxij points out in [1], this breakage is especially serious because
it changes the contents of built environments without a corresonding
change in their hashes. Also, the revert is easier than I thought.
This reverts commit 3cb745d5a6.
[1]: https://github.com/NixOS/nixpkgs/pull/27427#issuecomment-317293040
Besides deduplicating overlapping logic, clear warning messages were
added for:
- No glob/path for dynamic linker provided (use default glob)
- Glob did not expand to anything (don't append flag)
- glob expanded to multiple things (take first, like before)
This makes those files a bit easier to read. Also, for what it's worth,
it brings us one baby step closer to handling spaces in store paths.
Also, I optimized handling of many transitive deps with read. Probably,
not very beneficial, but nice to enforce the pkg-per-line structure.
Doing so let me find much dubious code and fix it.
Two misc notes:
- `propagated-user-env-packages` also needed to be adjusted as
sometimes it is copied to/from the propagated input files.
- `local fd` should ensure that file descriptors aren't clobbered
during recursion.
