The goal is to print all store references into $out.
First, $out itself is not defined with structuredAttrs, but we can work
around that with placeholder. Alternatively we could source
$stdenv/setup after sourcing the attrs.sh file, but that feels like
overkill.
To support structuredAttrs we source the attrs.sh file. export will not
be enough anymore, because the attrs file sets bash variables, not
environment variables. Thus we resort to declare -p.
Resolves#321005
Without the change build for packages that use `disallowedReferences`
fails in `contentAddressedByDefault = true` mode:
$ nix build -f. ruby_3_1 --arg config '{ contentAddressedByDefault = true; }'
...
error: derivation contains an illegal reference specifier '/0j3hif3ni7zl5zhlzzr5q2q23z66136mnzp75pyiqp5c72q14im2'
error: 1 dependencies of derivation '/nix/store/39ji7qp225pxvrm8cgvzmyjqsyis2n0h-ruby-3.1.2.drv' failed to build
Original intent of https://github.com/NixOS/nixpkgs/pull/211783 was to
avoid pulling in actual derivation for reference scanning purposes.
Unfortunately CA derivations's outputPath are placeholders until they
are instantiated.
Let's restore string context for CA derivations for now.
Replaces / Closes#353131
A while ago `postgresql` switched to using structured attrs[1]. In the
PR it was reported that this made postgresql notably slower when
importing SQL dumps[2].
After a bit of debugging it turned out that the hardening was entirely
missing and the following combination of settings was the culprit:
hardeningEnable = [ "pie" ];
__structuredAttrs = true;
I.e. the combination of custom hardening settings and structured attrs.
What happened here is that internally the default and enabled hardening
flags get written into `NIX_HARDENING_ENABLE`. However, the value is a list
and the setting is not in the `env` section. This means that in the
structured-attrs case we get something like
declare -ax NIX_HARDENING_ENABLE=([0]="bindnow" [1]="format" [2]="fortify" [3]="fortify3" [4]="pic" [5]="relro" [6]="stackprotector" [7]="strictoverflow" [8]="zerocallusedregs" [9]="pie")
i.e. an actual array rather than a string with all hardening flags being
space-separated which is what the hardening code of the cc-wrapper
expects[3].
This only happens if `hardeningEnable` or `hardeningDisable` are
explicitly set by a derivation: if none of those are set,
`NIX_HARDENING_ENABLE` won't be set by `stdenv.mkDerivation` and the
default hardening flags are configured by the setup hook of the
cc-wrapper[4].
In other words, this _only_ applies to derivations that have both custom
hardening settings _and_ `__structuredAttrs = true;`.
All values of `NIX_HARDENING_ENABLE` are well-known, so we don't have to
worry about escaping issues. Just forcing it to a string by
concatenating the list everytime solves the issue without additional
issues like eval errors when inheriting `env` from a structuredAttrs
derivation[5]. The price we're paying is a full rebuild.
[1] https://github.com/NixOS/nixpkgs/pull/294504
[2] https://github.com/NixOS/nixpkgs/pull/294504#issuecomment-2451482522
[3] cf3e5d3744/pkgs/build-support/cc-wrapper/add-hardening.sh (L9)
[4] cf3e5d3744/pkgs/build-support/cc-wrapper/setup-hook.sh (L114)
[5] 1e84a7fb95
I've often run into substituteInPlace doing nothing when I invoked it
wrong, like:
substituteInPlace --replace-fail from to file.txt
The substitute function would pass file.txt on to substituteStream,
which complain about the extraneous argument. However, if no files are
passed, substitute is never called and substituteInPlace silently does
nothing.
It would be more annoying to make substituteInPlace accept file
arguments anywhere (since it would need to understand the arguments to
substituteStream), so let's error on this incorrect usage instead.
Change-Id: Ib2daad6f49bbe516a273a35d7f2a31fc42fd053c
Currently stdenv requires Bash 4.x that was released in 2009. This
change bumps the required version to Bash 5.x (2019, 5 years ago).
See https://mywiki.wooledge.org/BashFAQ/061 for more details.
Using a relatively modern Bash version allows us to rely on newer
features (e.g. ${var@a}) and remove workarounds for older quirks (e.g.
https://stackoverflow.com/a/7577209, “old bash empty array problem”).
Note that many setup hooks are using features added after 4.0 version,
e.g. makeWrapper uses ${var@Q} from 4.4, but some even require >5.0,
e.g. cargoBuildHook uses ${var@U} from 5.1.
The previously used pattern was introduced in #318614, but technically
leaked the default flags into the global scope. While this would
probably not make much of a practical difference, making concatTo
support default values is a much cleaner approach.
This fixes the regression introduced by c47a1e701d
on Darwin. The creation of the file using `install` and process
substitution does not work on Darwin, you get the following complain:
```
install: skipping file '/dev/fd/63', as it was replaced while being copied
```
Fixes#335016
Instead of checking for __structuredAttrs everywhere, it's easier to
just set the default value via parameter expansion and then hand the
array construction off to "concatTo".
Once more setup-hooks will be made structuredAttrs-aware, this pattern
will reduce the use of this implementation detail even more.
structuredAttrs was used here to make an assumption about the type of
the named variables passed as arguments. This can be done better by
looking at the actual types of those variables.
This gives a bit more backwards compatibility as well: Once you turn to
structuredAttrs, you should still be able to pass a bare string instead
of a list and have it behave as a whitespace-separated string like
before.
When running `nix develop` for a package, Nix records the stdenv
environment with NIX_LOG_FD set. That is, when the actual development
shell runs, it uses the functions that attempt to log to NIX_LOG_FD, but
this variable is not actually set.
As a workaround, check whether NIX_LOG_FD is set at runtime.
Example (before this change):
```console
$ nix develop --file . bash
$ echo "${NIX_LOG_FD-unset}"
unset
$ runPhase unpackPhase
bash: "$NIX_LOG_FD": Bad file descriptor
Running phase: unpackPhase
unpacking source archive /nix/store/v28dv6l0qk3j382kp40bksa1v6h7dx9p-bash-5.2.tar.gz
bash: "$NIX_LOG_FD": Bad file descriptor
source root is bash-5.2
bash: "$NIX_LOG_FD": Bad file descriptor
setting SOURCE_DATE_EPOCH to timestamp 1663942708 of file bash-5.2/y.tab.h
```
After this change:
```console
$ nix develop --file . bash
$ runPhase unpackPhase
Running phase: unpackPhase
unpacking source archive /nix/store/v28dv6l0qk3j382kp40bksa1v6h7dx9p-bash-5.2.tar.gz
source root is bash-5.2
setting SOURCE_DATE_EPOCH to timestamp 1663942708 of file bash-5.2/y.tab.h
```
