I've often run into substituteInPlace doing nothing when I invoked it
wrong, like:
substituteInPlace --replace-fail from to file.txt
The substitute function would pass file.txt on to substituteStream,
which complain about the extraneous argument. However, if no files are
passed, substitute is never called and substituteInPlace silently does
nothing.
It would be more annoying to make substituteInPlace accept file
arguments anywhere (since it would need to understand the arguments to
substituteStream), so let's error on this incorrect usage instead.
Change-Id: Ib2daad6f49bbe516a273a35d7f2a31fc42fd053c
Currently stdenv requires Bash 4.x that was released in 2009. This
change bumps the required version to Bash 5.x (2019, 5 years ago).
See https://mywiki.wooledge.org/BashFAQ/061 for more details.
Using a relatively modern Bash version allows us to rely on newer
features (e.g. ${var@a}) and remove workarounds for older quirks (e.g.
https://stackoverflow.com/a/7577209, “old bash empty array problem”).
Note that many setup hooks are using features added after 4.0 version,
e.g. makeWrapper uses ${var@Q} from 4.4, but some even require >5.0,
e.g. cargoBuildHook uses ${var@U} from 5.1.
The previously used pattern was introduced in #318614, but technically
leaked the default flags into the global scope. While this would
probably not make much of a practical difference, making concatTo
support default values is a much cleaner approach.
This fixes the regression introduced by c47a1e701d
on Darwin. The creation of the file using `install` and process
substitution does not work on Darwin, you get the following complain:
```
install: skipping file '/dev/fd/63', as it was replaced while being copied
```
Fixes#335016
Instead of checking for __structuredAttrs everywhere, it's easier to
just set the default value via parameter expansion and then hand the
array construction off to "concatTo".
Once more setup-hooks will be made structuredAttrs-aware, this pattern
will reduce the use of this implementation detail even more.
structuredAttrs was used here to make an assumption about the type of
the named variables passed as arguments. This can be done better by
looking at the actual types of those variables.
This gives a bit more backwards compatibility as well: Once you turn to
structuredAttrs, you should still be able to pass a bare string instead
of a list and have it behave as a whitespace-separated string like
before.
When running `nix develop` for a package, Nix records the stdenv
environment with NIX_LOG_FD set. That is, when the actual development
shell runs, it uses the functions that attempt to log to NIX_LOG_FD, but
this variable is not actually set.
As a workaround, check whether NIX_LOG_FD is set at runtime.
Example (before this change):
```console
$ nix develop --file . bash
$ echo "${NIX_LOG_FD-unset}"
unset
$ runPhase unpackPhase
bash: "$NIX_LOG_FD": Bad file descriptor
Running phase: unpackPhase
unpacking source archive /nix/store/v28dv6l0qk3j382kp40bksa1v6h7dx9p-bash-5.2.tar.gz
bash: "$NIX_LOG_FD": Bad file descriptor
source root is bash-5.2
bash: "$NIX_LOG_FD": Bad file descriptor
setting SOURCE_DATE_EPOCH to timestamp 1663942708 of file bash-5.2/y.tab.h
```
After this change:
```console
$ nix develop --file . bash
$ runPhase unpackPhase
Running phase: unpackPhase
unpacking source archive /nix/store/v28dv6l0qk3j382kp40bksa1v6h7dx9p-bash-5.2.tar.gz
source root is bash-5.2
setting SOURCE_DATE_EPOCH to timestamp 1663942708 of file bash-5.2/y.tab.h
```
Without the change `runPhase` fails on tarballs like
`diffoscope-269` that contain single top-level `-269` root as:
diffoscope> unpacking source archive /nix/store/p620nidkm73vrp0z6kk5krmrm4vg7bxd-diffoscope-269.tar.bz2
diffoscope> source root is -269
diffoscope> setting SOURCE_DATE_EPOCH to timestamp 1717143039 of file ./-269/tests/utils/versions.py
diffoscope> chmod: invalid mode: ‘-269’
diffoscope> Try 'chmod --help' for more information.
Currently `diffoscope-269` has a `sourceRoot = "./-269";` workaround to
bypass the failure.
A second take at eb28e5e72e, which was reverted for the extra logging
during the internals of `nix-shell -p`. This commit does the same
logging, but to $NIX_LOG_FD instead, which is echoed during any normal
build, but not during the internals of `nix-shell -p`.
[1]: eb28e5e72e
This is a small simplification of the control flow surrounding these cases. It should make it more obvious when each case happens, and also explicitly defines the current behaviour of --replace.
Provide a `runPhase` function which wraps the phase running action of
genericBuild. The new function can be used as an interface by `nix
develop`, i.e. `nix develop some#flake --build` may just call `runPhase
build`, which makes its behavior more consistent with `nix build`.
In preparation of fixing https://github.com/NixOS/nix/issues/6202
Relying on `.attrs.sh` to exist in `$NIX_BUILD_TOP` is problematic
because that's not compatible with how `nix-shell(1)` behaves. It places
`.attrs.{json,sh}` into a temporary directory and makes them accessible via
`$NIX_ATTRS_{SH,JSON}_FILE` in the environment[1]. The sole reason that
`nix-shell(1)` still works with structured-attrs enabled derivations
is that the contents of `.attrs.sh` are sourced into the
shell before sourcing `$stdenv/setup` (if `$stdenv` exists) by `nix-shell`.
However, the assumption that two files called `.attrs.sh` and
`.attrs.json` exist in `$NIX_BUILD_TOP` is wrong in an interactive shell
session and thus an inconsistency between shell debug session and actual
builds which can lead to unexpected problems.
To be precise, we currently have the following problem: an expression
like
with import ./. {};
runCommand "foo" { __structuredAttrs = true; foo.bar = [ 1 2 3 ]; }
''
echo "''${__structuredAttrs@Q}"
touch $out
''
prints `1` in its build-log. However when building interactively in a
`nix-shell`, it doesn't.
Because of that, I'm considering to propose a full deprecation of
`$NIX_BUILD_TOP/.attrs.{json,sh}`. A first step is to only mention the
environment variables, but not the actual paths anymore in Nix's
manual[2]. The second step - this patch - is to fix nixpkgs' stdenv
accordingly.
Please note that we cannot check for `-e "$NIX_ATTRS_JSON_FILE"` because
certain outdated Nix minors (that are still in the range of supported
Nix versions in `nixpkgs`) have a bug where `NIX_ATTRS_JSON_FILE` points
to the wrong file while building[3].
Also, for compatibility with Nix 2.3 which doesn't provide these
environment variables at all we still need to check for the existence of
.attrs.json/.attrs.sh here. As soon as we bump nixpkgs' minver to 2.4,
this can be dropped.
Finally, dropped the check for ATTRS_SH_FILE because that was never
relevant. In nix#4770 the ATTRS_SH_FILE variable was introduced[4] and
in a review iteration prefixed with NIX_[5]. In other words, these
variables were never part of a release and you'd only have this problem
if you'd use a Nix from a git revision of my branch from back then. In
other words, that's dead code.
[1] https://github.com/nixos/nix/pull/4770#issuecomment-834718851
[2] https://github.com/NixOS/nix/pull/9032
[3] https://github.com/NixOS/nix/issues/6736
[4] 3944a120ec
[5] 27ce722638
In the default `fixupPhase` the output of `substituteAllStream` is
streamed to setup-hook.
`stdenv.cc.bintools.overrideAttrs { NIX_DEBUG = 6; }`
With `NIX_DEBUG` contains:
```
@expandResponseParams@ -> /nix/store/yl01rd58vp4m8bbhkihpk132cprfmx6f-expand-response-params/bin/expand-response-params
...
```
