Commit Graph

412 Commits

Author SHA1 Message Date
Connor Baker
8aa11323bb stdenv: add note in nix logging functions about not cluttering nix-shell 2025-01-23 04:14:49 +00:00
Connor Baker
60749f9cdc stdenv: rename nixLogWithLevel to _nixLogWithLevel to indicate it is meant to be private 2025-01-23 04:11:44 +00:00
Connor Baker
a2b293e3c9 stdenv: introduce nixLog and nixLogWithLevel 2025-01-03 22:43:56 +00:00
Wolfgang Walther
2332759d7a
stdenv: fix nix develop
This was introduced in #360466.
2024-12-09 19:17:26 +01:00
Philip Taron
fdee9525f3
various: fix "list contains" check (#363191) 2024-12-09 09:17:28 -08:00
Wolfgang Walther
80344cb8d5
various: fix "list contains" check
Testing for the presence of an item in a space separated list with this
pattern needs to pad the list with spaces at the beginning and end.
Otherwise the first and last element will never be matched.

Resolves #354220
2024-12-08 13:11:59 +01:00
Martin Weinelt
41d4a66bba
Revert "stdenv: runPhase returns status" 2024-12-04 02:18:53 +01:00
Philip Taron
6f40e94649
stdenv: runPhase returns status (#330751) 2024-12-02 08:03:36 -08:00
Wolfgang Walther
56b09628e3
stdenv: support multi-char separators in concatStringsSep
One prominent use-case for this is pytestCheckHook. This will help
making it work with structuredAttrs in the future.
2024-11-30 16:38:02 +01:00
Romanos Skiadas
bdac0fa35d pkgs/stdenv: fix defaultUnpack on nfs
-p preserves permission,mode,timestamps.

When trying to copy with permission on NFS, it can fail;

The closest documentation I found on this was:

https://access.redhat.com/solutions/725513

One suggestion is to change the netapp configuration, but another that
should work for everyone is to use the cp command presented in this
patch.

With this, builds on nfs work for me. I however cannot say that I'm
certain that removing the permission preservation won't cause any other
troubles elsewhere, e.g. by perhaps introducing some sort of
non-determinism.
2024-11-28 17:26:43 +02:00
Philip Taron
f2b00cb5ef
pkgs/stdenv/generic/setup.sh: lint with ShellCheck (#351849) 2024-11-01 08:52:44 -07:00
Yueh-Shun Li
34ebbd650c pkgs/stdenv/generic/setup.sh: lint with ShellCheck
The behaviour of [[ -n/-z "${FOO[@]}" ]] is unspecified.
Use [[ -n/-z "${FOO[*]-}" ]] instead
2024-10-28 22:25:30 +08:00
Linus Heckemann
9767bb9bf1 stdenv/setup: make substituteInPlace require actual files
I've often run into substituteInPlace doing nothing when I invoked it
wrong, like:

substituteInPlace --replace-fail from to file.txt

The substitute function would pass file.txt on to substituteStream,
which complain about the extraneous argument. However, if no files are
passed, substitute is never called and substituteInPlace silently does
nothing.

It would be more annoying to make substituteInPlace accept file
arguments anywhere (since it would need to understand the arguments to
substituteStream), so let's error on this incorrect usage instead.

Change-Id: Ib2daad6f49bbe516a273a35d7f2a31fc42fd053c
2024-10-21 13:13:42 +02:00
Philip Taron
05e5a56f04
auto-patchelf: refactor structuredAttrs support (#340858) 2024-09-12 15:13:20 -07:00
Wolfgang Walther
bc0395ee6f
auto-patchelf: refactor structuredAttrs support
stdenv now provides better tooling to support structuredAttrs without
depending on $__structuredAttrs itself.
2024-09-11 19:29:39 +02:00
Philip Taron
f4cd623ed3
stdenv: bump required Bash version from 4 to 5 (#340765) 2024-09-11 07:32:57 -07:00
Ivan Trubach
799d0c8a7d stdenv: bump required Bash version from 4 to 5
Currently stdenv requires Bash 4.x that was released in 2009. This
change bumps the required version to Bash 5.x (2019, 5 years ago).

See https://mywiki.wooledge.org/BashFAQ/061 for more details.

Using a relatively modern Bash version allows us to rely on newer
features (e.g. ${var@a}) and remove workarounds for older quirks (e.g.
https://stackoverflow.com/a/7577209, “old bash empty array problem”).
Note that many setup hooks are using features added after 4.0 version,
e.g. makeWrapper uses ${var@Q} from 4.4, but some even require >5.0,
e.g. cargoBuildHook uses ${var@U} from 5.1.
2024-09-09 15:22:11 +03:00
Wolfgang Walther
1efcffa700
stdenv: support default values in concatTo
The previously used pattern was introduced in #318614, but technically
leaked the default flags into the global scope. While this would
probably not make much of a practical difference, making concatTo
support default values is a much cleaner approach.
2024-08-24 12:23:35 +02:00
Thomas Gerbet
d00775c1d9 stdenv: create env-vars file before writing data to it
This fixes the regression introduced by c47a1e701d
on Darwin. The creation of the file using `install` and process
substitution does not work on Darwin, you get the following complain:
```
install: skipping file '/dev/fd/63', as it was replaced while being copied
```

Fixes #335016
2024-08-20 19:02:41 +02:00
Someone Serge
ebf6790c39 stdenv: concatTo: fall back to old behaviour for "*Array" variables
Fixes e.g. the `sudo` build failure
2024-08-16 12:51:51 +00:00
Someone
ccaaa9ca53
Merge pull request #318614 from wolfgangwalther/structured-attrs-setup-hooks
treewide: support structuredAttrs in setup hooks
2024-08-13 19:29:36 +00:00
Someone Serge
64eaa63181 stdenv: concatStringsSep: quote ${sep} 2024-08-10 23:38:10 +00:00
Philip Taron
d8fbb16219
stdenv: change the logging in _allFlags to talkative 2024-08-05 18:35:57 -07:00
Philip Taron
e844424e4f
stdenv: replace other $NIX_DEBUG log statements 2024-08-05 18:35:57 -07:00
Philip Taron
465dbd2ddf
stdenv: log hooks at nixTalkativeLog level
Nix filters out messages with level ≥ 4 by default as of this commit.
2024-08-05 18:35:57 -07:00
Philip Taron
624463391d
stdenv: introduce specific logging functions 2024-08-05 18:35:57 -07:00
Philip Taron
bbb9f2f1c9
stdenv: set the phase in showPhaseHeader 2024-08-05 18:35:57 -07:00
Wolfgang Walther
471cbdd062
stdenv: add concatStringsSep helper
This can be used to separate lists for example with commas, when
creating argument strings. This works with both structuredAttrs disabled
and enabled.
2024-08-02 22:11:41 +02:00
Wolfgang Walther
8cb51ec38e
stdenv: refactor default flags without __structuredAttrs use
Instead of checking for __structuredAttrs everywhere, it's easier to
just set the default value via parameter expansion and then hand the
array construction off to "concatTo".

Once more setup-hooks will be made structuredAttrs-aware, this pattern
will reduce the use of this implementation detail even more.
2024-08-02 22:11:38 +02:00
Wolfgang Walther
6bdfef9d2d
stdenv: generalize _accumFlagsArray to concatTo
Passing "flagsArray" as the first argument allows using this function
in a few more places.
2024-08-02 22:11:38 +02:00
Wolfgang Walther
bfd97a691f
stdenv: make _accumFlagsArray independent of structuredAttrs
structuredAttrs was used here to make an assumption about the type of
the named variables passed as arguments.  This can be done better by
looking at the actual types of those variables.

This gives a bit more backwards compatibility as well: Once you turn to
structuredAttrs, you should still be able to pass a bare string instead
of a list and have it behave as a whitespace-separated string like
before.
2024-08-02 21:54:25 +02:00
Thomas Gerbet
c47a1e701d stdenv: make sure the env-vars file created is not world readable
Under some circumstances this file might contain private information
that should not be accessible to everybody.
2024-08-01 19:26:05 +02:00
Wolfgang Walther
cdb2f2971c
stdenv: refactor appendToVar and prependToVar
No need to call declare -p twice. The case statement is easier to read
than the multi-if.
2024-07-30 21:38:14 +02:00
Ivan Trubach
bd872b4a77 stdenv: fix unbound NIX_LOG_FD in nix develop
When running `nix develop` for a package, Nix records the stdenv
environment with NIX_LOG_FD set. That is, when the actual development
shell runs, it uses the functions that attempt to log to NIX_LOG_FD, but
this variable is not actually set.

As a workaround, check whether NIX_LOG_FD is set at runtime.

Example (before this change):
```console
$ nix develop --file . bash
$ echo "${NIX_LOG_FD-unset}"
unset
$ runPhase unpackPhase
bash: "$NIX_LOG_FD": Bad file descriptor
Running phase: unpackPhase
unpacking source archive /nix/store/v28dv6l0qk3j382kp40bksa1v6h7dx9p-bash-5.2.tar.gz
bash: "$NIX_LOG_FD": Bad file descriptor
source root is bash-5.2
bash: "$NIX_LOG_FD": Bad file descriptor
setting SOURCE_DATE_EPOCH to timestamp 1663942708 of file bash-5.2/y.tab.h
```

After this change:
```console
$ nix develop --file . bash
$ runPhase unpackPhase
Running phase: unpackPhase
unpacking source archive /nix/store/v28dv6l0qk3j382kp40bksa1v6h7dx9p-bash-5.2.tar.gz
source root is bash-5.2
setting SOURCE_DATE_EPOCH to timestamp 1663942708 of file bash-5.2/y.tab.h
```
2024-07-29 13:08:17 +03:00
Mitchell Allain
e93ca9b43c Change from status to retval 2024-07-28 17:41:41 -07:00
Mitchell Allain
8e1442adb2 Add return value to runPhase bash function in stdenv 2024-07-28 17:17:17 -07:00
Yueh-Shun Li
c65cf1b9c5
pkgs/stdenv/generic/setup.sh: fix unbound variables and pass ShellCheck checks (#298831)
* pkgs/stdenv/generic/setup.sh: supress ShellCheck error about Bash array indexing syntax

* pkgs/stdenv/generic/setup.sh: loop by per-line read

* pkgs/stdenv/generic/setup.sh: fix variable quoting

* pkgs/stdenv/generic/setup.sh: separate declaration and assignment
2024-07-14 19:17:55 +03:00
Ryan Hendrickson
40428630bc
Merge pull request #317106 from trofi/dashed-source
stdenv: handle $sourceRoot that starts with dash
2024-07-02 23:48:25 -04:00
K900
e2665b3080 stdenv: fix typo 2024-06-24 10:17:10 +03:00
Ryan Lahfa
193b2c7637
Merge pull request #310387 from Qyriad/diag/log-hooks2
stdenv: log hooks as they run (take II)
2024-06-19 14:22:10 +02:00
Sergei Trofimovich
193fd8a237 generic/setup.sh: handle $sourceRoot that starts with dash
Without the change `runPhase` fails on tarballs like
`diffoscope-269` that contain single top-level `-269` root as:

    diffoscope> unpacking source archive /nix/store/p620nidkm73vrp0z6kk5krmrm4vg7bxd-diffoscope-269.tar.bz2
    diffoscope> source root is -269
    diffoscope> setting SOURCE_DATE_EPOCH to timestamp 1717143039 of file ./-269/tests/utils/versions.py
    diffoscope> chmod: invalid mode: ‘-269’
    diffoscope> Try 'chmod --help' for more information.

Currently `diffoscope-269` has a `sourceRoot = "./-269";` workaround to
bypass the failure.
2024-06-10 23:56:46 +01:00
Gerd Flaig
1949b0d16b Annotate substituteStream deprecation warning
This adds the derivation name to the warning to make debugging easier.
2024-05-25 18:32:10 +02:00
Qyriad
7acc356606 stdenv: log hooks as they run (take II)
A second take at eb28e5e72e, which was reverted for the extra logging
during the internals of `nix-shell -p`. This commit does the same
logging, but to $NIX_LOG_FD instead, which is echoed during any normal
build, but not during the internals of `nix-shell -p`.

[1]: eb28e5e72e
2024-05-22 16:28:31 -06:00
Vladimír Čunát
3299c31f44
stdenv: avoid setuid issues
See #300635.  Maybe in time we'll have a better solution.
2024-04-10 07:52:44 +02:00
binarycat
f9094faf70 stdenv: pass --mode=+w to tar, ensuring extraction is possible
fixes #191137
2024-03-18 20:54:31 -04:00
Vladimír Čunát
eff11adc57
Merge #286721: stdenv: fix rare tar.xz decompression issues
...into staging
2024-02-17 08:45:01 +01:00
Isidor Zeuner
11a19109b6 stdenv: disregard xz exit status in order to fix subtle decompression issues
There is a subtle bug with unpacking `tar.xz` archives which seems to happen only on some setups, and sometimes not in a reproducible manner (https://github.com/NixOS/nixpkgs/issues/278130, https://github.com/NixOS/nixpkgs/issues/20950). On the last occurrence, it could be tracked down to `xz` failing from a `SIGPIPE`, which can happen when it's connected to `tar` through a pipe and `tar` exits earlier (see e.g. https://www.linuxquestions.org/questions/slackware-14/%5Bpatch%5D-tar-issuing-a-sigpipe-in-installpkg-4175637923/ or https://bugs.gentoo.org/573642#c5).

Since `tar` should be able by itself to detect whether the archive is complete, I suggest to disregard the exit code from the `xz` invocation, done in this PR.

Fixes  https://github.com/NixOS/nixpkgs/issues/278130 (script tested here: https://github.com/NixOS/nixpkgs/pull/286579)
Probably also fixes https://github.com/NixOS/nixpkgs/issues/20950 (issue not reproduced here, feedback therefore welcome)
2024-02-13 17:00:48 +01:00
Kait Lam
355ab764b4 stdenv: refactor of --replace-{quiet,warn,fail} logic
This is a small simplification of the control flow surrounding these cases. It should make it more obvious when each case happens, and also explicitly defines the current behaviour of --replace.
2024-02-11 10:48:37 +10:00
Peder Bergebakken Sundt
cb4c41f93a stdenv: fix substituteStream --replace-quiet deprecation warning 2024-02-05 17:28:02 +01:00
Peder Bergebakken Sundt
4c1b74bac7 stdenv: substituteStream: escape echoed pattern in --replace mismatch warning 2024-01-14 22:08:03 +01:00