file is part of the final stdenv and llvm_19 requires it for tests. add
file to the path to the early stage stdenv's for the upcoming switch to
llvm_19
Replaces / Closes#353131
A while ago `postgresql` switched to using structured attrs[1]. In the
PR it was reported that this made postgresql notably slower when
importing SQL dumps[2].
After a bit of debugging it turned out that the hardening was entirely
missing and the following combination of settings was the culprit:
hardeningEnable = [ "pie" ];
__structuredAttrs = true;
I.e. the combination of custom hardening settings and structured attrs.
What happened here is that internally the default and enabled hardening
flags get written into `NIX_HARDENING_ENABLE`. However, the value is a list
and the setting is not in the `env` section. This means that in the
structured-attrs case we get something like
declare -ax NIX_HARDENING_ENABLE=([0]="bindnow" [1]="format" [2]="fortify" [3]="fortify3" [4]="pic" [5]="relro" [6]="stackprotector" [7]="strictoverflow" [8]="zerocallusedregs" [9]="pie")
i.e. an actual array rather than a string with all hardening flags being
space-separated which is what the hardening code of the cc-wrapper
expects[3].
This only happens if `hardeningEnable` or `hardeningDisable` are
explicitly set by a derivation: if none of those are set,
`NIX_HARDENING_ENABLE` won't be set by `stdenv.mkDerivation` and the
default hardening flags are configured by the setup hook of the
cc-wrapper[4].
In other words, this _only_ applies to derivations that have both custom
hardening settings _and_ `__structuredAttrs = true;`.
All values of `NIX_HARDENING_ENABLE` are well-known, so we don't have to
worry about escaping issues. Just forcing it to a string by
concatenating the list everytime solves the issue without additional
issues like eval errors when inheriting `env` from a structuredAttrs
derivation[5]. The price we're paying is a full rebuild.
[1] https://github.com/NixOS/nixpkgs/pull/294504
[2] https://github.com/NixOS/nixpkgs/pull/294504#issuecomment-2451482522
[3] cf3e5d3744/pkgs/build-support/cc-wrapper/add-hardening.sh (L9)
[4] cf3e5d3744/pkgs/build-support/cc-wrapper/setup-hook.sh (L114)
[5] 1e84a7fb95
This is basically harmless for the same reason as it is for Clang, and
lets us avoid doing wrapper hacks to fix things like the .NET build.
This reverts commit 4340a5addb.
I've often run into substituteInPlace doing nothing when I invoked it
wrong, like:
substituteInPlace --replace-fail from to file.txt
The substitute function would pass file.txt on to substituteStream,
which complain about the extraneous argument. However, if no files are
passed, substitute is never called and substituteInPlace silently does
nothing.
It would be more annoying to make substituteInPlace accept file
arguments anywhere (since it would need to understand the arguments to
substituteStream), so let's error on this incorrect usage instead.
Change-Id: Ib2daad6f49bbe516a273a35d7f2a31fc42fd053c
- Drop libSystem. It’s no longer needed. The SDK can be downloaded and
built with existing tools.
- Add jq and tapi. Adding these allows the stdenv bootstrap to stop
special-casing stage 0.
- Update tests for updated ld64. It handles code-signing properly, so
the signatures aren’t broken.
The bootstrap tools linker sometimes crashes when trying to link the
sqlite3 tests, which causes the bootstrap Python not to have the sqlite3
module. This causes the freezegun module to fail to build later in the
bootstrap. Using the 11.0 SDK fixes the problem.
Upstream Python supports building with a newer SDK and back-deploying,
so this change should not negatively affect users on pre-11.0 releases.
Change the adapter to add the requested SDK to the target’s build
inputs. Note that only the “11.0” (i.e., 11.3) and 12.3 SDKs are
supported. This adapter is retained for compatibility. The preferred way
to override the SDK is to add it to the appropriate inputs directly.
`${stdenv.cc.libc}/lib/libSystem.B.dylib` does not exist and not existed
for as long as I have used nixpkgs. Since `/usr/lib/libSystem.B.dylib`
is already linked via text-based stubs, continue relying on those.
While it would be nice if this could be split, there are too many
changes as part of the cleanup and improvements, including:
- Refactoring all propagated packages into functions that can be used to
ensure that packages are propagated only at the expected stages;
- Using a sanity-checking merge function to ensure that packages are
only propagated by one of the above functions;
- Reducing the number of Python builds during the bootstrap to one;
- Removing the extra sysctl stage;
- Using the LLVM bootstrap to build LLVM, clang, libc++, etc;
- Propagating llvmPackages_<version> in the final stdenv, so that
packages needing that version specifically don’t have to rebuild it;
- Bootstrapping with the new Darwin SDK; and
- Reducing the overall number of paths build during a bootstrap by ~33%.
After discussing on Matrix in the [#staging](https://matrix.to/#/#staging:nixos.org) room, I'm merging this as-is without waiting for changes to the `enableParallelBuilding = false;` line.
There are several in-the-wings PRs (one from @emilazy) that attempt to bring load-limit, which is a more sensible accounting of the work that a derivation is doing, into Nix itself. If and when that lands, we'll be able to just set `enableParallelBuilding = true;` again, without any specific casing around the `NIX_BUILD_CORES` stdenv variable.
Currently stdenv requires Bash 4.x that was released in 2009. This
change bumps the required version to Bash 5.x (2019, 5 years ago).
See https://mywiki.wooledge.org/BashFAQ/061 for more details.
Using a relatively modern Bash version allows us to rely on newer
features (e.g. ${var@a}) and remove workarounds for older quirks (e.g.
https://stackoverflow.com/a/7577209, “old bash empty array problem”).
Note that many setup hooks are using features added after 4.0 version,
e.g. makeWrapper uses ${var@Q} from 4.4, but some even require >5.0,
e.g. cargoBuildHook uses ${var@U} from 5.1.
