Setting the SDK root by default allows `overrideSDK` to correctly set
the SDK version when using a different SDK. It also allows the correct
SDK version to be set when using an older deployment target. Not setting
the correct SDK version can result in unexpected behavior at runtime.
Examples:
* Automatic dark mode switching requires linking against an SDK version
of 10.14 or newer. With the current behavior, the only way to do this
is by using a 10.14+ deployment target even when the application
supports older platforms when build with a newer SDK.
* MetalD3D checks that the system version is at least 14.0. The API it
uses returns a compatibility version when the the SDK is older than
11.0, which causes it to display an error and terminate the
application even when even when its requirements are all met.
This is effectively a rewrite of `overrideSDK`. It was required because
`wrapGAppsHook` propagates `depsTargetTarget` with the expectation that
it will effectively be `buildInputs` when the hook is itself used as a
`nativeBuildInput`. This propagates Gtk, which itself propagates the
default Dariwn SDK, making it effectively impossible to override the SDK
when a package depends on Gtk and uses `wrapGAppsHook`.
This rewrite implements the following improvements:
* Cross-compilation should be supported correctly (untested);
* Supports public and private frameworks;
* Supports SDK `libs`;
* Remaps instead of replacing extra (native) build inputs in the stdenv;
* Updates any Darwin framework references in `nix-support`; and
* It updates `xcodebuild` regardless of which input its in.
The implementation avoids recursion for performance reasons. Instead, it
enumerates transitive dependencies and walks the list from the leaf
packages backwards to the parent packages.
- update the hashes and tools needed to extract the bootstrap-tools archive
- unify the x64 and aarch64 unpack process
- unpin libxml for python-minimal
In the code examples for using `allowUnfreePredicate` and
`allowNonSourcePredicate`, we use `lib.getName` (which removes
the version) and not the local `getName` (which keeps the version).
`finalAttrs` is never the first argument. This should have been
called `prevAttrs` all along.
It confused me for a bit, because the callback _must not_ be strict
in `finalAttrs` (the first of _two_ parameters), or it will
recurse infinitely while trying to figure out what the attrNames are.
- unpin LLVM11. fix discrepancy with freshBootstrapTools and the tools
built on hydra. pinning the stdenv for the hydra build doesn't pin the
tools as the included packages are able to change.
- remove unused LLVM tools & libs which reduces the uncompressed and
compressed file sizes by more than 1/2. compressed tarball is now 40M
and uncompressed is around 200M
- add @loader_path/. to dylibs that reference other libs in the archive.
this is needed for libraries with re-exports.
- validate shared objects with @rpath references contain the reference
in lib
- add a test to verify that the @loader_path/ works for libc++ as it
re-exports libc++abi
The Darwin bootstrap currently requires curl, but it is not strictly
required. The bootstrap requires it for two things:
* Fetchers; and
* As a transitive dependency of llvm-manpages (via Sphinx).
For the fetchers, the bootstrap curl can be used. For hatch-vcs, the
dependency, its tests can be disabled. Doing this allows curl to be
dropped from the Darwin stdenv bootstrap.
These two commits make for a cleaner commit history and git blame than
https://github.com/NixOS/nixpkgs/pull/295105, where this refactor
was developed.
See its commit messages for details and design choices, esp. up to
and including 37f76fd4c3.
- merge libcxxabi into libcxx for LLVM 12, 13, 14, 15, 16, 17, and git.
- remove the link time workaround `-lc++ -lc++abi` from 58 packages as it is no longer required.
- fixes https://github.com/NixOS/nixpkgs/issues/166205
- provides alternative fixes for. https://github.com/NixOS/nixpkgs/issues/269548https://github.com/NixOS/nix/issues/9640
- pkgsCross.x86_64-freebsd builds work again
This change can be represented in 3 stages
1. merge libcxxabi into libcxx -- files: pkgs/development/compilers/llvm/[12, git]/{libcxx, libcxxabi}
2. update stdenv to account for merge -- files: stdenv.{adapters, cc.wrapper, darwin}
3. remove all references to libcxxabi outside of llvm (about 58 packages modified)
### merging libcxxabi into libcxx
- take the union of the libcxxabi and libcxx cmake flags
- eliminate the libcxx-headers-only package - it was only needed to break libcxx <-> libcxxabi circular dependency
- libcxx.cxxabi is removed. external cxxabi (freebsd) will symlink headers / libs into libcxx.
- darwin will re-export the libcxxabi symbols into libcxx so linking `-lc++` is sufficient.
- linux/freebsd `libc++.so` is a linker script `LINK(libc++.so.1, -lc++abi)` making `-lc++` sufficient.
- libcxx/default.nix [12, 17] are identical except for patches and `LIBCXX_ADDITIONAL_LIBRARIES` (only used in 16+)
- git/libcxx/defaul.nix does not link with -nostdlib when useLLVM is true so flag is removed. this is not much different than before as libcxxabi used -nostdlib where libcxx did not, so libc was linked in anyway.
### stdenv changes
- darwin bootstrap, remove references to libcxxabi and cxxabi
- cc-wrapper: remove c++ link workaround when libcxx.cxxabi doesn't exist (still exists for LLVM pre 12)
- adapter: update overrideLibcxx to account for a pkgs.stdenv that only has libcxx
### 58 package updates
- remove `NIX_LDFLAGS = "-l${stdenv.cc.libcxx.cxxabi.libName}` as no longer needed
- swift, nodejs_v8 remove libcxxabi references in the clang override
https://github.com/NixOS/nixpkgs/pull/292043
this equates to -ftrivial-auto-var-init=pattern
clang has removed support for -ftrivial-auto-var-init=zero and
are unlikely to re-add it, so use -ftrivial-auto-var-init=pattern
on both compilers if only to make behaviour more consistent
between the two.
add to pkgsExtraHardening's defaultHardeningFlags.
This is a small simplification of the control flow surrounding these cases. It should make it more obvious when each case happens, and also explicitly defines the current behaviour of --replace.
Example with `clangUseLLVM` which is the default when using `useLLVM`
```nix
config.replaceCrossStdenv = { buildPackages, baseStdenv }:
if baseStdenv.targetPlatform.useLLVM or false
then (buildPackages.stdenvAdapters.overrideCC baseStdenv buildPackages.llvmPackages_16.clangUseLLVM)
else baseStdenv;
```
The conditional necessary, otherwise the other sets(such as `pkgsCross.aarch64-multiplatform.llvmPackages`)
without `useLLVM` will use the stdenv without the necessary conditions to avoid infinite
recursion because of [targetLlvmLibraries](644b234e1c/pkgs/development/compilers/llvm/16/default.nix (L208))
usage.
[`replaceStdenv` is not used when cross-compiling](d77bda728d/pkgs/stdenv/cross/default.nix (L12-L13))
`replaceStdenv` uses an additional stage to replace the stdenv to avoid
infinite recursion and other issues but that should not be necessary for cross.
The aarch64 musl bootstrap tools are woefully outdated. Just getting
them to the point of being able to build new versions of themselves
required a number of hacks[1] that can be reverted once we have new
bootstrap tools, and before that it stdenv didn't even build for the
preceding three years.
[1]: https://github.com/NixOS/nixpkgs/pull/169764
So, following the script established by previous bootstrap tools
updates:
Files came from this Hydra build:
https://hydra.nixos.org/build/246470544
…which used nixpkgs revision dd5621df6d
to instantiate:
/nix/store/g480ass2vjmakaq03z7k2j95xnxh206a-stdenv-bootstrap-tools.drv
…and then built:
/nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools
I downloaded these files from Hydra and prefetched them into the Nix
store with the following commands:
STOREPATH=95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools
OPTIONS="--option binary-caches https://cache.nixos.org --option trusted-public-keys cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
nix --extra-experimental-features nix-command store prefetch-file \
file://$(nix --extra-experimental-features nix-command store add-file --name bootstrap-tools.tar.xz $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz)
nix --extra-experimental-features nix-command store prefetch-file --executable \
file://$(nix --extra-experimental-features nix-command store add-path --name busybox $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox)
These commands produced the following output:
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
Downloaded 'file:///nix/store/fm8ys5bb737j82xgyiciyzpcwmzyi9b2-bootstrap-tools.tar.xz' to '/nix/store/jml0gh0q2rnc9sgr87irz0jfbl0fq545-fm8ys5bb737j82xgyiciyzpcwmzyi9b2-bootstrap-tools.tar.xz' (hash 'sha256-ZY9IMOmx1VOn6uoFDpdJbTnPX59TEkrVCzWNtjQ8/QE=').
warning: you did not specify '--add-root'; the result might be removed by the garbage collector
Downloaded 'file:///nix/store/rn9fx55mw2s0qg90zdjxa9xn11nlmjpg-busybox' to '/nix/store/9qylz8gqll63pprwkwlyfs9g4zilak2m-rn9fx55mw2s0qg90zdjxa9xn11nlmjpg-busybox' (hash 'sha256-WuOaun7U5enbOy8SuuCo6G1fbGwsO16jhy/oM8K0lAs=').
I used the hashes from the output above to create the fetchurl
invocation which is part of this commit.
I then started the bootstrap with the following command:
nix --extra-experimental-features nix-command build -L -f . --arg localSystem '(import ./lib).systems.examples.aarch64-multiplatform-musl' hello
As @lovesegfault requested, here are the the sha256sums of all the
on-server components for extra verification:
sha256sum /nix/store/${STOREPATH}/on-server/*
…which produced the following output:
658f4830e9b1d553a7eaea050e97496d39cf5f9f53124ad50b358db6343cfd01 /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz
20cdfecb084ddb6b6b958f2b78fd2cc1d9641632f81ec7d5a48fae0a963ad0fa /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools/on-server/busybox
This PR updates the bootstrap tarballs for riscv64-linux with new Hydra-generated ones.
Fixes#275848 (bootstrap assembler too old to build gcc 13).
I'll be following the script used in #151399, #168199, #183487, and #188334.
Files came from [this](https://hydra.nixos.org/build/246376732#tabs-summary) Hydra build, which used nixpkgs revision 160cedc144 to instantiate:
```
/nix/store/cpiajh4l83b08pynwiwkpxj53d78pcxr-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu.drv
```
and then built:
```
/nix/store/8a92pj40awdw585mcb9dvm4nyb03k3q3-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu
```
I downloaded these files from Hydra and prefetched them into the nix store with the following commands:
```
STOREPATH=8a92pj40awdw585mcb9dvm4nyb03k3q3-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu
OPTIONS="--option binary-caches https://cache.nixos.org --option trusted-public-keys cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
nix store prefetch-file \
file://$(nix store add-file --name bootstrap-tools.tar.xz $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz)
nix store prefetch-file --executable \
file://$(nix store add-path --name busybox $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox)
```
These commands produced the following output:
```
Downloaded 'file:///nix/store/xs74wcxq3qc12plfc70fds8inbndhcfm-bootstrap-tools.tar.xz' to '/nix/store/3fal4gikp92013kac6rdmfbrch2s859b-xs74wcxq3qc12plfc70fds8inbndhcfm-bootstrap-tools.tar.xz' (hash 'sha256-0LxRd7fdafQezNJ+N2tuOfm0KEwgfRSts5fhP0e0r0s=').
Downloaded 'file:///nix/store/9ndpna6jrlac4y9fappdjm0sxx0g2bja-busybox' to '/nix/store/kb7wyy30y1gxcmdajljr26kxxac606qa-9ndpna6jrlac4y9fappdjm0sxx0g2bja-busybox' (hash 'sha256-OGO96QUzs2n5pGipn/V87AxzUY9OWKZl417nE8HdZIE=').
```
I used the hashes from the output above to create the `fetchurl` invocation which is part of this commit.
I then started the bootstrap with the following command:
```
nix build -L -f . --arg localSystem '(import ./lib).systems.examples.riscv64' hello
```
As @lovesegfault requested, here are the the `sha256sum`s of all the `on-server` components for extra verification:
```
sha256sum /nix/store/${STOREPATH}/on-server/*
```
which produced the following output:
```
d0bc5177b7dd69f41eccd27e376b6e39f9b4284c207d14adb397e13f47b4af4b /nix/store/8a92pj40awdw585mcb9dvm4nyb03k3q3-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu/on-server/bootstrap-tools.tar.xz
65f9433abb598f63c932d33351b14f686551512b1cece1e64c2d0e76aa0ec52e /nix/store/8a92pj40awdw585mcb9dvm4nyb03k3q3-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu/on-server/busybox
```
stdenv.cc may throw, e.g. in the case of pkgsCross.ghcjs where we must
not force it for the purpose of attribute accessing (`or` doesn't
implicitly tryEval…).
Regression introduced in 1a5bd697ad.
Before the change `pkgsLLVM` attributes were failing to pull in
`compiler-rt` on `x86_64-linux`:
$ nix build --no-link -f. pkgsLLVM.asciidoc-full
error:
error: attribute 'llvmPackages_13' missing
at pkgs/stdenv/adapters.nix:86:32:
85| inherit libcxx;
86| extraPackages = [ cxxabi pkgs.pkgsTargetTarget."llvmPackages_${lib.versions.major llvmLibcxxVersion}".compiler-rt ];
| ^
87| });
It happens because `pkgs.pkgsTargetTarget` are always empty for
cross-packages like `pkgsLLVM.`, `pkgsCross.*.` or
`--arg crossSystem '...'`.
Before the change `pkgsMusl.adobe-reader` was failing the interpreter:
$ nix-instantiate --eval --strict --expr 'with import ./. {}; builtins.tryEval pkgsMusl.adobe-reader'
error:
error: evaluation aborted with the following error message: 'unsupported platform for the pure Linux stdenv'
After the change `pkgsMusl.adobe-reader` returns catchable excation:
$ nix-instantiate --eval --strict --expr 'with import ./. {}; builtins.tryEval pkgsMusl.adobe-reader'
{ success = false; value = false; }
Noticed when was exploring `nixpkgs` for uncatchable evaluation errors.
Ideally those should only happen when there is a code bug in the use
site. In this case it's just a package with incompatible constraints for
`musl`.
Changed uncatchable `abort` to `throw`.
Aka `checkMeta` goes brrr.
Using the module system type checking works OK & generates good error messages.
The performance of using it however is terrible because of the value merging it does being very allocation heavy.
By implementing a very minimal type checker we can drastically improve the performance when nixpkgs is evaluated with `checkMeta = true`.
Without the change bootstrapTools build fails as:
https://cache.nixos.org/log/g5wyq9xqshan6m3kl21bjn1z88hx48rh-stdenv-bootstrap-tools.drv
error: install_name_tool: changing install names or rpaths can't be redone for: /nix/store/0hxg356h7lnl2hck23wrdpbib3cckx41-stdenv-bootstrap-tools/bin/tac (for architecture x86_64) because larger updated load commands do not fit (the program must be relinked, and you may need to use -headerpad or -headerpad_max_install_names)
