This reduces the number of Python builds in the bootstrap to two: a minimal build and a normal build. Both have LTO disabled, which is required due to missing LLVM LTO libraries. This is necessary to correctly enable LTO builds in Python because it needs `llvm-ar` from `stdenv.cc.cc.libllvm`, which does not exist in the bootstrap.
Fetchers can use the `curl` binary from the bootstrap tools. Allowing packages in the Darwin bootstrap to link curl makes any curl update cause a full rebuild on Darwin, which is undesirable.
After https://github.com/NixOS/nixpkgs/pull/322388, darwin.cctools is just Apple’s cctools again. The replacement for what the bootstrap tools wants is darwin.binutils-unwrapped.
GNU binutils is not preferred on Darwin, and newer versions have issues building. Make it an evaluation error to use it in the Darwin stdenv bootstrap.
- Only link binaries that exist for stage 0 cctools and LLVM bintools;
- Drop cctools-llvm in favor of the updated darwin.binutils; and
- Update llvm-manages Python overrides (needed for newer versions of LLVM).
I realized what rhelmot did in 61202561d9
(specify what packages just need `stdenvNoLibc`) is definitely the right
approach for this, and adjusted NetBSD and OpenBSD to likewise use it.
With that change, we don't need these confusing and ugly `*bsdCross`
package sets at all!
We can get rid of a lot more libc-related `*Cross`, and I will do so
soon, but this is the first step.
(adapted from commit 51f1ecaa59)
For a long time, we've had `crossLibcStdenv`, `*Cross` libc attributes,
and `*bsdCross` pre-libc package sets. This was always bad because
having "cross" things is "not declarative": the naming doesn't reflect
what packages *need* but rather how we *provide* something. This is
ugly, and creates needless friction between cross and native building.
Now, almost all of these `*Cross` attributes are gone: just these are
kept:
- Glibc's and Musl's are kept, because those packages are widely used
and I didn't want to risk changing the native builds of those at this
time.
- generic `libcCross`, `theadsCross`, and friends, because these relate
to the convolulted GCC bootstrap which still needs to be redone.
The BSD and obscure Linux or freestnanding libcs have conversely all
been made to use a new `stdenvNoLibc`, which is like the old
`crossLibcStdenv` except:
1. It usable for native and cross alike
2. It named according to what it *is* ("a standard environment without
libc but with a C compiler"), rather than some non-compositional
jargon ("the stdenv used for building libc when cross compiling",
yuck).
I should have done this change long ago, but I was stymied because of
"infinite recursions". The problem was that in too many cases we are
overriding `stdenv` to *remove* things we don't need, and this risks
cyles since those more minimal stdenvs are used to build things in the
more maximal stdenvs.
The solution is to pass `stage.nix` `stdenvNoCC`, so we can override to
*build up* rather than *tear down*. For now, the full `stdenv` is also
passed, so I don't need to change the native bootstraps, but I can see
this changing as we make things more uniform and clean those up.
Finally, the BSDs also had to be cleaned up, since they have a few
pre-libc dependencies, demanding a systematic approach. I realized what
rhelmot did in 61202561d9 (specify what
packages just need `stdenvNoLibc`) is definitely the right approach for
this, and adjusted NetBSD and OpenBSD to likewise use it.
Without the change `runPhase` fails on tarballs like
`diffoscope-269` that contain single top-level `-269` root as:
diffoscope> unpacking source archive /nix/store/p620nidkm73vrp0z6kk5krmrm4vg7bxd-diffoscope-269.tar.bz2
diffoscope> source root is -269
diffoscope> setting SOURCE_DATE_EPOCH to timestamp 1717143039 of file ./-269/tests/utils/versions.py
diffoscope> chmod: invalid mode: ‘-269’
diffoscope> Try 'chmod --help' for more information.
Currently `diffoscope-269` has a `sourceRoot = "./-269";` workaround to
bypass the failure.
Most Linux distributions are enabling this these days and it does
protect against real world vulnerabilities as demonstrated by
CVE-2018-16864 and CVE-2018-16865.
Fix#53753.
Information on llvm version support gleaned from
6609892a2d68e07da3e5092507a730
Information on gcc version support a lot harder to gather,
but both 32bit and 64bit arm do appear to be supported
based on the test suite.
A second take at eb28e5e72e, which was reverted for the extra logging
during the internals of `nix-shell -p`. This commit does the same
logging, but to $NIX_LOG_FD instead, which is echoed during any normal
build, but not during the internals of `nix-shell -p`.
[1]: eb28e5e72e
libiconv-darwin depends on Meson, which (indirectly) depends on
libiconv. When libiconv-darwin is set as libiconv, it will cause an
infinite recursion. Avoid the infinite recursion by using libiconvReal
in stage 1. Every stage after that can use libiconv-darwin.
The cc and bintools wrapper contained ad hoc bootstrapping logic for
expand-response-params (which was callPackage-ed in a let binding). This
lead to the strange situation that the bootstrapping logic related to
expand-response-params is split between the wrapper derivations (where
it is duplicated) and the actual stdenv bootstrapping.
To clean this up, the wrappers simply should take expand-response-params
as an ordinary input: They need an adjacent expand-response-params (i.e.
one that runs on their host platform), but don't care about the how.
Providing this is only problematic during stdenv bootstrapping where we
have to pull it from the previous stage at times.
We don't need to artificially make sure that we can execute the wrapper
scripts on the build platform by using stdenv's shell (which comes from
buildPackages) since our cross infrastructure will get us the wrapper
from buildPackages. The upside of this change is that cross-compiled
wrappers (e.g. pkgsCross.aarch64-multiplatform.gcc) will actually work
when executed!
For bootstrapping this is also not a problem, since we have a long
build->build platform chain so runtimeShell is just as good as
stdenvNoCC.shell. We do fall back to old ways, though, by explicitly
using the bootstrap-tools shell in stage2, so the adjacent bash is only
used from stage4 onwards. This is unnecessary in principle (I'll try
removing this hack in the future), but ensures this change causes zero
rebuilds.
