This PR adds Hydra-generated bootstrap tarballs for mips64el-linux. I'll be following the script established in https://github.com/NixOS/nixpkgs/pull/151399, which I previously used in https://github.com/NixOS/nixpkgs/pull/168199.
Files came from [this](https://hydra.nixos.org/build/182757245) Hydra build, which used nixpkgs revision ef3fe254f3 to instantiate:
```
/nix/store/a2bvv663wjnyhq8m7v84aspsd3sgf9h6-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64.drv
```
and then built:
```
/nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64
```
I downloaded these files from Hydra with the following commands:
```
STOREPATH=aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64
OPTIONS="--option binary-caches https://cache.nixos.org"
nix store add-file \
--name bootstrap-tools.tar.xz \
$(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz
nix store add-path \
--name busybox \
$(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox
```
I then prefetched them into `/nix/store` with:
```
$ nix store prefetch-file --executable file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox
Downloaded 'file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox' to '/nix/store/ai30ss23914syz6j8m95arkwffbbx44k-busybox' (hash 'sha256-sTE58ofjqAqX3Xtq1g9wDxzIe6Vo//GHbicfqJoivDI=').
$ nix store prefetch-file file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz
Downloaded 'file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz' to '/nix/store/nr6zjrbwbxcxs6brf042zsyqllwbwj9v-bootstrap-tools.tar.xz' (hash 'sha256-tTgjeXpd2YgnfP4JvRuO0bXd2j8GqzBcd57JI3wH9x0=').
```
And started the bootstrap with the following command (the `--arg localSystem` is needed because #161159 has not merged):
```
nix build -f . -L hello --arg localSystem '(import ./lib).systems.examples.mips64el-linux-gnuabi64'
```
As @lovesegfault requested in #151399, here are the the `sha256sum`s of all the `on-server` components for extra verification:
```
$ sha256sum /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/*
b53823797a5dd988277cfe09bd1b8ed1b5ddda3f06ab305c779ec9237c07f71d /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz
598e05abb69b2c1a0db46585cd2131212077c0937ce2a665daf3811f059ae767 /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox
```
During stdenv bootstrapping, coreutils is built twice. This makes
troubleshooting very difficult, because both packages have
name="coreutils", so it is a hassle to figure out "which coreutils am
I using / is not building"?
The first of these builds is used only in stage4, and is not part of
the final stdenv. Let's label that one with a different `name`
attribute to make it obvious which is which.
The usage of `makeStaticLibraries` in stdenv/linux/default.nix is
prefaced by this comment:
# Link GCC statically against GMP etc. This makes sense because
# these builds of the libraries are only used by GCC, so it
# reduces the size of the stdenv closure.
However "these builds of the libraries are only used by GCC" is not
actually true. As currently written, the stage4 coreutils links
against these customized, static-ified libraries.
Beside the fact that the code doesn't actually do what it says, this
causes other problems as well. One example is #168983, which arises
because have a dynamically-linked binary (coreutils) which is built
from statically-linked libraries (libgmp.a); doing this causes mayhem
on platforms where `-fstack-protector` needs an auxiliary
`libssp.{so,a}` library; we end up with link failures because some
parts of the resulting binary want `libssp.so` and other parts want
`libssp_nonshared.a`.
Let's make the code actually do what the comment says, by moving these
definitions into the `gcc-unwrapped` override. This will cause the
stage4-coreutils to link against libgmp dynamically, rather than
statically. For this reason this commit depends on the previous
commit, which allows that to be done without creating a forbidden
reference from stdenv-final to the bootstrap-files.
As explained in the comment, this ensures that stage4-coreutils does
not leak a reference to the bootstrap-files by way of libgmp. This
will allow the next patch in this series to build stage4-coreutils
using a dynamically-linked (rather than statically-linked) libgmp.
libtool's libtool.m4 script assumes that `file` is available, and can
be found at `/usr/bin/file` (this path is hardwired). Furthermore,
the script with this assumption is vendored into the ./configure
scripts of an enormous number of packages. Without this commit, you
will frequently see errors like this during the configurePhase with
the sandbox enabled:
./configure: line 9595: /usr/bin/file: command not found
Due mostly to luck, this error does not affect native compiles on
nixpkgs' two most popular platforms, x86_64-linux and aarch64-linux.
However it will cause incorrect linker flag detection and a failure to
generate shared libraries for sandboxed cross-builds to a x86_64-linux
host as well as any sandboxed build (cross or native) for the following
hosts: x86_64-freebsd, *-hpux, *-irix, mips64*-linux, powerpc*-linux,
s390x-linux, s390x-tpf, sparc-linux, and *-solaris.
This commit fixes the problem by adding an extra line to fixLibtool()
in pkgs/stdenv/generic/setup.sh. This extra line will scan the
unpacked source code for executable files named "configure" which
contain the following text:
'GNU Libtool is free software; you can redistribute it and/or modify'
This text is taken to be an indicator of a vendored libtool.m4. When
it is found, the configure script containing it is subjected to `sed
-i s_/usr/bin/file_file_` which replaces all occurrences of
`/usr/bin/file` with `file`.
Additionally, the `file` package is now considered to be part of
`stdenv`. It has been added to `common-path.nix` so that the `file`
binary will be found in the `$PATH` of every build, except for the
bootstrap-tools and the first few stages of stdenv boostrapping.
Verified no regressions under:
nix-build --arg pkgs 'import ./. {}' ./lib/tests/release.nix
This commit allows the following commands to complete, which should
enable Hydra to produce bootstrap-files for mips64el:
nix-build \
--option sandbox true \
--option sandbox-fallback false \
pkgs/top-level/release-cross.nix \
-A bootstrapTools.mips64el-linux-gnuabi64.build
nix-build \
--option sandbox true \
--option sandbox-fallback false \
. \
-A pkgsCross.mips64el-linux-gnuabi64.nix_2_4
canExecute is like isCompatible, but also checks that the Kernels are
_equal_, i.e. that both platforms use the same syscall interface. This
is crucial in order to actually be able to execute binaries for the
other platform.
isCompatible is dropped, since it has changed semantically and there's
no use case left in nixpkgs.
This PR adds Hydra-generated bootstrap tarballs for powerpc64le-linux.
I'll be following the script established in PR to tarballs.nixos.org.
Files came from this Hydra build:
https://hydra.nixos.org/build/172142499
Which used nixpkgs revision 49a83445c2
to instantiate:
/nix/store/gj272sd56gsj6qpyzh4njpfzwdhviliz-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu.drv
and then built:
/nix/store/n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu
I downloaded these files from Hydra with the following commands:
```
STOREPATH=n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu
OPTIONS="--option binary-caches https://cache.nixos.org"
nix store add-file \
--name bootstrap-tools.tar.xz \
$(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz
nix store add-path \
--name busybox \
$(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox
```
As @lovesegfault requested in #151399, here are the the `sha256sum`s
of all the `on-server` components for extra verification:
```
$ sha256sum /nix/store/n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/*
036d062869f7accf0ad89714d12029469dfe6af504f9b226d61eb7d808ad4735 /nix/store/n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/bootstrap-tools.tar.xz
6bc05832180f5075f4458c32eb0a5e2b673f605884dce01822be474f4e0a63ee /nix/store/n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/busybox
```
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
`--enable-deterministic-archives` is a GNU specific strip flag and
causes other strip implementations (for example LLVM's, see #138013)
to fail. Since strip failures are ignored, this means that stripping
doesn't work at all in certain situation (causing unnecessary
dependencies etc.).
To fix this, no longer pass `--enable-deterministic-archives`
unconditionally, but instead add it in a GNU binutils specific strip
wrapper only.
`commonStripFlags` was only used for this flag, so we can remove
it altogether.
Future work could be to make a generic strip wrapper, with support for
nix-support/strip-flags-{before,after} and NIX_STRIP_FLAGS_{BEFORE,AFTER}.
This possibly overkill and unnecessary though -- also with the
additional challenge of incorporating the darwin strip wrapper somehow.
Patch every `derivation` call in the bootsrap process to add it a
conditional `__contentAddressed` parameter.
That way, passing `contentAddressedByDefault` means that the entire
build closure of a system can be content addressed
These files never existed, so best to not leave the reference. If
someone want to step up to maintain this, that would be fine. I don’t
have the hardware to test these out. In addition, someone tried to use
the bootstrap-tools currently built by Hydra and found that they were
broken in some unclear way.
crossOverlays only apply to the packages being built, not the build
packages. It is useful when you don’t care what is used to build your
packages, just what is being built. The idea relies heavily on the
cross compiling infrastructure. Using this implies that we need to
create a cross stdenv.
02c09e0171 (NixOS/nixpkgs#44558) was reverted in
c981787db9 but, as it turns out, it fixed an issue
I didn't know about at the time: the values of `propagateDoc` options were
(and now again are) inconsistent with the underlying things those wrappers wrap
(see NixOS/nixpkgs#46119), which was (and now is) likely to produce more instances
of NixOS/nixpkgs#43547, if not now, then eventually as stdenv changes.
This patch (which is a simplified version of the original reverted patch) is the
simplest solution to this whole thing: it forces wrappers to directly inspect the
outputs of the things they are wrapping instead of making stdenv guess the correct
values.
