2005-02-21 20:42:07 +00:00
|
|
|
|
# This file constructs the standard build environment for the
|
2022-12-31 10:34:10 +00:00
|
|
|
|
# Linux platform. It's completely pure; that is, it relies on no
|
2005-02-21 20:42:07 +00:00
|
|
|
|
# external (non-Nix) tools, such as /usr/bin/gcc, and it contains a C
|
|
|
|
|
# compiler and linker that do not search in default locations,
|
|
|
|
|
# ensuring purity of components produced by it.
|
2022-12-31 10:34:10 +00:00
|
|
|
|
#
|
|
|
|
|
# It starts from prebuilt seed bootstrapFiles and creates a series of
|
|
|
|
|
# nixpkgs instances (stages) to gradually rebuild stdenv, which
|
|
|
|
|
# is used to build all other packages (including the bootstrapFiles).
|
|
|
|
|
#
|
|
|
|
|
# Goals of the bootstrap process:
|
|
|
|
|
# 1. final stdenv must not reference any of the bootstrap files.
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
# 2. final stdenv must not contain any of the bootstrap files.
|
2022-12-31 10:34:10 +00:00
|
|
|
|
# 3. final stdenv must not contain any of the files directly
|
|
|
|
|
# generated by the bootstrap code generators (assembler, linker,
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
# compiler).
|
2022-12-31 10:34:10 +00:00
|
|
|
|
#
|
|
|
|
|
# These goals ensure that final packages and final stdenv are built
|
|
|
|
|
# exclusively using nixpkgs package definitions and don't depend
|
|
|
|
|
# on bootstrapTools (via direct references, inclusion
|
|
|
|
|
# of copied code, or code compiled directly by bootstrapTools).
|
|
|
|
|
#
|
|
|
|
|
# Stages are described below along with their definitions.
|
|
|
|
|
#
|
|
|
|
|
# Debugging stdenv dependency graph:
|
|
|
|
|
# An useful tool to explore dependencies across stages is to use
|
|
|
|
|
# '__bootPackages' attribute of 'stdenv. Examples of last 3 stages:
|
|
|
|
|
# - stdenv
|
|
|
|
|
# - stdenv.__bootPackages.stdenv
|
|
|
|
|
# - stdenv.__bootPackages.stdenv.__bootPackages.stdenv
|
|
|
|
|
# - ... and so on.
|
|
|
|
|
#
|
|
|
|
|
# To explore build-time dependencies in graphical form one can use
|
|
|
|
|
# the following:
|
|
|
|
|
# $ nix-store --query --graph $(nix-instantiate -A stdenv) |
|
|
|
|
|
# grep -P -v '[.]sh|[.]patch|bash|[.]tar' | # avoid clutter
|
|
|
|
|
# dot -Tsvg > stdenv-final.svg
|
|
|
|
|
#
|
|
|
|
|
# To find all the packages built by a particular stdenv instance:
|
|
|
|
|
# $ for stage in 0 1 2 3 4; do
|
|
|
|
|
# echo "stage${stage} used in:"
|
|
|
|
|
# nix-store --query --graph $(nix-instantiate -A stdenv) |
|
|
|
|
|
# grep -P ".*bootstrap-stage${stage}-stdenv.*->.*" |
|
|
|
|
|
# sed 's/"[0-9a-z]\{32\}-/"/g'
|
|
|
|
|
# done
|
|
|
|
|
#
|
|
|
|
|
# To verify which stdenv was used to build a given final package:
|
|
|
|
|
# $ nix-store --query --graph $(nix-instantiate -A stdenv) |
|
|
|
|
|
# grep -P -v '[.]sh|[.]patch|bash|[.]tar' |
|
|
|
|
|
# grep -P '.*stdenv.*->.*glibc-2'
|
|
|
|
|
# "...-bootstrap-stage2-stdenv-linux.drv" -> "...-glibc-2.35-224.drv";
|
|
|
|
|
#
|
|
|
|
|
# For a TUI (rather than CLI) view, you can use:
|
|
|
|
|
#
|
|
|
|
|
# $ nix-tree --derivation $(nix-instantiate -A stdenv)
|
2016-12-16 13:22:02 +00:00
|
|
|
|
{ lib
|
2018-12-05 03:06:46 +00:00
|
|
|
|
, localSystem, crossSystem, config, overlays, crossOverlays ? []
|
2016-12-24 18:55:11 +00:00
|
|
|
|
|
2018-01-12 22:19:22 +00:00
|
|
|
|
, bootstrapFiles ?
|
|
|
|
|
let table = {
|
2019-08-13 21:52:01 +00:00
|
|
|
|
glibc = {
|
|
|
|
|
i686-linux = import ./bootstrap-files/i686.nix;
|
|
|
|
|
x86_64-linux = import ./bootstrap-files/x86_64.nix;
|
|
|
|
|
armv5tel-linux = import ./bootstrap-files/armv5tel.nix;
|
|
|
|
|
armv6l-linux = import ./bootstrap-files/armv6l.nix;
|
|
|
|
|
armv7l-linux = import ./bootstrap-files/armv7l.nix;
|
|
|
|
|
aarch64-linux = import ./bootstrap-files/aarch64.nix;
|
2023-04-02 06:16:46 +00:00
|
|
|
|
mipsel-linux = import ./bootstrap-files/mipsel.nix;
|
2022-08-17 20:40:33 +00:00
|
|
|
|
mips64el-linux = import
|
|
|
|
|
(if localSystem.isMips64n32
|
|
|
|
|
then ./bootstrap-files/mips64el-n32.nix
|
|
|
|
|
else ./bootstrap-files/mips64el.nix);
|
2022-04-07 00:30:43 +00:00
|
|
|
|
powerpc64le-linux = import ./bootstrap-files/powerpc64le.nix;
|
2021-02-27 18:53:22 +00:00
|
|
|
|
riscv64-linux = import ./bootstrap-files/riscv64.nix;
|
2018-01-12 22:19:22 +00:00
|
|
|
|
};
|
2019-08-13 21:52:01 +00:00
|
|
|
|
musl = {
|
|
|
|
|
aarch64-linux = import ./bootstrap-files/aarch64-musl.nix;
|
|
|
|
|
armv6l-linux = import ./bootstrap-files/armv6l-musl.nix;
|
|
|
|
|
x86_64-linux = import ./bootstrap-files/x86_64-musl.nix;
|
2018-01-12 22:19:22 +00:00
|
|
|
|
};
|
|
|
|
|
};
|
2019-04-10 22:38:20 +00:00
|
|
|
|
|
|
|
|
|
# Try to find an architecture compatible with our current system. We
|
|
|
|
|
# just try every bootstrap we’ve got and test to see if it is
|
|
|
|
|
# compatible with or current architecture.
|
|
|
|
|
getCompatibleTools = lib.foldl (v: system:
|
|
|
|
|
if v != null then v
|
2022-04-26 20:17:48 +00:00
|
|
|
|
else if localSystem.canExecute (lib.systems.elaborate { inherit system; }) then archLookupTable.${system}
|
2019-04-10 22:38:20 +00:00
|
|
|
|
else null) null (lib.attrNames archLookupTable);
|
|
|
|
|
|
2018-01-12 22:19:22 +00:00
|
|
|
|
archLookupTable = table.${localSystem.libc}
|
|
|
|
|
or (abort "unsupported libc for the pure Linux stdenv");
|
2019-04-10 22:38:20 +00:00
|
|
|
|
files = archLookupTable.${localSystem.system} or (if getCompatibleTools != null then getCompatibleTools
|
|
|
|
|
else (abort "unsupported platform for the pure Linux stdenv"));
|
2018-01-12 22:19:22 +00:00
|
|
|
|
in files
|
2016-03-03 12:46:21 +00:00
|
|
|
|
}:
|
2006-08-23 16:18:02 +00:00
|
|
|
|
|
2018-12-05 03:06:46 +00:00
|
|
|
|
assert crossSystem == localSystem;
|
2016-11-30 23:51:13 +00:00
|
|
|
|
|
2016-12-24 15:38:56 +00:00
|
|
|
|
let
|
2021-01-24 22:51:12 +00:00
|
|
|
|
inherit (localSystem) system;
|
2006-08-23 16:18:02 +00:00
|
|
|
|
|
2023-01-10 10:19:17 +00:00
|
|
|
|
isFromNixpkgs = pkg: !(isFromBootstrapFiles pkg);
|
|
|
|
|
isFromBootstrapFiles =
|
|
|
|
|
pkg: pkg.passthru.isFromBootstrapFiles or false;
|
|
|
|
|
isBuiltByNixpkgsCompiler =
|
|
|
|
|
pkg: isFromNixpkgs pkg && isFromNixpkgs pkg.stdenv.cc.cc;
|
|
|
|
|
isBuiltByBootstrapFilesCompiler =
|
|
|
|
|
pkg: isFromNixpkgs pkg && isFromBootstrapFiles pkg.stdenv.cc.cc;
|
|
|
|
|
|
2023-01-09 08:09:53 +00:00
|
|
|
|
commonGccOverrides = {
|
|
|
|
|
# Use a deterministically built compiler
|
|
|
|
|
# see https://github.com/NixOS/nixpkgs/issues/108475 for context
|
|
|
|
|
reproducibleBuild = true;
|
|
|
|
|
profiledCompiler = false;
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
|
|
|
|
|
# It appears that libcc1 (which is not a g++ plugin; it is a gdb plugin) gets linked against
|
|
|
|
|
# the libstdc++ from the compiler that *built* g++, not the libstdc++ which was just built.
|
|
|
|
|
# This causes a reference chain from stdenv to the bootstrapFiles:
|
|
|
|
|
#
|
|
|
|
|
# stdenv -> gcc-lib -> xgcc-lib -> bootstrapFiles
|
|
|
|
|
#
|
|
|
|
|
disableGdbPlugin = true;
|
2023-01-09 08:09:53 +00:00
|
|
|
|
};
|
|
|
|
|
|
2009-03-25 17:34:38 +00:00
|
|
|
|
commonPreHook =
|
|
|
|
|
''
|
2016-02-29 11:10:26 +00:00
|
|
|
|
export NIX_ENFORCE_PURITY="''${NIX_ENFORCE_PURITY-1}"
|
2016-03-15 16:35:07 +00:00
|
|
|
|
export NIX_ENFORCE_NO_NATIVE="''${NIX_ENFORCE_NO_NATIVE-1}"
|
2023-02-06 20:49:02 +00:00
|
|
|
|
${lib.optionalString (system == "x86_64-linux") "NIX_LIB64_IN_SELF_RPATH=1"}
|
|
|
|
|
${lib.optionalString (system == "mipsel-linux") "NIX_LIB32_IN_SELF_RPATH=1"}
|
2009-03-25 17:34:38 +00:00
|
|
|
|
'';
|
|
|
|
|
|
|
|
|
|
|
2005-02-21 20:42:07 +00:00
|
|
|
|
# The bootstrap process proceeds in several steps.
|
|
|
|
|
|
2012-12-28 15:41:56 +00:00
|
|
|
|
|
2014-08-23 19:26:37 +00:00
|
|
|
|
# Create a standard environment by downloading pre-built binaries of
|
|
|
|
|
# coreutils, GCC, etc.
|
2005-02-22 09:55:03 +00:00
|
|
|
|
|
2012-12-28 15:41:56 +00:00
|
|
|
|
|
2009-02-01 21:44:56 +00:00
|
|
|
|
# Download and unpack the bootstrap tools (coreutils, GCC, Glibc, ...).
|
2023-01-10 10:19:17 +00:00
|
|
|
|
bootstrapTools = (import (if localSystem.libc == "musl" then ./bootstrap-tools-musl else ./bootstrap-tools) {
|
2021-04-26 15:21:17 +00:00
|
|
|
|
inherit system bootstrapFiles;
|
2023-01-10 10:19:17 +00:00
|
|
|
|
extraAttrs = lib.optionalAttrs config.contentAddressedByDefault {
|
|
|
|
|
__contentAddressed = true;
|
|
|
|
|
outputHashAlgo = "sha256";
|
|
|
|
|
outputHashMode = "recursive";
|
|
|
|
|
};
|
|
|
|
|
}) // { passthru.isFromBootstrapFiles = true; };
|
2012-12-28 15:41:56 +00:00
|
|
|
|
|
2018-01-12 00:15:40 +00:00
|
|
|
|
getLibc = stage: stage.${localSystem.libc};
|
2012-12-28 15:41:56 +00:00
|
|
|
|
|
2004-11-04 12:19:32 +00:00
|
|
|
|
|
2009-02-01 21:44:56 +00:00
|
|
|
|
# This function builds the various standard environments used during
|
2014-08-23 19:26:37 +00:00
|
|
|
|
# the bootstrap. In all stages, we build an stdenv and the package
|
|
|
|
|
# set that can be built with that stdenv.
|
2016-12-24 15:07:20 +00:00
|
|
|
|
stageFun = prevStage:
|
2017-08-15 15:30:45 +00:00
|
|
|
|
{ name, overrides ? (self: super: {}), extraNativeBuildInputs ? [] }:
|
2004-11-04 12:19:32 +00:00
|
|
|
|
|
2014-08-23 19:26:37 +00:00
|
|
|
|
let
|
2014-08-24 16:08:23 +00:00
|
|
|
|
|
|
|
|
|
thisStdenv = import ../generic {
|
2018-02-01 00:00:00 +00:00
|
|
|
|
name = "${name}-stdenv-linux";
|
2017-07-06 01:47:48 +00:00
|
|
|
|
buildPlatform = localSystem;
|
|
|
|
|
hostPlatform = localSystem;
|
|
|
|
|
targetPlatform = localSystem;
|
2017-08-15 15:30:45 +00:00
|
|
|
|
inherit config extraNativeBuildInputs;
|
2014-08-24 16:08:23 +00:00
|
|
|
|
preHook =
|
|
|
|
|
''
|
|
|
|
|
# Don't patch #!/interpreter because it leads to retained
|
|
|
|
|
# dependencies on the bootstrapTools in the final stdenv.
|
|
|
|
|
dontPatchShebangs=1
|
|
|
|
|
${commonPreHook}
|
|
|
|
|
'';
|
2016-02-24 10:01:11 +00:00
|
|
|
|
shell = "${bootstrapTools}/bin/bash";
|
2014-09-14 17:32:58 +00:00
|
|
|
|
initialPath = [bootstrapTools];
|
2016-02-27 19:27:24 +00:00
|
|
|
|
|
|
|
|
|
fetchurlBoot = import ../../build-support/fetchurl/boot.nix {
|
|
|
|
|
inherit system;
|
2014-08-24 16:08:23 +00:00
|
|
|
|
};
|
2014-09-10 23:18:14 +00:00
|
|
|
|
|
2019-04-24 03:48:22 +00:00
|
|
|
|
cc = if prevStage.gcc-unwrapped == null
|
2015-06-15 17:26:52 +00:00
|
|
|
|
then null
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
else (lib.makeOverridable (import ../../build-support/cc-wrapper) {
|
2018-02-01 00:00:00 +00:00
|
|
|
|
name = "${name}-gcc-wrapper";
|
2014-09-10 23:18:14 +00:00
|
|
|
|
nativeTools = false;
|
|
|
|
|
nativeLibc = false;
|
2017-06-29 16:06:55 +00:00
|
|
|
|
buildPackages = lib.optionalAttrs (prevStage ? stdenv) {
|
|
|
|
|
inherit (prevStage) stdenv;
|
|
|
|
|
};
|
2016-12-24 15:07:20 +00:00
|
|
|
|
cc = prevStage.gcc-unwrapped;
|
2017-08-26 15:43:30 +00:00
|
|
|
|
bintools = prevStage.binutils;
|
2015-05-11 21:30:13 +00:00
|
|
|
|
isGNU = true;
|
2018-01-12 00:15:40 +00:00
|
|
|
|
libc = getLibc prevStage;
|
2021-01-24 03:02:59 +00:00
|
|
|
|
inherit lib;
|
2017-08-26 15:43:30 +00:00
|
|
|
|
inherit (prevStage) coreutils gnugrep;
|
2017-09-20 15:07:52 +00:00
|
|
|
|
stdenvNoCC = prevStage.ccWrapperStdenv;
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
}).overrideAttrs(a: lib.optionalAttrs (prevStage.gcc-unwrapped.passthru.isXgcc or false) {
|
|
|
|
|
# This affects only `xgcc` (the compiler which compiles the final compiler).
|
|
|
|
|
postFixup = (a.postFixup or "") + ''
|
|
|
|
|
echo "--sysroot=${lib.getDev (getLibc prevStage)}" >> $out/nix-support/cc-cflags
|
|
|
|
|
'';
|
|
|
|
|
});
|
2014-09-10 23:18:14 +00:00
|
|
|
|
|
2016-12-19 16:10:47 +00:00
|
|
|
|
overrides = self: super: (overrides self super) // { fetchurl = thisStdenv.fetchurlBoot; };
|
2014-08-23 18:34:01 +00:00
|
|
|
|
};
|
2004-11-04 12:19:32 +00:00
|
|
|
|
|
2016-12-16 13:22:02 +00:00
|
|
|
|
in {
|
2016-12-24 18:55:11 +00:00
|
|
|
|
inherit config overlays;
|
2016-12-16 13:22:02 +00:00
|
|
|
|
stdenv = thisStdenv;
|
|
|
|
|
};
|
2009-02-01 21:44:56 +00:00
|
|
|
|
|
2016-12-24 15:38:56 +00:00
|
|
|
|
in
|
2023-01-10 10:19:17 +00:00
|
|
|
|
assert bootstrapTools.passthru.isFromBootstrapFiles or false; # sanity check
|
2016-12-24 15:38:56 +00:00
|
|
|
|
[
|
|
|
|
|
|
|
|
|
|
({}: {
|
2016-12-24 15:28:40 +00:00
|
|
|
|
__raw = true;
|
|
|
|
|
|
2016-12-24 15:07:20 +00:00
|
|
|
|
gcc-unwrapped = null;
|
2014-09-10 23:18:14 +00:00
|
|
|
|
binutils = null;
|
|
|
|
|
coreutils = null;
|
2016-01-24 05:18:32 +00:00
|
|
|
|
gnugrep = null;
|
2016-12-24 15:38:56 +00:00
|
|
|
|
})
|
2016-12-24 15:07:20 +00:00
|
|
|
|
|
|
|
|
|
# Build a dummy stdenv with no GCC or working fetchurl. This is
|
|
|
|
|
# because we need a stdenv to build the GCC wrapper and fetchurl.
|
2016-12-24 15:38:56 +00:00
|
|
|
|
(prevStage: stageFun prevStage {
|
2018-02-01 00:00:00 +00:00
|
|
|
|
name = "bootstrap-stage0";
|
2012-12-28 15:41:56 +00:00
|
|
|
|
|
2016-12-19 16:10:47 +00:00
|
|
|
|
overrides = self: super: {
|
2016-12-24 15:28:40 +00:00
|
|
|
|
# We thread stage0's stdenv through under this name so downstream stages
|
|
|
|
|
# can use it for wrapping gcc too. This way, downstream stages don't need
|
|
|
|
|
# to refer to this stage directly, which violates the principle that each
|
|
|
|
|
# stage should only access the stage that came before it.
|
|
|
|
|
ccWrapperStdenv = self.stdenv;
|
2014-08-23 19:26:37 +00:00
|
|
|
|
# The Glibc include directory cannot have the same prefix as the
|
|
|
|
|
# GCC include directory, since GCC gets confused otherwise (it
|
|
|
|
|
# will search the Glibc headers before the GCC headers). So
|
|
|
|
|
# create a dummy Glibc here, which will be used in the stdenv of
|
|
|
|
|
# stage1.
|
2018-01-12 00:15:40 +00:00
|
|
|
|
${localSystem.libc} = self.stdenv.mkDerivation {
|
2021-05-10 15:12:59 +00:00
|
|
|
|
pname = "bootstrap-stage0-${localSystem.libc}";
|
2022-05-21 16:58:22 +00:00
|
|
|
|
strictDeps = true;
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
version = "bootstrapFiles";
|
2022-05-14 01:13:49 +00:00
|
|
|
|
enableParallelBuilding = true;
|
2014-08-23 19:26:37 +00:00
|
|
|
|
buildCommand = ''
|
|
|
|
|
mkdir -p $out
|
|
|
|
|
ln -s ${bootstrapTools}/lib $out/lib
|
2018-01-11 21:09:34 +00:00
|
|
|
|
'' + lib.optionalString (localSystem.libc == "glibc") ''
|
2014-08-23 19:26:37 +00:00
|
|
|
|
ln -s ${bootstrapTools}/include-glibc $out/include
|
2018-01-11 21:09:34 +00:00
|
|
|
|
'' + lib.optionalString (localSystem.libc == "musl") ''
|
|
|
|
|
ln -s ${bootstrapTools}/include-libc $out/include
|
2014-08-23 19:26:37 +00:00
|
|
|
|
'';
|
2023-01-10 10:19:17 +00:00
|
|
|
|
passthru.isFromBootstrapFiles = true;
|
2014-08-23 19:26:37 +00:00
|
|
|
|
};
|
2016-12-24 15:07:20 +00:00
|
|
|
|
gcc-unwrapped = bootstrapTools;
|
2017-08-26 15:43:30 +00:00
|
|
|
|
binutils = import ../../build-support/bintools-wrapper {
|
2018-02-01 00:00:00 +00:00
|
|
|
|
name = "bootstrap-stage0-binutils-wrapper";
|
2017-08-26 15:43:30 +00:00
|
|
|
|
nativeTools = false;
|
|
|
|
|
nativeLibc = false;
|
|
|
|
|
buildPackages = { };
|
2018-01-12 00:15:40 +00:00
|
|
|
|
libc = getLibc self;
|
2021-01-24 03:02:59 +00:00
|
|
|
|
inherit lib;
|
2017-08-26 15:43:30 +00:00
|
|
|
|
inherit (self) stdenvNoCC coreutils gnugrep;
|
|
|
|
|
bintools = bootstrapTools;
|
|
|
|
|
};
|
2016-12-24 15:07:20 +00:00
|
|
|
|
coreutils = bootstrapTools;
|
|
|
|
|
gnugrep = bootstrapTools;
|
2004-11-04 12:19:32 +00:00
|
|
|
|
};
|
2016-12-24 15:38:56 +00:00
|
|
|
|
})
|
2005-02-21 20:42:07 +00:00
|
|
|
|
|
|
|
|
|
|
|
|
|
|
# Create the first "real" standard environment. This one consists
|
2009-02-01 21:44:56 +00:00
|
|
|
|
# of bootstrap tools only, and a minimal Glibc to keep the GCC
|
|
|
|
|
# configure script happy.
|
2014-08-24 13:55:14 +00:00
|
|
|
|
#
|
|
|
|
|
# For clarity, we only use the previous stage when specifying these
|
|
|
|
|
# stages. So stageN should only ever have references for stage{N-1}.
|
|
|
|
|
#
|
|
|
|
|
# If we ever need to use a package from more than one stage back, we
|
|
|
|
|
# simply re-export those packages in the middle stage(s) using the
|
|
|
|
|
# overrides attribute and the inherit syntax.
|
2023-01-10 10:19:17 +00:00
|
|
|
|
(prevStage:
|
|
|
|
|
# previous stage0 stdenv:
|
|
|
|
|
assert isFromBootstrapFiles prevStage.binutils.bintools;
|
|
|
|
|
assert isFromBootstrapFiles prevStage."${localSystem.libc}";
|
|
|
|
|
assert isFromBootstrapFiles prevStage.gcc-unwrapped;
|
|
|
|
|
assert isFromBootstrapFiles prevStage.coreutils;
|
|
|
|
|
assert isFromBootstrapFiles prevStage.gnugrep;
|
|
|
|
|
stageFun prevStage {
|
2018-02-01 00:00:00 +00:00
|
|
|
|
name = "bootstrap-stage1";
|
2014-09-10 23:18:14 +00:00
|
|
|
|
|
2014-08-23 19:26:37 +00:00
|
|
|
|
# Rebuild binutils to use from stage2 onwards.
|
2016-12-19 16:10:47 +00:00
|
|
|
|
overrides = self: super: {
|
2018-09-01 20:30:41 +00:00
|
|
|
|
binutils-unwrapped = super.binutils-unwrapped.override {
|
2022-02-10 19:50:33 +00:00
|
|
|
|
enableGold = false;
|
2018-09-01 20:30:41 +00:00
|
|
|
|
};
|
2016-12-24 15:28:40 +00:00
|
|
|
|
inherit (prevStage)
|
|
|
|
|
ccWrapperStdenv
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
gcc-unwrapped coreutils gnugrep binutils;
|
2018-01-12 00:15:40 +00:00
|
|
|
|
|
|
|
|
|
${localSystem.libc} = getLibc prevStage;
|
2014-09-10 23:18:14 +00:00
|
|
|
|
|
2014-09-04 14:17:39 +00:00
|
|
|
|
# A threaded perl build needs glibc/libpthread_nonshared.a,
|
|
|
|
|
# which is not included in bootstrapTools, so disable threading.
|
|
|
|
|
# This is not an issue for the final stdenv, because this perl
|
|
|
|
|
# won't be included in the final stdenv and won't be exported to
|
|
|
|
|
# top-level pkgs as an override either.
|
2022-09-26 20:04:58 +00:00
|
|
|
|
perl = super.perl.override { enableThreading = false; enableCrypt = false; };
|
2010-01-19 11:25:33 +00:00
|
|
|
|
};
|
2016-12-24 15:38:56 +00:00
|
|
|
|
})
|
2005-02-21 20:42:07 +00:00
|
|
|
|
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
# First rebuild of gcc; this is linked against all sorts of junk
|
|
|
|
|
# from the bootstrap-files, but we only care about the code that
|
|
|
|
|
# this compiler *emits*. The `gcc` binary produced in this stage
|
|
|
|
|
# is not part of the final stdenv.
|
|
|
|
|
(prevStage:
|
|
|
|
|
assert isBuiltByBootstrapFilesCompiler prevStage.binutils-unwrapped;
|
|
|
|
|
assert isFromBootstrapFiles prevStage."${localSystem.libc}";
|
|
|
|
|
assert isFromBootstrapFiles prevStage.gcc-unwrapped;
|
|
|
|
|
assert isFromBootstrapFiles prevStage.coreutils;
|
|
|
|
|
assert isFromBootstrapFiles prevStage.gnugrep;
|
2023-04-06 05:15:10 +00:00
|
|
|
|
assert isBuiltByBootstrapFilesCompiler prevStage.patchelf;
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
stageFun prevStage {
|
|
|
|
|
name = "bootstrap-stage-xgcc";
|
|
|
|
|
overrides = final: prev: {
|
2023-04-06 05:15:42 +00:00
|
|
|
|
inherit (prevStage) ccWrapperStdenv coreutils gnugrep gettext bison texinfo zlib gnum4 perl patchelf;
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
${localSystem.libc} = getLibc prevStage;
|
|
|
|
|
gmp = prev.gmp.override { cxx = false; };
|
|
|
|
|
gcc-unwrapped =
|
|
|
|
|
(prev.gcc-unwrapped.override (commonGccOverrides // {
|
|
|
|
|
# The most logical name for this package would be something like
|
|
|
|
|
# "gcc-stage1". Unfortunately "stage" is already reserved for the
|
|
|
|
|
# layers of stdenv, so using "stage" in the name of this package
|
|
|
|
|
# would cause massive confusion.
|
|
|
|
|
#
|
|
|
|
|
# Gcc calls its "stage1" compiler `xgcc` (--disable-bootstrap results
|
|
|
|
|
# in `xgcc` being copied to $prefix/bin/gcc). So we imitate that.
|
|
|
|
|
#
|
|
|
|
|
name = "xgcc";
|
|
|
|
|
|
2023-04-04 06:35:03 +00:00
|
|
|
|
# xgcc uses ld linked against nixpkgs' glibc and gcc built
|
|
|
|
|
# against bootstrapTools glibc. We can't allow loading
|
|
|
|
|
# $out/libexec/gcc/x86_64-unknown-linux-gnu/13.0.1/liblto_plugin.so
|
|
|
|
|
# to mix libc.so:
|
|
|
|
|
# ...-binutils-patchelfed-ld-2.40/bin/ld: ...-xgcc-13.0.0/libexec/gcc/x86_64-unknown-linux-gnu/13.0.1/liblto_plugin.so:
|
|
|
|
|
# error loading plugin: ...-bootstrap-tools/lib/libpthread.so.0: undefined symbol: __libc_vfork, version GLIBC_PRIVATE
|
|
|
|
|
enableLTO = false;
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
})).overrideAttrs (a: {
|
|
|
|
|
|
|
|
|
|
# This signals to cc-wrapper (as overridden above in this file) to add `--sysroot`
|
|
|
|
|
# to `$out/nix-support/cc-cflags`.
|
|
|
|
|
passthru = a.passthru // { isXgcc = true; };
|
|
|
|
|
|
|
|
|
|
# Gcc will look for the C library headers in
|
|
|
|
|
#
|
|
|
|
|
# ${with_build_sysroot}${native_system_header_dir}
|
|
|
|
|
#
|
|
|
|
|
# The ordinary gcc expression sets `--with-build-sysroot=/` and sets
|
|
|
|
|
# `native-system-header-dir` to `"${lib.getDev stdenv.cc.libc}/include`.
|
|
|
|
|
#
|
|
|
|
|
# Unfortunately the value of "--with-native-system-header-dir=" gets "burned in" to the
|
|
|
|
|
# compiler, and it is quite difficult to get the compiler to change or ignore it
|
|
|
|
|
# afterwards. On the other hand, the `sysroot` is very easy to change; you can just pass
|
|
|
|
|
# a `--sysroot` flag to `gcc`.
|
|
|
|
|
#
|
|
|
|
|
# So we override the expression to remove the default settings for these flags, and
|
|
|
|
|
# replace them such that the concatenated value will be the same as before, but we split
|
|
|
|
|
# the value between the two variables differently: `--native-system-header-dir=/include`,
|
|
|
|
|
# and `--with-build-sysroot=${lib.getDev stdenv.cc.libc}`.
|
|
|
|
|
#
|
|
|
|
|
configureFlags = (a.configureFlags or []) ++ [
|
|
|
|
|
"--with-native-system-header-dir=/include"
|
|
|
|
|
"--with-build-sysroot=${lib.getDev final.stdenv.cc.libc}"
|
|
|
|
|
];
|
|
|
|
|
|
|
|
|
|
# This is a separate phase because gcc assembles its phase scripts
|
|
|
|
|
# in bash instead of nix (we should fix that).
|
|
|
|
|
preFixupPhases = (a.preFixupPhases or []) ++ [ "preFixupXgccPhase" ];
|
|
|
|
|
|
|
|
|
|
# This is needed to prevent "error: cycle detected in build of '...-xgcc-....drv'
|
|
|
|
|
# in the references of output 'lib' from output 'out'"
|
|
|
|
|
preFixupXgccPhase = ''
|
|
|
|
|
find $lib/lib/ -name \*.so\* -exec patchelf --shrink-rpath {} \; || true
|
|
|
|
|
'';
|
|
|
|
|
});
|
|
|
|
|
};
|
|
|
|
|
})
|
2014-01-07 16:46:47 +00:00
|
|
|
|
|
2014-08-23 19:26:37 +00:00
|
|
|
|
# 2nd stdenv that contains our own rebuilt binutils and is used for
|
|
|
|
|
# compiling our own Glibc.
|
2022-12-31 10:34:10 +00:00
|
|
|
|
#
|
2023-01-10 10:19:17 +00:00
|
|
|
|
(prevStage:
|
|
|
|
|
# previous stage1 stdenv:
|
|
|
|
|
assert isBuiltByBootstrapFilesCompiler prevStage.binutils-unwrapped;
|
|
|
|
|
assert isFromBootstrapFiles prevStage."${localSystem.libc}";
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
assert isBuiltByBootstrapFilesCompiler prevStage.gcc-unwrapped;
|
2023-01-10 10:19:17 +00:00
|
|
|
|
assert isFromBootstrapFiles prevStage.coreutils;
|
|
|
|
|
assert isFromBootstrapFiles prevStage.gnugrep;
|
2023-04-06 05:15:42 +00:00
|
|
|
|
assert isBuiltByBootstrapFilesCompiler prevStage.patchelf;
|
2023-01-10 10:19:17 +00:00
|
|
|
|
stageFun prevStage {
|
2018-02-01 00:00:00 +00:00
|
|
|
|
name = "bootstrap-stage2";
|
2014-09-10 23:18:14 +00:00
|
|
|
|
|
2016-12-19 16:10:47 +00:00
|
|
|
|
overrides = self: super: {
|
2016-12-24 15:07:20 +00:00
|
|
|
|
inherit (prevStage)
|
2023-01-10 11:42:27 +00:00
|
|
|
|
ccWrapperStdenv gettext
|
2017-08-26 15:43:30 +00:00
|
|
|
|
gcc-unwrapped coreutils gnugrep
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
perl gnum4 bison texinfo which;
|
2019-08-18 10:46:38 +00:00
|
|
|
|
dejagnu = super.dejagnu.overrideAttrs (a: { doCheck = false; } );
|
2019-08-18 18:58:56 +00:00
|
|
|
|
|
|
|
|
|
# We need libidn2 and its dependency libunistring as glibc dependency.
|
|
|
|
|
# To avoid the cycle, we build against bootstrap libc, nuke references,
|
|
|
|
|
# and use the result as input for our final glibc. We also pass this pair
|
|
|
|
|
# through, so the final package-set uses exactly the same builds.
|
|
|
|
|
libunistring = super.libunistring.overrideAttrs (attrs: {
|
|
|
|
|
postFixup = attrs.postFixup or "" + ''
|
|
|
|
|
${self.nukeReferences}/bin/nuke-refs "$out"/lib/lib*.so.*.*
|
|
|
|
|
'';
|
|
|
|
|
# Apparently iconv won't work with bootstrap glibc, but it will be used
|
|
|
|
|
# with glibc built later where we keep *this* build of libunistring,
|
|
|
|
|
# so we need to trick it into supporting libiconv.
|
2022-12-08 19:11:35 +00:00
|
|
|
|
env = attrs.env or {} // { am_cv_func_iconv_works = "yes"; };
|
2019-08-18 18:58:56 +00:00
|
|
|
|
});
|
|
|
|
|
libidn2 = super.libidn2.overrideAttrs (attrs: {
|
|
|
|
|
postFixup = attrs.postFixup or "" + ''
|
|
|
|
|
${self.nukeReferences}/bin/nuke-refs -e '${lib.getLib self.libunistring}' \
|
|
|
|
|
"$out"/lib/lib*.so.*.*
|
|
|
|
|
'';
|
|
|
|
|
});
|
|
|
|
|
|
2014-08-23 19:26:37 +00:00
|
|
|
|
# This also contains the full, dynamically linked, final Glibc.
|
2017-08-26 15:43:30 +00:00
|
|
|
|
binutils = prevStage.binutils.override {
|
|
|
|
|
# Rewrap the binutils with the new glibc, so both the next
|
|
|
|
|
# stage's wrappers use it.
|
2018-01-12 00:15:40 +00:00
|
|
|
|
libc = getLibc self;
|
2021-06-30 17:47:13 +00:00
|
|
|
|
|
|
|
|
|
# Unfortunately, when building gcc in the next stage, its LTO plugin
|
|
|
|
|
# would use the final libc but `ld` would use the bootstrap one,
|
|
|
|
|
# and that can fail to load. Therefore we upgrade `ld` to use newer libc;
|
|
|
|
|
# apparently the interpreter needs to match libc, too.
|
|
|
|
|
bintools = self.stdenvNoCC.mkDerivation {
|
2023-01-14 10:07:44 +00:00
|
|
|
|
pname = prevStage.bintools.bintools.pname + "-patchelfed-ld";
|
|
|
|
|
inherit (prevStage.bintools.bintools) version;
|
2023-01-10 10:19:17 +00:00
|
|
|
|
passthru = { inherit (prevStage.bintools.passthru) isFromBootstrapFiles; };
|
2022-05-14 01:13:49 +00:00
|
|
|
|
enableParallelBuilding = true;
|
2021-06-30 17:47:13 +00:00
|
|
|
|
dontUnpack = true;
|
|
|
|
|
dontBuild = true;
|
2022-05-21 16:58:22 +00:00
|
|
|
|
strictDeps = true;
|
2021-06-30 17:47:13 +00:00
|
|
|
|
# We wouldn't need to *copy* all, but it's easier and the result is temporary anyway.
|
|
|
|
|
installPhase = ''
|
|
|
|
|
mkdir -p "$out"/bin
|
|
|
|
|
cp -a '${prevStage.bintools.bintools}'/bin/* "$out"/bin/
|
|
|
|
|
chmod +w "$out"/bin/ld.bfd
|
|
|
|
|
patchelf --set-interpreter '${getLibc self}'/lib/ld*.so.? \
|
|
|
|
|
--set-rpath "${getLibc self}/lib:$(patchelf --print-rpath "$out"/bin/ld.bfd)" \
|
|
|
|
|
"$out"/bin/ld.bfd
|
|
|
|
|
'';
|
|
|
|
|
};
|
2017-08-26 15:43:30 +00:00
|
|
|
|
};
|
2023-04-06 05:26:01 +00:00
|
|
|
|
|
|
|
|
|
# TODO(amjoseph): It is not yet entirely clear why this is necessary.
|
|
|
|
|
# Something strange is going on with xgcc and libstdc++ on pkgsMusl.
|
|
|
|
|
patchelf = super.patchelf.overrideAttrs(previousAttrs:
|
|
|
|
|
lib.optionalAttrs super.stdenv.hostPlatform.isMusl {
|
|
|
|
|
NIX_CFLAGS_COMPILE = (previousAttrs.NIX_CFLAGS_COMPILE or "") + " -static-libstdc++";
|
|
|
|
|
});
|
|
|
|
|
|
2011-12-14 14:31:56 +00:00
|
|
|
|
};
|
2021-03-31 15:56:38 +00:00
|
|
|
|
|
|
|
|
|
# `libtool` comes with obsolete config.sub/config.guess that don't recognize Risc-V.
|
|
|
|
|
extraNativeBuildInputs =
|
|
|
|
|
lib.optional (localSystem.isRiscV) prevStage.updateAutotoolsGnuConfigScriptsHook;
|
2016-12-24 15:38:56 +00:00
|
|
|
|
})
|
2010-08-25 13:22:48 +00:00
|
|
|
|
|
2012-12-28 15:41:56 +00:00
|
|
|
|
|
2014-08-23 19:26:37 +00:00
|
|
|
|
# Construct a third stdenv identical to the 2nd, except that this
|
|
|
|
|
# one uses the rebuilt Glibc from stage2. It still uses the recent
|
|
|
|
|
# binutils and rest of the bootstrap tools, including GCC.
|
2023-01-10 10:19:17 +00:00
|
|
|
|
(prevStage:
|
|
|
|
|
# previous stage2 stdenv:
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.binutils-unwrapped;
|
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.${localSystem.libc};
|
|
|
|
|
assert isBuiltByBootstrapFilesCompiler prevStage.gcc-unwrapped;
|
2023-01-10 10:19:17 +00:00
|
|
|
|
assert isFromBootstrapFiles prevStage.coreutils;
|
|
|
|
|
assert isFromBootstrapFiles prevStage.gnugrep;
|
2023-04-06 05:15:10 +00:00
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.patchelf;
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
assert lib.all isBuiltByNixpkgsCompiler (with prevStage; [ gmp isl_0_20 libmpc mpfr ]);
|
2023-01-10 10:19:17 +00:00
|
|
|
|
stageFun prevStage {
|
2018-02-01 00:00:00 +00:00
|
|
|
|
name = "bootstrap-stage3";
|
2014-09-10 23:18:14 +00:00
|
|
|
|
|
2016-12-19 16:10:47 +00:00
|
|
|
|
overrides = self: super: rec {
|
2016-12-24 15:07:20 +00:00
|
|
|
|
inherit (prevStage)
|
2016-12-24 15:28:40 +00:00
|
|
|
|
ccWrapperStdenv
|
2023-01-10 11:42:27 +00:00
|
|
|
|
binutils coreutils gnugrep gettext
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
perl patchelf linuxHeaders gnum4 bison libidn2 libunistring libxcrypt;
|
|
|
|
|
# We build a special copy of libgmp which doesn't use libstdc++, because
|
|
|
|
|
# xgcc++'s libstdc++ references the bootstrap-files (which is what
|
|
|
|
|
# compiles xgcc++).
|
|
|
|
|
gmp = super.gmp.override { cxx = false; };
|
|
|
|
|
} // {
|
2018-01-12 00:15:40 +00:00
|
|
|
|
${localSystem.libc} = getLibc prevStage;
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
gcc-unwrapped = (super.gcc-unwrapped.override (commonGccOverrides // {
|
|
|
|
|
inherit (prevStage) which;
|
|
|
|
|
}
|
|
|
|
|
)).overrideAttrs (a: {
|
|
|
|
|
# so we can add them to allowedRequisites below
|
|
|
|
|
passthru = a.passthru // { inherit (self) gmp mpfr libmpc isl; };
|
|
|
|
|
});
|
2010-01-19 11:25:33 +00:00
|
|
|
|
};
|
2018-12-22 11:49:41 +00:00
|
|
|
|
extraNativeBuildInputs = [ prevStage.patchelf ] ++
|
2016-03-05 00:28:23 +00:00
|
|
|
|
# Many tarballs come with obsolete config.sub/config.guess that don't recognize aarch64.
|
2018-09-11 16:47:09 +00:00
|
|
|
|
lib.optional (!localSystem.isx86 || localSystem.libc == "musl")
|
|
|
|
|
prevStage.updateAutotoolsGnuConfigScriptsHook;
|
2016-12-24 15:38:56 +00:00
|
|
|
|
})
|
2005-02-21 20:42:07 +00:00
|
|
|
|
|
2012-12-28 15:41:56 +00:00
|
|
|
|
|
2014-08-23 19:26:37 +00:00
|
|
|
|
# Construct a fourth stdenv that uses the new GCC. But coreutils is
|
|
|
|
|
# still from the bootstrap tools.
|
2022-12-31 10:34:10 +00:00
|
|
|
|
#
|
2023-01-10 10:19:17 +00:00
|
|
|
|
(prevStage:
|
|
|
|
|
# previous stage3 stdenv:
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.binutils-unwrapped;
|
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.${localSystem.libc};
|
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.gcc-unwrapped;
|
|
|
|
|
assert isFromBootstrapFiles prevStage.coreutils;
|
|
|
|
|
assert isFromBootstrapFiles prevStage.gnugrep;
|
2023-04-06 05:15:10 +00:00
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.patchelf;
|
2023-01-10 10:19:17 +00:00
|
|
|
|
stageFun prevStage {
|
2018-02-01 00:00:00 +00:00
|
|
|
|
name = "bootstrap-stage4";
|
2014-09-10 23:18:14 +00:00
|
|
|
|
|
2016-12-19 16:10:47 +00:00
|
|
|
|
overrides = self: super: {
|
2014-09-10 23:10:49 +00:00
|
|
|
|
# Zlib has to be inherited and not rebuilt in this stage,
|
|
|
|
|
# because gcc (since JAR support) already depends on zlib, and
|
|
|
|
|
# then if we already have a zlib we want to use that for the
|
|
|
|
|
# other purposes (binutils and top-level pkgs) too.
|
2022-04-20 00:00:36 +00:00
|
|
|
|
inherit (prevStage) gettext gnum4 bison perl texinfo zlib linuxHeaders libidn2 libunistring;
|
2018-01-12 00:15:40 +00:00
|
|
|
|
${localSystem.libc} = getLibc prevStage;
|
2017-08-26 15:43:30 +00:00
|
|
|
|
binutils = super.binutils.override {
|
|
|
|
|
# Don't use stdenv's shell but our own
|
|
|
|
|
shell = self.bash + "/bin/bash";
|
|
|
|
|
# Build expand-response-params with last stage like below
|
|
|
|
|
buildPackages = {
|
|
|
|
|
inherit (prevStage) stdenv;
|
|
|
|
|
};
|
|
|
|
|
};
|
|
|
|
|
|
2022-07-14 22:42:24 +00:00
|
|
|
|
# To allow users' overrides inhibit dependencies too heavy for
|
|
|
|
|
# bootstrap, like guile: https://github.com/NixOS/nixpkgs/issues/181188
|
|
|
|
|
gnumake = super.gnumake.override { inBootstrap = true; };
|
|
|
|
|
|
2015-01-09 19:22:12 +00:00
|
|
|
|
gcc = lib.makeOverridable (import ../../build-support/cc-wrapper) {
|
2014-09-10 23:18:14 +00:00
|
|
|
|
nativeTools = false;
|
|
|
|
|
nativeLibc = false;
|
2015-05-11 21:30:13 +00:00
|
|
|
|
isGNU = true;
|
2017-06-29 16:06:55 +00:00
|
|
|
|
buildPackages = {
|
|
|
|
|
inherit (prevStage) stdenv;
|
|
|
|
|
};
|
2016-12-24 15:07:20 +00:00
|
|
|
|
cc = prevStage.gcc-unwrapped;
|
2017-08-26 15:43:30 +00:00
|
|
|
|
bintools = self.binutils;
|
2018-01-12 00:15:40 +00:00
|
|
|
|
libc = getLibc self;
|
2021-01-24 03:02:59 +00:00
|
|
|
|
inherit lib;
|
2017-08-26 15:43:30 +00:00
|
|
|
|
inherit (self) stdenvNoCC coreutils gnugrep;
|
2016-12-19 16:10:47 +00:00
|
|
|
|
shell = self.bash + "/bin/bash";
|
2014-09-10 23:18:14 +00:00
|
|
|
|
};
|
2010-01-19 11:25:33 +00:00
|
|
|
|
};
|
2017-08-15 15:30:45 +00:00
|
|
|
|
extraNativeBuildInputs = [ prevStage.patchelf prevStage.xz ] ++
|
2016-03-05 00:28:23 +00:00
|
|
|
|
# Many tarballs come with obsolete config.sub/config.guess that don't recognize aarch64.
|
2018-09-11 16:47:09 +00:00
|
|
|
|
lib.optional (!localSystem.isx86 || localSystem.libc == "musl")
|
|
|
|
|
prevStage.updateAutotoolsGnuConfigScriptsHook;
|
2016-12-24 15:38:56 +00:00
|
|
|
|
})
|
2012-12-28 15:41:56 +00:00
|
|
|
|
|
2014-08-23 19:26:37 +00:00
|
|
|
|
# Construct the final stdenv. It uses the Glibc and GCC, and adds
|
|
|
|
|
# in a new binutils that doesn't depend on bootstrap-tools, as well
|
|
|
|
|
# as dynamically linked versions of all other tools.
|
2009-02-01 21:44:56 +00:00
|
|
|
|
#
|
2014-08-23 19:26:37 +00:00
|
|
|
|
# When updating stdenvLinux, make sure that the result has no
|
|
|
|
|
# dependency (`nix-store -qR') on bootstrapTools or the first
|
|
|
|
|
# binutils built.
|
2022-12-31 10:34:10 +00:00
|
|
|
|
#
|
2023-01-10 10:19:17 +00:00
|
|
|
|
(prevStage:
|
|
|
|
|
# previous stage4 stdenv; see stage3 comment regarding gcc,
|
|
|
|
|
# which applies here as well.
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.binutils-unwrapped;
|
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.${localSystem.libc};
|
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.gcc-unwrapped;
|
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.coreutils;
|
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.gnugrep;
|
2023-04-06 05:15:10 +00:00
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.patchelf;
|
2023-01-10 10:19:17 +00:00
|
|
|
|
{
|
2016-12-24 18:55:11 +00:00
|
|
|
|
inherit config overlays;
|
2016-12-24 15:38:56 +00:00
|
|
|
|
stdenv = import ../generic rec {
|
2018-02-01 00:00:00 +00:00
|
|
|
|
name = "stdenv-linux";
|
|
|
|
|
|
2017-07-06 01:47:48 +00:00
|
|
|
|
buildPlatform = localSystem;
|
|
|
|
|
hostPlatform = localSystem;
|
|
|
|
|
targetPlatform = localSystem;
|
2017-05-22 01:37:16 +00:00
|
|
|
|
inherit config;
|
2012-12-28 15:41:56 +00:00
|
|
|
|
|
2021-09-17 14:59:12 +00:00
|
|
|
|
preHook = commonPreHook;
|
2012-12-28 15:41:56 +00:00
|
|
|
|
|
2016-12-24 15:38:56 +00:00
|
|
|
|
initialPath =
|
2022-07-29 22:24:00 +00:00
|
|
|
|
((import ../generic/common-path.nix) {pkgs = prevStage;});
|
2005-02-21 20:42:07 +00:00
|
|
|
|
|
2018-12-22 11:49:41 +00:00
|
|
|
|
extraNativeBuildInputs = [ prevStage.patchelf ] ++
|
2016-03-05 00:28:23 +00:00
|
|
|
|
# Many tarballs come with obsolete config.sub/config.guess that don't recognize aarch64.
|
2018-09-11 16:47:09 +00:00
|
|
|
|
lib.optional (!localSystem.isx86 || localSystem.libc == "musl")
|
|
|
|
|
prevStage.updateAutotoolsGnuConfigScriptsHook;
|
2012-12-28 15:41:56 +00:00
|
|
|
|
|
2016-12-24 15:38:56 +00:00
|
|
|
|
cc = prevStage.gcc;
|
2012-12-28 15:41:56 +00:00
|
|
|
|
|
2016-12-24 15:38:56 +00:00
|
|
|
|
shell = cc.shell;
|
2010-08-06 10:34:34 +00:00
|
|
|
|
|
2016-12-24 15:38:56 +00:00
|
|
|
|
inherit (prevStage.stdenv) fetchurlBoot;
|
2014-08-29 20:09:01 +00:00
|
|
|
|
|
2016-12-24 15:38:56 +00:00
|
|
|
|
extraAttrs = {
|
2021-01-24 22:51:12 +00:00
|
|
|
|
inherit bootstrapTools;
|
2016-12-24 15:38:56 +00:00
|
|
|
|
shellPackage = prevStage.bash;
|
|
|
|
|
};
|
2015-01-03 14:54:02 +00:00
|
|
|
|
|
2022-04-14 06:13:33 +00:00
|
|
|
|
disallowedRequisites = [ bootstrapTools.out ];
|
|
|
|
|
|
2017-08-13 09:44:36 +00:00
|
|
|
|
# Mainly avoid reference to bootstrap tools
|
|
|
|
|
allowedRequisites = with prevStage; with lib;
|
|
|
|
|
# Simple executable tools
|
2018-09-04 22:03:51 +00:00
|
|
|
|
concatMap (p: [ (getBin p) (getLib p) ]) [
|
|
|
|
|
gzip bzip2 xz bash binutils.bintools coreutils diffutils findutils
|
2022-04-20 00:00:36 +00:00
|
|
|
|
gawk gmp gnumake gnused gnutar gnugrep gnupatch patchelf ed file
|
2017-08-13 09:44:36 +00:00
|
|
|
|
]
|
|
|
|
|
# Library dependencies
|
2017-08-24 08:57:50 +00:00
|
|
|
|
++ map getLib (
|
2019-08-18 18:58:56 +00:00
|
|
|
|
[ attr acl zlib pcre libidn2 libunistring ]
|
2017-08-24 08:57:50 +00:00
|
|
|
|
++ lib.optional (gawk.libsigsegv != null) gawk.libsigsegv
|
|
|
|
|
)
|
2017-08-13 09:44:36 +00:00
|
|
|
|
# More complicated cases
|
2018-01-12 00:15:40 +00:00
|
|
|
|
++ (map (x: getOutput x (getLibc prevStage)) [ "out" "dev" "bin" ] )
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
++ [ linuxHeaders # propagated from .dev
|
|
|
|
|
binutils gcc gcc.cc gcc.cc.lib gcc.expand-response-params gcc.cc.libgcc glibc.passthru.libgcc
|
2017-08-13 09:44:36 +00:00
|
|
|
|
]
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
++ lib.optionals (!localSystem.isx86 || localSystem.libc == "musl")
|
|
|
|
|
[ prevStage.updateAutotoolsGnuConfigScriptsHook prevStage.gnu-config ]
|
|
|
|
|
++ (with gcc-unwrapped.passthru; [
|
|
|
|
|
gmp libmpc mpfr isl
|
|
|
|
|
])
|
|
|
|
|
;
|
2016-12-24 15:38:56 +00:00
|
|
|
|
|
|
|
|
|
overrides = self: super: {
|
|
|
|
|
inherit (prevStage)
|
2017-02-11 23:15:12 +00:00
|
|
|
|
gzip bzip2 xz bash coreutils diffutils findutils gawk
|
2022-07-14 22:42:24 +00:00
|
|
|
|
gnused gnutar gnugrep gnupatch patchelf
|
2022-06-01 16:18:07 +00:00
|
|
|
|
attr acl zlib pcre libunistring;
|
2018-01-12 00:15:40 +00:00
|
|
|
|
${localSystem.libc} = getLibc prevStage;
|
2022-06-01 16:18:07 +00:00
|
|
|
|
|
|
|
|
|
# Hack: avoid libidn2.{bin,dev} referencing bootstrap tools. There's a logical cycle.
|
|
|
|
|
libidn2 = import ../../development/libraries/libidn2/no-bootstrap-reference.nix {
|
|
|
|
|
inherit lib;
|
|
|
|
|
inherit (prevStage) libidn2;
|
|
|
|
|
inherit (self) stdenv runCommandLocal patchelf libunistring;
|
|
|
|
|
};
|
|
|
|
|
|
2022-07-14 22:42:24 +00:00
|
|
|
|
gnumake = super.gnumake.override { inBootstrap = false; };
|
2018-08-20 18:43:41 +00:00
|
|
|
|
} // lib.optionalAttrs (super.stdenv.targetPlatform == localSystem) {
|
2017-02-11 23:15:12 +00:00
|
|
|
|
# Need to get rid of these when cross-compiling.
|
2018-04-03 16:50:25 +00:00
|
|
|
|
inherit (prevStage) binutils binutils-unwrapped;
|
2017-02-11 23:15:12 +00:00
|
|
|
|
gcc = cc;
|
2016-12-24 15:38:56 +00:00
|
|
|
|
};
|
2006-02-09 17:04:18 +00:00
|
|
|
|
};
|
2016-12-24 15:38:56 +00:00
|
|
|
|
})
|
|
|
|
|
|
2023-01-10 10:19:17 +00:00
|
|
|
|
# This "no-op" stage is just a place to put the assertions about stage5.
|
|
|
|
|
(prevStage:
|
|
|
|
|
# previous stage5 stdenv; see stage3 comment regarding gcc,
|
|
|
|
|
# which applies here as well.
|
stdenv: Nix-driven bootstrap of gcc
#### Summary
By default, when you type `make`, GCC will compile itself three
times. This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.
#### Immediate Benefits
- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
`bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
(i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
This allows Nix to build stageCompare in parallel with the rest of
nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
derivations
- No more Frankenstein compiler: the final gcc and the libraries it
links against (mpfr, mpc, isl, glibc) are all built by the same
compiler (xgcc) instead of a mixture of the bootstrapFiles'
compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.
#### Future Benefits
- This should allow using a [foreign] `bootstrap-files` so long as
`hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
(lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
built in separate (one-liner) derivations which `inherit src;`
from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343
#### Incorporates
- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995
#### Closes
- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412
#### Credits
This project was made possible by three important insights, none of
which were mine:
1. @ericson2314 was the first to advocate for this change, and
probably the first to appreciate its advantages. Nix-driven
(external) bootstrap is "cross by default".
2. @trofi has figured out a lot about how to get gcc to not mix up
the copy of `libstdc++` that it depends on with the copy that it
builds, by moving the `bootstrapFiles`' `libstdc++` into a
[versioned directory]. This allows a Nix-driven bootstrap of gcc
without the final gcc would still having references to the
`bootstrapFiles`.
3. Using the undocumented variable [`user-defined-trusted-dirs`]
when building glibc. When glibc `dlopen()`s `libgcc_s.so`, it
uses a completely different and totally special set of rules for
finding `libgcc_s.so`. This trick is the only way we can put
`libgcc_s.so` in its own separate outpath without creating
circular dependencies or dependencies on the bootstrapFiles. I
would never have guessed to use this (or that it existed!) if it
were not for a [comment in guix] which @Mic92 [mentioned].
My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].
[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: https://github.com/guix-mirror/guix/blob/5e4ec8218142eee8e6e148e787381a5ef891c5b1/gnu/packages/gcc.scm#L253
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: https://github.com/NixOS/nixpkgs/blob/2f1948af9c984ebb82dfd618e67dc949755823e2/pkgs/stdenv/linux/default.nix#L380
2023-02-24 05:32:36 +00:00
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.binutils-unwrapped;
|
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.${localSystem.libc};
|
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.gcc-unwrapped;
|
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.coreutils;
|
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.gnugrep;
|
2023-04-06 05:15:10 +00:00
|
|
|
|
assert isBuiltByNixpkgsCompiler prevStage.patchelf;
|
2023-01-10 10:19:17 +00:00
|
|
|
|
{ inherit (prevStage) config overlays stdenv; })
|
2016-12-24 15:38:56 +00:00
|
|
|
|
]
|