Commit Graph

406 Commits

Author SHA1 Message Date
Adam Joseph
c5a4cc8396 glibc: suppress warning about IEEE-standard long double 2023-04-05 15:54:26 -07:00
Vladimír Čunát
4eaca2b138
Merge #188492: glibc: 2.35-224 -> 2.37-8
...into staging
2023-04-03 20:29:07 +02:00
Adam Joseph
7553d0fe29 stdenv: Nix-driven bootstrap of gcc
#### Summary

By default, when you type `make`, GCC will compile itself three
times.  This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.

 #### Immediate Benefits

- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
  `bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
  (i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
  This allows Nix to build stageCompare in parallel with the rest of
  nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
  derivations
- No more Frankenstein compiler: the final gcc and the libraries it
  links against (mpfr, mpc, isl, glibc) are all built by the same
  compiler (xgcc) instead of a mixture of the bootstrapFiles'
  compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.

 #### Future Benefits

- This should allow using a [foreign] `bootstrap-files` so long as
  `hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
  (lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
  offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
  built in separate (one-liner) derivations which `inherit src;`
  from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343

 #### Incorporates

- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995

 #### Closes

- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412

 #### Credits

This project was made possible by three important insights, none of
which were mine:

1. @ericson2314 was the first to advocate for this change, and
   probably the first to appreciate its advantages.  Nix-driven
   (external) bootstrap is "cross by default".

2. @trofi has figured out a lot about how to get gcc to not mix up
   the copy of `libstdc++` that it depends on with the copy that it
   builds, by moving the `bootstrapFiles`' `libstdc++` into a
   [versioned directory].  This allows a Nix-driven bootstrap of gcc
   without the final gcc would still having references to the
   `bootstrapFiles`.

3. Using the undocumented variable [`user-defined-trusted-dirs`]
   when building glibc.  When glibc `dlopen()`s `libgcc_s.so`, it
   uses a completely different and totally special set of rules for
   finding `libgcc_s.so`.  This trick is the only way we can put
   `libgcc_s.so` in its own separate outpath without creating
   circular dependencies or dependencies on the bootstrapFiles.  I
   would never have guessed to use this (or that it existed!) if it
   were not for a [comment in guix] which @Mic92 [mentioned].

My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].

[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: 5e4ec82181/gnu/packages/gcc.scm (L253)
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: 2f1948af9c/pkgs/stdenv/linux/default.nix (L380)
2023-04-02 13:49:41 -07:00
Maximilian Bosch
047f379d38
glibc: use patch from ArchLinux to re-enable DT_HASH 2023-03-29 09:50:29 +02:00
Maximilian Bosch
b42ee8b817
glibc: 2.35-224 -> 2.37-8
Announcement: https://sourceware.org/pipermail/libc-alpha/2022-August/141193.html
Announcement: https://sourceware.org/pipermail/libc-alpha/2023-February/145190.html
2023-03-29 09:50:24 +02:00
Artturin
6b2a05e190 treewide: manual fixups for
treewide: use toString on list NIX_CFLAGS_COMPILE
treewide: move NIX_CFLAGS_COMPILE to the env attrset
2023-02-22 21:23:04 +02:00
Artturin
f9fdf2d402 treewide: move NIX_CFLAGS_COMPILE to the env attrset
with structuredAttrs lists will be bash arrays which cannot be exported
which will be a issue with some patches and some wrappers like cc-wrapper

this makes it clearer that NIX_CFLAGS_COMPILE must be a string as lists
in env cause a eval failure
2023-02-22 21:23:04 +02:00
Felix Buehler
cdb39a86e0 treewide: use optionalString 2023-02-13 21:52:34 +01:00
Sergei Trofimovich
6728277e19 libc: wipe out all references from copied libgcc_s.so.1
Without the change a copy from freshly built `gcc` still retains a
reference even after `patchelf --remove-rpath` because `patchelf` does
not cleanup dynamic sprintgs section.

The change stubs the reference out and fixes build on bootstraps where
`gcc` is built before `glibc`.
2023-01-14 18:23:56 +00:00
Sergei Trofimovich
76f5618e1e glibc: copy libgcc_s.so from .lib output if it exists
Otherwise copy it from the default output. The difference is visible
when we build `glibc` with:

- `bootstrapTools` `gcc`: ${stdenv.cc.cc.out}/lib/libgcc_s.so.1 is used
- `nixpkgs` `gcc`: ${stdenv.cc.cc.lib}/lib/libgcc_s.so.1 is used

Noticed when experimented with multiple `gcc` rebuilds in bootstrap.

While at it killing `RUNPATH` reference to bootstrap `glibc`.
2023-01-14 11:58:31 +00:00
github-actions[bot]
4712ed9439
Merge master into staging-next 2023-01-14 00:02:26 +00:00
Adam Joseph
97c335216d glibc: allow overriding common.nix 2023-01-13 19:16:30 +02:00
Vladimír Čunát
4835c7a74c
glibc, python3Packages.twisted: resolve temporary workaround
It certainly seems better to patch the twisted test than glibc.
2023-01-04 20:28:54 +01:00
Robert Hensing
16f5747575
Merge pull request #175649 from Artturin/opt-in-structured-attrs
stdenv: support opt-in __structuredAttrs
2022-12-10 21:12:43 +01:00
Artturin
c01f509e44 treewide: source .attrs in builders
if theres a source $stdenv then this is needed

for structuredAttrs
2022-12-08 21:09:02 +02:00
Artturin
adc8900df1 treewide: fix some core package structuredAttrs 2022-12-08 21:05:28 +02:00
Vladimír Čunát
5dffcba8fe
glibc: revert one patch from those added in parent commit
For now.  It causes issues in python3Packages.twisted testsuite,
so maybe it's buggy.  The tests might be buggy instead, but so far
we've had no report of the issues that are to be fixed by the patch.
https://github.com/NixOS/nixpkgs/pull/201805#issuecomment-1342735635
2022-12-08 14:34:53 +01:00
Vladimír Čunát
ec7567ba19
glibc: 2.35-163 -> 2.35-224 2022-12-06 15:51:01 +01:00
Dmitry Kalinkin
68e63f0ee3
Merge pull request #202827 from vcunat/p/glibc-kernel-versions
glibc: bump the minimum kernel version
2022-12-03 18:30:33 -05:00
Vladimír Čunát
68b44c86c2
glibc: bump the minimum kernel version
I don't think anyone sane is really using kernel < 3.10 nowadays.
(At least in a use case with glibc from nixpkgs.)

As another suspect with old kernels, Ubuntu seems to start at 3.13 now.
2022-11-25 11:38:48 +01:00
Sergei Trofimovich
ca95ee0977 glibc: backport make-4.4 fix
Not updating the whole snapshot as unrelated patches break some of
timezone-related tests:
    https://github.com/NixOS/nixpkgs/pull/201805#issuecomment-1320917345

Let's unblock `make-4.4` first.
2022-11-19 16:49:53 +00:00
Victor Fuentes
40ade45200
glib-locales: store SUPPORTED locales file 2022-11-02 13:59:53 -04:00
Jan Tojnar
457f28f6f8 Merge branch 'master' into staging-next
; Conflicts:
;	pkgs/development/tools/codespell/default.nix

codespell 2.2.2 switched to pyproject & setuptools_scm:
https://github.com/codespell-project/codespell/pull/2523
2022-10-19 05:24:28 +02:00
Zhaofeng Li
22b6046192 glibc: Don't inject CoreFoundation RUNPATH on Darwin
Ref: #137877
2022-10-18 07:24:18 -04:00
Martin Weinelt
ff30c899d8
glibc: make crypt support optional
The libcrypt library is going to be replaced with libxcrypt in packages
that require it.
2022-10-09 18:07:53 +02:00
Graham Christensen
c2b898da76 treewide: drop -l$NIX_BUILD_CORES
Passing `-l$NIX_BUILD_CORES` improperly limits the overall system load.

For a build machine which is configured to run `$B` builds where each
build gets `total cores / B` cores (`$C`), passing `-l $C` to make will
improperly limit the load to `$C` instead of `$B * $C`.

This effect becomes quite pronounced on machines with 80 cores, with
40 simultaneous builds and a cores limit of 2. On a machine with this
configuration, Nix will run 40 builds and make will limit the overall
system load to approximately 2. A build machine with this many cores
can happily run with a load approaching 80.

A non-solution is to oversubscribe the machine, by picking a larger
`$C`. However, there is no way to divide the number of cores in a way
which fairly subdivides the available cores when `$B` is greater than
1.

There has been exploration of passing a jobserver in to the sandbox,
or sharing a jobserver between all the builds. This is one option, but
relatively complicated and only supports make. Lots of other software
uses its own implementation of `-j` and doesn't support either `-l` or
the Make jobserver.

For the case of an interactive user machine, the user should limit
overall system load using `$B`, `$C`, and optionally systemd's
cpu/network/io limiting features.

Making this change should significantly improve the utilization of our
build farm, and improve the throughput of Hydra.
2022-09-22 16:01:23 -04:00
Minijackson
81c37edce4
glibcLocales: follow host platform endianness 2022-09-05 12:57:53 +02:00
github-actions[bot]
4ad33a5c7c
Merge master into staging-next 2022-08-18 06:01:24 +00:00
Sergei Trofimovich
198a940c61 glibc: add a few TODOs aroung libgcc_s.so hack
Make it clearer why the hack is still there.
2022-08-18 06:36:13 +01:00
Sergei Trofimovich
b3b672d5a1 setup-hooks/separate-debug-info.sh: don't inhibit strip hook
Before the change separate-debug-info.sh did the stripping itself.
This scheme has a few problems:
1. Stripping happens only on ELF files. *.a and *.o files are skipped.
   Derivations have to do it manually. Usually incorrectly
   as they don't run $RANLIB (true for `glibc` and `musl`).
2. Stripping happens on all paths. Ideally only `stripDebugList` paths
   should be considered.
3. Host strip is called on Target files.

This change offloads stripping logic to strip hook. This strips more
files for `glibc` and `musl`. Now we can remove most $STRIP calls
from individual derivations.

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2022-08-07 12:49:37 +01:00
Jörg Thalheim
5de6b3ecd7
Merge pull request #182281 from helsinki-systems/feat/glibc-sec
glibc: improve configure options
2022-07-27 22:09:44 +01:00
Bernardo Meurer
272fc53af1 glibc: 2.34-210 -> 2.35-163 2022-07-22 22:31:14 -07:00
ajs124
1487fabf60 glibc: enable Intel CET on x86 2022-07-21 00:32:03 +02:00
ajs124
8f3c8aee8a glibc: explicitly enable stack-protector
should be detected automatically
2022-07-21 00:32:03 +02:00
ajs124
34b92568d1 glibc: remove obsolete configure option
see https://sourceware.org/bugzilla/show_bug.cgi?id=27872
2022-07-21 00:32:03 +02:00
github-actions[bot]
1767ba1baa
Merge master into staging-next 2022-05-29 18:01:10 +00:00
Artturin
513b7f1010 glibc_multi: match output ordering of glibc
glibc has an exception in that 'out' is the default output instead of 'bin'

it should be matched here for consistency
2022-05-29 19:54:32 +03:00
sternenseemann
dd0b96be19
Merge pull request #173893 from trofi/fix-glibc-for-gnat6
glibc: apply pending PR29162 to unbreak gnat6
2022-05-23 20:17:48 +02:00
Sergei Trofimovich
3c211fb591 glibc: apply pending PR29162 to unbreak gnat6
commit e938c0274 "Don't add access size hints to fortifiable functions"
converted a few '__attr_access ((...))' into '__fortified_attr_access (...)'
calls.

But one of conversions had double parentheses of '__fortified_attr_access (...)'.

Noticed as a gnat6 build failure:

    /<<NIX>>-glibc-2.34-210-dev/include/bits/string_fortified.h:110:50: error: macro "__fortified_attr_access" requires 3 arguments, but only 1 given

The change fixes parentheses.
2022-05-23 18:40:49 +01:00
Artturin
0f9ee45a6d glibc: enable strictDeps
verified by building and diffing 'glibc' and 'glibcInfo'
2022-05-22 16:40:40 +03:00
Maximilian Bosch
5a3e803bf3
glibc: 2.34-115 -> 2.34-210 2022-05-06 10:28:53 +02:00
Sergei Trofimovich
bf990cc3cc glibc: unconditionally disable pie
glibc already has to be careful not to create extra dynamic relocations
in ld.so. For that it enables -fPIC/-fPIE selectively.
2022-04-18 16:06:50 +01:00
Anders Kaseorg
a20e363e6b
glibc: Fix segfault in getpwuid when stat fails (#167932)
Fixes #167785 using this patch submitted upstream:

https://sourceware.org/bugzilla/show_bug.cgi?id=28752
https://patchwork.sourceware.org/project/glibc/patch/20220314175316.3239120-2-sam@gentoo.org/

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2022-04-10 07:11:57 +02:00
Maximilian Bosch
e110983ed3
glibc: add empty libpthread.a
See https://github.com/NixOS/nixpkgs/pull/133431#discussion_r805338099
2022-02-27 14:06:23 +01:00
Maximilian Bosch
748faeb623
glibc: don't hide symbol __nss_files_fopen
I realized that `libredirect` doesn't wrap `__nss_files_fopen` anymore
which subsequently broke the `checkPhase` of `pkgs.openssh`[1] which was
implemented in 85c63a515d.

The reason for that is what `libc_hidden_{def,proto}` are used for, to
quote `libc-symbols.h`[2]:

    The following macros are used for PLT bypassing within libc.so
    (and if needed other libraries similarly).

This basically means that these symbols are altered in a way so that
they cannot be wrapped by `libredirect` even though it re-defines
`__nss_files_fopen` while being loaded at first with `LD_PRELOAD`.

The problem was caused by a change in 2.34[3] where the `files`-module
for NSS isn't loaded from `libnss_files.so.2` anymore, but the relevant
symbols are directly obtained from `libc.so.6` since a lot of glibc's
APIs were merged into `libc.so.6` before.

As this would effectively break `libredirect` whenever it tries to wrap
C APIs that interact with NSS (e.g. to use `getpwnam(3)`), I decided to
undo this change for `__nss_files_fopen`.

[1] https://hydra.nixos.org/build/160688523
[2] https://sourceware.org/git/?p=glibc.git;a=blob;f=include/libc-symbols.h;h=d41ecf4384d07f9108698dba1d4e9e6a7a9b975f;hb=ae37d06c7d127817ba43850f0f898b793d42aea7
[3] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=f9c8b11ed7726b858cd7b7cea0d3d7c5233d78cf;hp=6212bb67f4695962748a5981e1b9fea105af74f6
2022-02-27 10:26:53 +01:00
Maximilian Bosch
1b8aa881ea
glibc: revert /bin/bash usage
Rather than `$BASH`, `glibc` now hardcodes `/bin/bash` as
interpreter[1] in several scripts (including `ldd`).

This is a problem because relevant programs such as `ldd(1)` won't work
properly without this since we set `BASH` to `/bin/sh` to avoid
dependencies to the bootstrap tools for `bash` as runtime-dependency (since
NixOS doesn't have `/bin/bash`).

Considering that this was only done as an "improvement" to their
build-system and not because they wanted to use some bashisms here (the
variable was always called `BASH` and we still used `/bin/sh` anyways),
I'd consider this to be relatively safe.

[1] 5188a9d0265cc6f7235a8af1d31ab02e4a24853d
2022-02-27 10:25:35 +01:00
Maximilian Bosch
9fe34ccfbe
glibc: also create backwards-compat symlinks for libdl and libutil
This fixes at least the openssl build.
2022-02-27 10:25:34 +01:00
Maximilian Bosch
7bc32b3e5c
glibc: symlink libpthread.so -> libpthread.so.0 (same for -lrt) for backwards compatibility
Linking via `-lpthread` (or `-pthread`) is not needed anymore since
`glibc-2.34` since all the functionality is part of `libc.so.6` and
`libpthread.so.6` only exists for backwards-compatibility.

However, e.g. `gcc` (`libgomp` to be precise) expects a `libpthread.so`
to link against, otherwise the configure script will fail. As already
stated in the glibc release-notes itself, it is to expect that a lot
more applications will have issues with this, so I decided to re-add
`libpthread.so` as well.

For `librt.so.1`, the same thing is needed to make sure that Perl still
compiles:

    /nix/store/d6y5r7m93x14bmgn2p75fannz39jz66f-binutils-2.35.1/bin/ld: cannot find -lrt
    collect2: error: ld returned 1 exit status
    make[1]: *** [Makefile:490: ../../lib/auto/Time/HiRes/HiRes.so] Error 1
    make[1]: Leaving directory '/build/perl-5.34.0/dist/Time-HiRes'
2022-02-27 10:25:34 +01:00
Maximilian Bosch
97acaf6d65
glibc: 2.33-108 -> 2.34-115
Announcement: https://sourceware.org/pipermail/libc-alpha/2021-August/129718.html
2022-02-27 10:25:33 +01:00
TredwellGit
f02dc53800 glibc: 2.33-108 -> 2.33-117 2022-02-04 08:01:11 +00:00