Commit Graph

457 Commits

Author SHA1 Message Date
Robert Scott
df02fcb79b cc-wrapper: don't use fortify-headers for non-gcc compilers 2023-08-28 15:06:44 +01:00
github-actions[bot]
8a62479e06
Merge staging-next into staging 2023-08-20 00:03:24 +00:00
github-actions[bot]
f6a4c6f912
Merge master into staging-next 2023-08-20 00:02:29 +00:00
Robert Scott
658ab4b45b cc-wrapper: add fortify flags after invocation args, not before
this splits hardeningCFlags into hardeningCFlagsAfter and
hardeningCFlagsBefore (where most flags still remain) to allow
us to *append* `-D_FORTIFY_SOURCE=` values to the command-line,
forcing our choice of fortify level and avoiding potential
redefinition warnings/errors through use of `-U_FORTIFY_SOURCE`
2023-08-19 14:59:06 +01:00
Yang, Bo
1b8ca87a83
Merge branch 'master' into stdenv.cc.libcxx 2023-08-12 14:19:01 -07:00
Robert Scott
95c4a1fe96 cc-wrapper: include fortify-headers before libc includes for musl 2023-08-06 17:52:28 +01:00
Weijia Wang
7c1239a54a
Merge pull request #245282 from reckenrode/gccStdenv-fix
cc-wrapper: use a temporary file for reponse file
2023-08-02 11:04:25 +02:00
Randy Eckenrode
6f2b3ba027 cc-wrapper: use a temporary file for reponse file
The Darwin stdenv rework conditionally sets `NIX_CC_USE_RESPONSE_FILE`
depending on the `ARG_MAX` of the build system. If it is at least 1 MiB,
the stdenv passes the arguments on the command-line (like Linux).
Otherwise, it falls back to the response file. This was done to prevent
intermitent failures with clang 16 being unable to read the response
file. Unfortunately, this breaks `gccStdenv` on older Darwin platforms.

Note: While the stdenv logic will also be reverted, this change is
needed for compatibility with clang 16.

GCC is capable of using a response file, but it does not work correctly
when the response file is a file descriptor. This can be reproduced
using the following sequence of commands:

    $ nix shell nixpkgs#gcc; NIX_CC_USE_RESPONSE_FILE=1 gcc
    # Linux
    /nix/store/9n9gjvzci75gp2sh1c4rh626dhizqynl-binutils-2.39/bin/ld: unrecognized option '-B/nix/store/vnwdak3n1w2jjil119j65k8mw1z23p84-glibc-2.35-224/lib/'
    /nix/store/9n9gjvzci75gp2sh1c4rh626dhizqynl-binutils-2.39/bin/ld: use the --help option for usage information
    collect2: error: ld returned 1 exit status
    # Darwin
    ld: unknown option: -mmacosx-version-min=11.0
    collect2: error: ld returned 1 exit status

Instead of using process substitution, create a temporary file and
remove it in a trap. This should also prevent the intermitent build
failures with clang 16 on older Darwin systems.

Fixes #245167
2023-08-01 10:32:56 +02:00
Vladimír Čunát
88dec0c7a9
Merge #243595: cc-wrapper: -fwrapv instead of -fno-strict-overflow in clang
..into staging
2023-07-26 11:55:59 +02:00
Felix Bühler
0a2745684e
Merge pull request #239624 from Stunkymonkey/use-optionalString-then
treewide: use optionalString instead of 'then ""'
2023-07-22 13:02:47 +02:00
Theodore Ni
acb182363b
cc-wrapper: use -fwrapv instead of -fno-strict-overflow in clang 2023-07-17 23:41:33 -07:00
Vladimír Čunát
58392652f0
Merge #224822: hardening flags: enable fortify3 by default
...into staging
2023-07-06 10:33:24 +02:00
Artturi
8bf1b878cf
Merge pull request #239331 from pwaller/fix-pie-hardening 2023-07-06 02:14:35 +03:00
Artturi
359e1136a6
Merge pull request #239120 from LibreCybernetics/arch-stuff 2023-07-05 00:20:25 +03:00
Felix Buehler
6672dde558 treewide: use optionalAttrs instead of 'else {}' 2023-06-25 11:01:34 -03:00
Robert Scott
db3e94c3b2 hardening flags: enable fortify3 by default 2023-06-25 11:50:46 +01:00
Felix Buehler
f3719756b5 treewide: use optionalString instead of 'then ""' 2023-06-24 20:19:19 +02:00
Fabián Heredia Montiel
79dfc50bb8 lib.systems.architectures: add microarchitecture levels
Variation on:
- https://github.com/NixOS/nixpkgs/pull/208398
- https://github.com/NixOS/nixpkgs/pull/224978

Co-authored-by: Sandro Jäckel <sandro.jaeckel@gmail.com>
Co-authored-by: Shawn8901 <shawn8901@googlemail.com>
Co-authored-by: AveryanAlex <alex@averyan.ru>
2023-06-24 00:50:40 -06:00
Peter Waller
e0d2053b87 build-support: Use response-expanded params in pie test
When a response file is in use, "$*" contains the response file and not
the parameters; both the linker and compiler wrappers are updated to use
the response-expanded params.

The compiler driver likes to pass parameters to the linker via a
response file, including -shared.

LLD rejects the combination of (-shared -pie), whereas other linkers
silently ignore the contradiction:

```
ld.lld: error: -shared and -pie may not be used together
```

This breaks certain configurations using LLD as a linker.

Changing `add-hardening.sh` results in a full rebuild. To avoid the
rebuild, here is a quick test case which shows the new hardening script
allows the link to succeed:

```
{ pkgs ? import <nixpkgs> {} }:

let
  # gcc silently accepts -shared -pie together, lld does not.
  linker = pkgs.wrapBintoolsWith { bintools = pkgs.llvmPackages.lld; };

  patchWrapper = prev: prev.overrideAttrs (final: prev: let
    prevScript = builtins.match (".*(/nix/store/[a-z0-9]+-add-hardening.sh).*") prev.postFixup;
  in {
    postFixup = (builtins.replaceStrings prevScript ["${./new-add-hardening.sh}"] prev.postFixup);
  });
in

pkgs.stdenv.mkDerivation {
  name = "nixpkgs-hardening-bug";

  src = pkgs.writeText "src.c" "int main(int argc, char* argv[]) { return 0; }";
  NIX_HARDENING_ENABLE = "pie";

  unpackPhase = ":";
  buildPhase = ''
    $CC -c -o src.o $src
    bash -x ${patchWrapper linker}/bin/ld.lld -o $out @${pkgs.writeText "responsefile" "-shared"} src.o
  '';
}
```

Fixes: #178162
Signed-off-by: Peter Waller <p@pwaller.net>
2023-06-23 11:09:49 +01:00
Sandro
9a670fec3b
Merge pull request #237167 from CHN-beta/master 2023-06-19 14:14:03 +02:00
Fabián Heredia Montiel
1b7776a3fb lib.systems: add znver4 architecture 2023-06-16 13:47:10 -06:00
chn
a41e973062 stdenv: add alderlake support
Signed-off-by: Haonan Chen <chn@chn.moe>
2023-06-11 21:11:03 +08:00
Vladimír Čunát
944c7fa720
Merge #235610: cc-wrapper: try to better guess meta.mainProgram 2023-06-11 09:11:13 +02:00
Jack Leightcap
4c2970da7e
gcj: fix compiler
Signed-off-by: Jack Leightcap <jack@leightcap.com>
2023-06-07 01:42:02 -04:00
Vladimír Čunát
295ff35f24
cc-wrapper: try to better guess meta.mainProgram
Otherwise nix will guess it from (p)name which contains "-wrapper".
Fixes #235585
2023-06-02 17:32:06 +02:00
figsoda
98b9e41f61 pkgs: fix typos 2023-05-19 22:31:04 -04:00
github-actions[bot]
b5b52a946a
Merge staging-next into staging 2023-05-09 18:02:32 +00:00
Adam Joseph
0e9ef0a07d cc-wrapper: when merging gcc32 and gcc64, merge libgcc as well
Our gcc_multi and glibc_multi expressions merge together a
32-bit-targeted and 64-bit-targeted gcc.  However they do not thread
through the passthru.libgcc from these merged gccs.

This commit corrects that.

It also extends passthru.libgcc to allow a *list* rather than just a
single outpath.

Resolves part of #221891 (at least getting it back to the error
message it gave before).
2023-05-09 00:16:24 -07:00
Rahul Butani
53b268ad4a
cc-wrapper: support --
Fixes #228136.
2023-05-03 16:42:15 -05:00
Alyssa Ross
bfc7aaa8af wrapCCWith: disable pic when building for Windows
According to <https://gcc.gnu.org/legacy-ml/gcc-patches/2015-08/msg00836.html>,
all code is position-independent on Windows.  Some compilers
apparently warn for -fPIC on Windows, and clang errors:

> clang-15: error: unsupported option '-fPIC' for target 'x86_64-pc-windows-msvc'

I'm guessing the check was hostPlatform instead of targetPlatform by mistake.
2023-04-28 10:01:22 +00:00
Vladimír Čunát
f2186222c6
Merge #225846: cc-wrapper: deunify clang/gcc handling of -B
...into staging
2023-04-16 09:59:54 +02:00
github-actions[bot]
a6e62de641
Merge staging-next into staging 2023-04-15 12:02:10 +00:00
Kira Bruneau
99a95083df
Merge pull request #178280 from veprbl/pr/ccache_clang_fix
cc-wrapper: disable response files for ccache
2023-04-15 06:47:01 -04:00
github-actions[bot]
6176f16de2
Merge staging-next into staging 2023-04-14 12:02:03 +00:00
Sandro
b04d4bad27
Merge pull request #216992 from SuperSandro2000/stdenvNative-fix-eval
{bintools,cc}-wrapper: don't fallback to version = null
2023-04-14 11:22:20 +02:00
Adam Joseph
c1e956e0a9 cc-wrapper: deunify clang/gcc handling of -B flag
Closes #225779
Closes #225780
2023-04-13 22:57:09 -07:00
Sandro Jäckel
7090651071
{bintools,cc}-wrapper: don't fallback to version = null
mkDerivation cannot handle that
2023-04-12 22:08:36 +02:00
Sandro Jäckel
a7dbdb7644
cc-wrapper: don't set env to null when nativeTools is used
This is not allowed and fails fatal
2023-04-12 22:08:36 +02:00
Adam Joseph
15e2a735f8 Revert "cc-wrapper: add optional temporary hack for -B"
This reverts commit ac3acd956f.
2023-04-12 10:26:23 -07:00
github-actions[bot]
f4a0b6d5fa
Merge staging-next into staging 2023-04-12 12:02:59 +00:00
Vladimír Čunát
ac3acd956f
cc-wrapper: add optional temporary hack for -B
This fixes parts in llvmPackages_{13,rocm}
e.g. build .clang for testing.
Longterm mass-rebuild fix should come in PR #225846
2023-04-12 09:37:24 +02:00
Adam Joseph
de8ce81ff2 cc-wrapper: deunify clang/gcc treatment of -isystem
In https://github.com/NixOS/nixpkgs/pull/209870 I tried to unify the
treatment of clang and gcc in cc-wrapper as much as possible.
However it appears that I went too far.

Clang requires -isystem flags in order to be able to find gcc's
libstdc++.  Gcc does not need these flags.  If they are added,
gfortran will get confused:

  https://github.com/NixOS/nixpkgs/pull/209870#issuecomment-1500550903

This commit deunifies the chunk of code that adds the -isystem
flags, and explains why this chunk applies only to clang.
2023-04-11 20:19:58 +03:00
Artturin
b1d4dfddaf Revert "julia{18,19,}: fix build by a temporary hack"
This reverts commit e2691227cd.
2023-04-11 20:19:58 +03:00
Vladimír Čunát
e2691227cd
julia{18,19,}: fix build by a temporary hack
This is a low-rebuild version of PR #225273
/cc the proper and hopefully complete fix in PR #225220
2023-04-10 16:36:55 +02:00
Vladimír Čunát
834334f2c2
Merge #169327: cc-wrapper: fix minor problems
...into staging
2023-04-04 09:25:59 +02:00
Bernardo Meurer
f1f6ca8bcd
Merge pull request #209870 from amjoseph-nixpkgs/pr/stdenv/external-gcc-bootstrap 2023-04-03 08:19:03 -07:00
Adam Joseph
7553d0fe29 stdenv: Nix-driven bootstrap of gcc
#### Summary

By default, when you type `make`, GCC will compile itself three
times.  This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.

 #### Immediate Benefits

- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
  `bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
  (i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
  This allows Nix to build stageCompare in parallel with the rest of
  nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
  derivations
- No more Frankenstein compiler: the final gcc and the libraries it
  links against (mpfr, mpc, isl, glibc) are all built by the same
  compiler (xgcc) instead of a mixture of the bootstrapFiles'
  compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.

 #### Future Benefits

- This should allow using a [foreign] `bootstrap-files` so long as
  `hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
  (lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
  offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
  built in separate (one-liner) derivations which `inherit src;`
  from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343

 #### Incorporates

- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995

 #### Closes

- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412

 #### Credits

This project was made possible by three important insights, none of
which were mine:

1. @ericson2314 was the first to advocate for this change, and
   probably the first to appreciate its advantages.  Nix-driven
   (external) bootstrap is "cross by default".

2. @trofi has figured out a lot about how to get gcc to not mix up
   the copy of `libstdc++` that it depends on with the copy that it
   builds, by moving the `bootstrapFiles`' `libstdc++` into a
   [versioned directory].  This allows a Nix-driven bootstrap of gcc
   without the final gcc would still having references to the
   `bootstrapFiles`.

3. Using the undocumented variable [`user-defined-trusted-dirs`]
   when building glibc.  When glibc `dlopen()`s `libgcc_s.so`, it
   uses a completely different and totally special set of rules for
   finding `libgcc_s.so`.  This trick is the only way we can put
   `libgcc_s.so` in its own separate outpath without creating
   circular dependencies or dependencies on the bootstrapFiles.  I
   would never have guessed to use this (or that it existed!) if it
   were not for a [comment in guix] which @Mic92 [mentioned].

My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].

[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: 5e4ec82181/gnu/packages/gcc.scm (L253)
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: 2f1948af9c/pkgs/stdenv/linux/default.nix (L380)
2023-04-02 13:49:41 -07:00
Adam Joseph
39c7885cd9 cc-wrapper: if isClang, add -L${gccForLibs.libgcc}/lib
When wrapping `clang` and using a `gccForLibs` whose `libgcc` is in
its own output (rather than the `lib` output), this commit will adds
`-L${gccForLibs.libgcc}/lib` to `cc-ldflags`.

If that flag is not added, `firefox` will fail to compile because it
invokes `clang-wrapper` with `-fuse-ld=lld` and passes `-lgcc_s` to
`lld`, but does not tell `lld` where to find `libgcc_s.so`.  In that
situation, firefox will fail to link.
2023-04-02 03:04:20 -07:00
Sergei Trofimovich
0ebe51030f cc-wrapper: make $tool-prefixed cpp wrapping unconditional 2023-03-17 18:04:54 +00:00
Sergei Trofimovich
4bdbae5ad1 cc-wrapper: wrap cpp for cross lust like to native
Without this change $target-cpp is used unwrapped and is missing
standard header search paths among other things).

Example failure:

    $ nix build -f. -L pkgsStatic.netbsd.compat
    ...
       > checking how to run the C preprocessor... x86_64-unknown-linux-musl-cpp
       > configure: error: in `/build/cvs-export/tools/compat':
       > configure: error: C preprocessor "x86_64-unknown-linux-musl-cpp" fails sanity check
       > See `config.log' for more details
2023-03-17 16:42:52 +00:00