this allows a compiler derivation to provide a
hardeningUnsupportedFlagsByTargetPlatform passthru attr
that will be called with the targetPlatform to determine
the unsupported hardening flags for that platform.
we can do this because even though a clang compiler is
multi-target by nature, cc-wrapper effectively fixes the
target platform at wrapping time. otherwise we'd have to
sniff the intended target at runtime, which wouldn't
be fun at all.
the advantage of using a new attribute instead of
allowing hardeningUnsupportedFlags to optionally be a
function is that hardeningUnsupportedFlags retains its
simple overriding pattern for simple cases (i.e.
`(prev.hardeningUnsupportedFlags or []) ++ [ "foo" ]`
) which will continue to work as long as the bottom-most
function of hardeningUnsupportedFlagsByTargetPlatform
falls back to hardeningUnsupportedFlags.
PR #275947, which was self-merged without approvals, inserted
functionality specific to a propriteary closed-source compiler
(CUDA) into cc-wrapper.
This commit relocates this CUDA-specific functionality into the
appropritate place: `cuda-modules`.
It is unclear to me exactly what this function is supposed to be
doing; much of it (like the `.kind` attributes) do not appear to be
used *anywhere* in nixpkgs. Making sure we don't insert unexplained
deadcode like this is one of the important functions of the review
process.
This commit deletes speculative comments which were self-merged with
no approvals in PR #275947.
If you think that "The above 'fix' may be incorrect" the correct
response is to submit a PR which removes the 'fix' and get it reviewed.
Likewise, if you think that "For clang it's not necessary" you
should submit a PR which wraps it in `if !isClang`.
`cc-wrapper` is full of too much junk as it is, let's not make
things worse.
The commit prior to this one, "gcc: fix c++ headers when same
triplet cross compiling" causes gcc's c++ headers to be in the same
outpath subdirectory regardless of whether the gcc build is a
host==target or host!=target compiler.
As a result of that change, the hack in cc-wrapper which adapted to
the different paths is no longer needed. And, in fact, it must be
removed, since if it is left in place builds such as
pkgsCross.aarch64-multiplatform.firefox will fail as shown below.
```
firefox-unwrapped> 0:02.01(B checking the host C compiler works... yes(B(B
firefox-unwrapped> 0:02.01(B checking for the host C++ compiler... /nix/store/1asqji9djmdlapzs70q7jw2j308ry7cn-clang-wrapper-16.0.6/bin/c++(B(B
firefox-unwrapped> 0:02.14(B checking whether the host C++ compiler can be used... yes(B(B
firefox-unwrapped> 0:02.14(B checking the host C++ compiler version... 16.0.6(B(B
firefox-unwrapped> 0:02.21(B checking the host C++ compiler works... yes(B(B
firefox-unwrapped> 0:02.40(B checking for target linker... lld(B(B
firefox-unwrapped> 0:02.51(B checking for host linker... lld(B(B
firefox-unwrapped> 0:02.60(B checking for 64-bit OS... yes(B(B
firefox-unwrapped> 0:02.67(B checking for new enough STL headers from libstdc++...(B(B
firefox-unwrapped> 0:02.67(B DEBUG: <truncated - see config.log for full output>(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | #if defined(__GLIBCXX__) && !defined(_GLIBCXX_RELEASE)(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | # error libstdc++ not new enough(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | #endif(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | #if defined(_GLIBCXX_RELEASE)(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | # if _GLIBCXX_RELEASE < 8(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | # error libstdc++ not new enough(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | # else(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | (void) 0(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | # endif(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | #endif(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | ;(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | return 0;(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | }(B(B
firefox-unwrapped> 0:02.67(B DEBUG: Executing: `/nix/store/7v4bi4q334yircaznwm353h1l5i7k98f-aarch64-unknown-linux-gnu-clang-wrapper-16.0.6/bin/aarch64-unknown-linux-gnu-clang++ /build/conftest.qvbo58dv.cpp -c`(B(B
firefox-unwrapped> 0:02.67(B DEBUG: The command returned non-zero exit status 1.(B(B
firefox-unwrapped> 0:02.67(B DEBUG: Its error output was:(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | /build/conftest.qvbo58dv.cpp:1:10: fatal error: 'cstddef' file not found(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | #include <cstddef>(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | ^~~~~~~~~(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | 1 error generated.(B(B
firefox-unwrapped> 0:02.67(B ERROR: The libstdc++ in use is not new enough. Please run ./mach bootstrap to update your compiler, or update your system libstdc++ installation.(B(B
firefox-unwrapped> *** Fix above errors and then restart with "./mach build"
error: build of '/nix/store/5in7xkji5hzqkl14ygwq3vxnni54lykk-firefox-unwrapped-aarch64-unknown-linux-gnu-119.0.1.drv' on 'ssh://root@192.168.22.103' failed: builder for '/nix/store/5in7xkji5hzqkl14ygwq3vxnni54lykk-firefox-unwrapped-aarch64-unknown-linux-gnu-119.0.1.drv' failed with exit code 1
error: builder for '/nix/store/5in7xkji5hzqkl14ygwq3vxnni54lykk-firefox-unwrapped-aarch64-unknown-linux-gnu-119.0.1.drv' failed with exit code 1;
```
this makes it a lot easier to create a modified stdenv with a
different set of defaultHardeningFlags and as a bonus allows us
to inject the correct defaultHardeningFlags into toolchain wrapper
scripts, reducing repetition.
while most hardening flags are arguably more of a compiler thing,
it works better to put them in bintools-wrapper because cc-wrapper
can easily refer to bintools but not vice-versa.
mkDerivation can still easily refer to either when it is constructed.
this also switches fortran-hook.sh to use the same defaults for
NIX_HARDENING_ENABLE as for C. previously NIX_HARDENING_ENABLE
defaults were apparently used to avoid passing problematic flags
to a fortran compiler, but this falls apart as soon as mkDerivation
sets its own NIX_HARDENING_ENABLE - cc.hardeningUnsupportedFlags
is a more appropriate mechanism for this as it actively filters
out flags from being used by the wrapper, so switch to using that
instead.
this is still an imperfect mechanism because it doesn't handle a
compiler which has both langFortran *and* langC very well - applying
the superset of the two's hardeningUnsupportedFlags to either
compiler's invocation. however this is nothing new - cc-wrapper
already poorly handles a langFortran+langC compiler, applying two
setup hooks that have contradictory options.
-B must be set to the root directory of avrlibc, otherwise gcc cannot
locate crt objects for some attiny devices. -L trains as set by
bintools-wrapper are not necessary with -B set correctly because gcc
takes care of that, and likewise we can drop the -B train from
cc-wrapper because the one spec is enough.
Before this commit, cc-wrapper/default.nix was using
`isGccArchSupported` to validate `-mtune=` values. This has two
problems:
- On x86, `-mtune=` can take the same values as `-march`, plus two
additional values `generic` and `intel` which are not valid for
`-march`.
- On ARM, `-mtune=` does not take the same values as `-march=`;
instead it takes the same values as `-mcpu`.
This commit fixes these two problems by adding a new
`isGccTuneSupported` function. For `isx86` this returns `true` for
the two special values and otherwise defers to `isGccArchSupported`.
This commit also adds support for `-mtune=` on Aarch64.
Unfortunately on Aarch64, Clang does not accept as wide a variety of
`-mtune=` values as Gcc does. In particular, Clang does not tune
for big.LITTLE mixed-model chips like the very popular RK3399, which
is targeted using `-march=cortex-a72.cortex-a53` in gcc.
To address this problem, this commit also adds a function
`findBestTuneApproximation` which can be used to map
clang-unsupported tunings like `cortex-a72.cortex-a53` to
less-precise tunings like `cortex-a53`.
The work which led to this commit arose because we now have
packages, like `crosvm`, which use *both* `clang` *and* `gcc`.
Previously I had been using `overrideAttrs` to set
`NIX_CFLAGS_COMPILE` on a package-by-package basis based on which
compiler that package used. Since we now have packages which use
*both* compilers, this strategy no longer works.
I briefly considered splitting `NIX_CFLAGS_COMPILE` into
`NIX_CFLAGS_COMPILE_GCC` and `NIX_CFLAGS_COMPILE_CLANG`, but since
`NIX_CFLAGS_COMPILE` is sort of a hack to begin with I figured that
adding the logic to `cc-wrapper` would be preferable.
The commit 6f2b3ba027 introduced a
`mktemp` invokation that uses the `--tmpdir` flag, which is not
available on MacOS.
This changes the invokation to a portable one based on the following
StackOverflow answer https://stackoverflow.com/a/31397073/841562 .
Removed workaround from llvm 16.
Fixes including cxxabi.h on llvm >=15 libcxxStdenv.
```c
int main() {}
```
```
/nix/store/qwnvng0cbyx0bijm654jpmpl0516hfhx-libcxxabi-15.0.7-dev/include/cxxabi.h:20:10: fatal error: '__cxxabi_config.h' file not found
```
Before llvm 15 this used to work because `libcxx` copied the headers
from `cxxabi` to it's own `include`, which was then picked up by the
line above this one
Alternative fix would be to copy all files from `${cxxabi.dev}/include/c++/v1` to `${cxxabi.dev}/include` so the cc-wrapper setup hook would pick them up, but that would depend on in cxxabi being in buildInputs.
Signed-off-by: Amneesh Singh <natto@weirdnatto.in>
Without this, pkgsStatic.pkgsLLVM.hello fails with segfaulting binaries
because of the issue described at [0].
In summary, llvm's linker has a different behaviour to GCC's when
supplied with both -static and -Wl,-dynamic-linker=...; GCC copes with
it, but LLVM produces a binary which segfaults on startup. It appears to
be necessary to omit the dynamic linker in this case.
nixpkgs' static adaptor passes -static via NIX_CFLAGS_LINK which was not
accounted for prior to this commit in the checkLinkType logic. For good
measure I put the other NIX_ flags affecting link in the same logic.
Additionally, $NIX_CFLAGS_LINK_@suffixSalt@ is not available until later
than it was originally set, so set $linkType close to its point of use.
I checked for earlier uses by studying the shell trace output and
couldn't find any.
[0] https://github.com/NixOS/nixpkgs/issues/111010#issuecomment-1536424163
Signed-off-by: Peter Waller <p@pwaller.net>
this splits hardeningCFlags into hardeningCFlagsAfter and
hardeningCFlagsBefore (where most flags still remain) to allow
us to *append* `-D_FORTIFY_SOURCE=` values to the command-line,
forcing our choice of fortify level and avoiding potential
redefinition warnings/errors through use of `-U_FORTIFY_SOURCE`
The Darwin stdenv rework conditionally sets `NIX_CC_USE_RESPONSE_FILE`
depending on the `ARG_MAX` of the build system. If it is at least 1 MiB,
the stdenv passes the arguments on the command-line (like Linux).
Otherwise, it falls back to the response file. This was done to prevent
intermitent failures with clang 16 being unable to read the response
file. Unfortunately, this breaks `gccStdenv` on older Darwin platforms.
Note: While the stdenv logic will also be reverted, this change is
needed for compatibility with clang 16.
GCC is capable of using a response file, but it does not work correctly
when the response file is a file descriptor. This can be reproduced
using the following sequence of commands:
$ nix shell nixpkgs#gcc; NIX_CC_USE_RESPONSE_FILE=1 gcc
# Linux
/nix/store/9n9gjvzci75gp2sh1c4rh626dhizqynl-binutils-2.39/bin/ld: unrecognized option '-B/nix/store/vnwdak3n1w2jjil119j65k8mw1z23p84-glibc-2.35-224/lib/'
/nix/store/9n9gjvzci75gp2sh1c4rh626dhizqynl-binutils-2.39/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
# Darwin
ld: unknown option: -mmacosx-version-min=11.0
collect2: error: ld returned 1 exit status
Instead of using process substitution, create a temporary file and
remove it in a trap. This should also prevent the intermitent build
failures with clang 16 on older Darwin systems.
Fixes#245167
When a response file is in use, "$*" contains the response file and not
the parameters; both the linker and compiler wrappers are updated to use
the response-expanded params.
The compiler driver likes to pass parameters to the linker via a
response file, including -shared.
LLD rejects the combination of (-shared -pie), whereas other linkers
silently ignore the contradiction:
```
ld.lld: error: -shared and -pie may not be used together
```
This breaks certain configurations using LLD as a linker.
Changing `add-hardening.sh` results in a full rebuild. To avoid the
rebuild, here is a quick test case which shows the new hardening script
allows the link to succeed:
```
{ pkgs ? import <nixpkgs> {} }:
let
# gcc silently accepts -shared -pie together, lld does not.
linker = pkgs.wrapBintoolsWith { bintools = pkgs.llvmPackages.lld; };
patchWrapper = prev: prev.overrideAttrs (final: prev: let
prevScript = builtins.match (".*(/nix/store/[a-z0-9]+-add-hardening.sh).*") prev.postFixup;
in {
postFixup = (builtins.replaceStrings prevScript ["${./new-add-hardening.sh}"] prev.postFixup);
});
in
pkgs.stdenv.mkDerivation {
name = "nixpkgs-hardening-bug";
src = pkgs.writeText "src.c" "int main(int argc, char* argv[]) { return 0; }";
NIX_HARDENING_ENABLE = "pie";
unpackPhase = ":";
buildPhase = ''
$CC -c -o src.o $src
bash -x ${patchWrapper linker}/bin/ld.lld -o $out @${pkgs.writeText "responsefile" "-shared"} src.o
'';
}
```
Fixes: #178162
Signed-off-by: Peter Waller <p@pwaller.net>
Our gcc_multi and glibc_multi expressions merge together a
32-bit-targeted and 64-bit-targeted gcc. However they do not thread
through the passthru.libgcc from these merged gccs.
This commit corrects that.
It also extends passthru.libgcc to allow a *list* rather than just a
single outpath.
Resolves part of #221891 (at least getting it back to the error
message it gave before).
