PR #275947, which was self-merged without approvals, inserted
functionality specific to a propriteary closed-source compiler
(CUDA) into cc-wrapper.
This commit relocates this CUDA-specific functionality into the
appropritate place: `cuda-modules`.
It is unclear to me exactly what this function is supposed to be
doing; much of it (like the `.kind` attributes) do not appear to be
used *anywhere* in nixpkgs. Making sure we don't insert unexplained
deadcode like this is one of the important functions of the review
process.
This commit deletes speculative comments which were self-merged with
no approvals in PR #275947.
If you think that "The above 'fix' may be incorrect" the correct
response is to submit a PR which removes the 'fix' and get it reviewed.
Likewise, if you think that "For clang it's not necessary" you
should submit a PR which wraps it in `if !isClang`.
`cc-wrapper` is full of too much junk as it is, let's not make
things worse.
The commit prior to this one, "gcc: fix c++ headers when same
triplet cross compiling" causes gcc's c++ headers to be in the same
outpath subdirectory regardless of whether the gcc build is a
host==target or host!=target compiler.
As a result of that change, the hack in cc-wrapper which adapted to
the different paths is no longer needed. And, in fact, it must be
removed, since if it is left in place builds such as
pkgsCross.aarch64-multiplatform.firefox will fail as shown below.
```
firefox-unwrapped> 0:02.01(B checking the host C compiler works... yes(B(B
firefox-unwrapped> 0:02.01(B checking for the host C++ compiler... /nix/store/1asqji9djmdlapzs70q7jw2j308ry7cn-clang-wrapper-16.0.6/bin/c++(B(B
firefox-unwrapped> 0:02.14(B checking whether the host C++ compiler can be used... yes(B(B
firefox-unwrapped> 0:02.14(B checking the host C++ compiler version... 16.0.6(B(B
firefox-unwrapped> 0:02.21(B checking the host C++ compiler works... yes(B(B
firefox-unwrapped> 0:02.40(B checking for target linker... lld(B(B
firefox-unwrapped> 0:02.51(B checking for host linker... lld(B(B
firefox-unwrapped> 0:02.60(B checking for 64-bit OS... yes(B(B
firefox-unwrapped> 0:02.67(B checking for new enough STL headers from libstdc++...(B(B
firefox-unwrapped> 0:02.67(B DEBUG: <truncated - see config.log for full output>(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | #if defined(__GLIBCXX__) && !defined(_GLIBCXX_RELEASE)(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | # error libstdc++ not new enough(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | #endif(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | #if defined(_GLIBCXX_RELEASE)(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | # if _GLIBCXX_RELEASE < 8(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | # error libstdc++ not new enough(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | # else(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | (void) 0(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | # endif(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | #endif(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | ;(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | return 0;(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | }(B(B
firefox-unwrapped> 0:02.67(B DEBUG: Executing: `/nix/store/7v4bi4q334yircaznwm353h1l5i7k98f-aarch64-unknown-linux-gnu-clang-wrapper-16.0.6/bin/aarch64-unknown-linux-gnu-clang++ /build/conftest.qvbo58dv.cpp -c`(B(B
firefox-unwrapped> 0:02.67(B DEBUG: The command returned non-zero exit status 1.(B(B
firefox-unwrapped> 0:02.67(B DEBUG: Its error output was:(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | /build/conftest.qvbo58dv.cpp:1:10: fatal error: 'cstddef' file not found(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | #include <cstddef>(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | ^~~~~~~~~(B(B
firefox-unwrapped> 0:02.67(B DEBUG: | 1 error generated.(B(B
firefox-unwrapped> 0:02.67(B ERROR: The libstdc++ in use is not new enough. Please run ./mach bootstrap to update your compiler, or update your system libstdc++ installation.(B(B
firefox-unwrapped> *** Fix above errors and then restart with "./mach build"
error: build of '/nix/store/5in7xkji5hzqkl14ygwq3vxnni54lykk-firefox-unwrapped-aarch64-unknown-linux-gnu-119.0.1.drv' on 'ssh://root@192.168.22.103' failed: builder for '/nix/store/5in7xkji5hzqkl14ygwq3vxnni54lykk-firefox-unwrapped-aarch64-unknown-linux-gnu-119.0.1.drv' failed with exit code 1
error: builder for '/nix/store/5in7xkji5hzqkl14ygwq3vxnni54lykk-firefox-unwrapped-aarch64-unknown-linux-gnu-119.0.1.drv' failed with exit code 1;
```
this makes it a lot easier to create a modified stdenv with a
different set of defaultHardeningFlags and as a bonus allows us
to inject the correct defaultHardeningFlags into toolchain wrapper
scripts, reducing repetition.
while most hardening flags are arguably more of a compiler thing,
it works better to put them in bintools-wrapper because cc-wrapper
can easily refer to bintools but not vice-versa.
mkDerivation can still easily refer to either when it is constructed.
this also switches fortran-hook.sh to use the same defaults for
NIX_HARDENING_ENABLE as for C. previously NIX_HARDENING_ENABLE
defaults were apparently used to avoid passing problematic flags
to a fortran compiler, but this falls apart as soon as mkDerivation
sets its own NIX_HARDENING_ENABLE - cc.hardeningUnsupportedFlags
is a more appropriate mechanism for this as it actively filters
out flags from being used by the wrapper, so switch to using that
instead.
this is still an imperfect mechanism because it doesn't handle a
compiler which has both langFortran *and* langC very well - applying
the superset of the two's hardeningUnsupportedFlags to either
compiler's invocation. however this is nothing new - cc-wrapper
already poorly handles a langFortran+langC compiler, applying two
setup hooks that have contradictory options.
-B must be set to the root directory of avrlibc, otherwise gcc cannot
locate crt objects for some attiny devices. -L trains as set by
bintools-wrapper are not necessary with -B set correctly because gcc
takes care of that, and likewise we can drop the -B train from
cc-wrapper because the one spec is enough.
Before this commit, cc-wrapper/default.nix was using
`isGccArchSupported` to validate `-mtune=` values. This has two
problems:
- On x86, `-mtune=` can take the same values as `-march`, plus two
additional values `generic` and `intel` which are not valid for
`-march`.
- On ARM, `-mtune=` does not take the same values as `-march=`;
instead it takes the same values as `-mcpu`.
This commit fixes these two problems by adding a new
`isGccTuneSupported` function. For `isx86` this returns `true` for
the two special values and otherwise defers to `isGccArchSupported`.
This commit also adds support for `-mtune=` on Aarch64.
Unfortunately on Aarch64, Clang does not accept as wide a variety of
`-mtune=` values as Gcc does. In particular, Clang does not tune
for big.LITTLE mixed-model chips like the very popular RK3399, which
is targeted using `-march=cortex-a72.cortex-a53` in gcc.
To address this problem, this commit also adds a function
`findBestTuneApproximation` which can be used to map
clang-unsupported tunings like `cortex-a72.cortex-a53` to
less-precise tunings like `cortex-a53`.
The work which led to this commit arose because we now have
packages, like `crosvm`, which use *both* `clang` *and* `gcc`.
Previously I had been using `overrideAttrs` to set
`NIX_CFLAGS_COMPILE` on a package-by-package basis based on which
compiler that package used. Since we now have packages which use
*both* compilers, this strategy no longer works.
I briefly considered splitting `NIX_CFLAGS_COMPILE` into
`NIX_CFLAGS_COMPILE_GCC` and `NIX_CFLAGS_COMPILE_CLANG`, but since
`NIX_CFLAGS_COMPILE` is sort of a hack to begin with I figured that
adding the logic to `cc-wrapper` would be preferable.
The commit 6f2b3ba027 introduced a
`mktemp` invokation that uses the `--tmpdir` flag, which is not
available on MacOS.
This changes the invokation to a portable one based on the following
StackOverflow answer https://stackoverflow.com/a/31397073/841562 .
Removed workaround from llvm 16.
Fixes including cxxabi.h on llvm >=15 libcxxStdenv.
```c
int main() {}
```
```
/nix/store/qwnvng0cbyx0bijm654jpmpl0516hfhx-libcxxabi-15.0.7-dev/include/cxxabi.h:20:10: fatal error: '__cxxabi_config.h' file not found
```
Before llvm 15 this used to work because `libcxx` copied the headers
from `cxxabi` to it's own `include`, which was then picked up by the
line above this one
Alternative fix would be to copy all files from `${cxxabi.dev}/include/c++/v1` to `${cxxabi.dev}/include` so the cc-wrapper setup hook would pick them up, but that would depend on in cxxabi being in buildInputs.
Signed-off-by: Amneesh Singh <natto@weirdnatto.in>
Without this, pkgsStatic.pkgsLLVM.hello fails with segfaulting binaries
because of the issue described at [0].
In summary, llvm's linker has a different behaviour to GCC's when
supplied with both -static and -Wl,-dynamic-linker=...; GCC copes with
it, but LLVM produces a binary which segfaults on startup. It appears to
be necessary to omit the dynamic linker in this case.
nixpkgs' static adaptor passes -static via NIX_CFLAGS_LINK which was not
accounted for prior to this commit in the checkLinkType logic. For good
measure I put the other NIX_ flags affecting link in the same logic.
Additionally, $NIX_CFLAGS_LINK_@suffixSalt@ is not available until later
than it was originally set, so set $linkType close to its point of use.
I checked for earlier uses by studying the shell trace output and
couldn't find any.
[0] https://github.com/NixOS/nixpkgs/issues/111010#issuecomment-1536424163
Signed-off-by: Peter Waller <p@pwaller.net>
this splits hardeningCFlags into hardeningCFlagsAfter and
hardeningCFlagsBefore (where most flags still remain) to allow
us to *append* `-D_FORTIFY_SOURCE=` values to the command-line,
forcing our choice of fortify level and avoiding potential
redefinition warnings/errors through use of `-U_FORTIFY_SOURCE`
The Darwin stdenv rework conditionally sets `NIX_CC_USE_RESPONSE_FILE`
depending on the `ARG_MAX` of the build system. If it is at least 1 MiB,
the stdenv passes the arguments on the command-line (like Linux).
Otherwise, it falls back to the response file. This was done to prevent
intermitent failures with clang 16 being unable to read the response
file. Unfortunately, this breaks `gccStdenv` on older Darwin platforms.
Note: While the stdenv logic will also be reverted, this change is
needed for compatibility with clang 16.
GCC is capable of using a response file, but it does not work correctly
when the response file is a file descriptor. This can be reproduced
using the following sequence of commands:
$ nix shell nixpkgs#gcc; NIX_CC_USE_RESPONSE_FILE=1 gcc
# Linux
/nix/store/9n9gjvzci75gp2sh1c4rh626dhizqynl-binutils-2.39/bin/ld: unrecognized option '-B/nix/store/vnwdak3n1w2jjil119j65k8mw1z23p84-glibc-2.35-224/lib/'
/nix/store/9n9gjvzci75gp2sh1c4rh626dhizqynl-binutils-2.39/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
# Darwin
ld: unknown option: -mmacosx-version-min=11.0
collect2: error: ld returned 1 exit status
Instead of using process substitution, create a temporary file and
remove it in a trap. This should also prevent the intermitent build
failures with clang 16 on older Darwin systems.
Fixes#245167
When a response file is in use, "$*" contains the response file and not
the parameters; both the linker and compiler wrappers are updated to use
the response-expanded params.
The compiler driver likes to pass parameters to the linker via a
response file, including -shared.
LLD rejects the combination of (-shared -pie), whereas other linkers
silently ignore the contradiction:
```
ld.lld: error: -shared and -pie may not be used together
```
This breaks certain configurations using LLD as a linker.
Changing `add-hardening.sh` results in a full rebuild. To avoid the
rebuild, here is a quick test case which shows the new hardening script
allows the link to succeed:
```
{ pkgs ? import <nixpkgs> {} }:
let
# gcc silently accepts -shared -pie together, lld does not.
linker = pkgs.wrapBintoolsWith { bintools = pkgs.llvmPackages.lld; };
patchWrapper = prev: prev.overrideAttrs (final: prev: let
prevScript = builtins.match (".*(/nix/store/[a-z0-9]+-add-hardening.sh).*") prev.postFixup;
in {
postFixup = (builtins.replaceStrings prevScript ["${./new-add-hardening.sh}"] prev.postFixup);
});
in
pkgs.stdenv.mkDerivation {
name = "nixpkgs-hardening-bug";
src = pkgs.writeText "src.c" "int main(int argc, char* argv[]) { return 0; }";
NIX_HARDENING_ENABLE = "pie";
unpackPhase = ":";
buildPhase = ''
$CC -c -o src.o $src
bash -x ${patchWrapper linker}/bin/ld.lld -o $out @${pkgs.writeText "responsefile" "-shared"} src.o
'';
}
```
Fixes: #178162
Signed-off-by: Peter Waller <p@pwaller.net>
Our gcc_multi and glibc_multi expressions merge together a
32-bit-targeted and 64-bit-targeted gcc. However they do not thread
through the passthru.libgcc from these merged gccs.
This commit corrects that.
It also extends passthru.libgcc to allow a *list* rather than just a
single outpath.
Resolves part of #221891 (at least getting it back to the error
message it gave before).
According to <https://gcc.gnu.org/legacy-ml/gcc-patches/2015-08/msg00836.html>,
all code is position-independent on Windows. Some compilers
apparently warn for -fPIC on Windows, and clang errors:
> clang-15: error: unsupported option '-fPIC' for target 'x86_64-pc-windows-msvc'
I'm guessing the check was hostPlatform instead of targetPlatform by mistake.
In https://github.com/NixOS/nixpkgs/pull/209870 I tried to unify the
treatment of clang and gcc in cc-wrapper as much as possible.
However it appears that I went too far.
Clang requires -isystem flags in order to be able to find gcc's
libstdc++. Gcc does not need these flags. If they are added,
gfortran will get confused:
https://github.com/NixOS/nixpkgs/pull/209870#issuecomment-1500550903
This commit deunifies the chunk of code that adds the -isystem
flags, and explains why this chunk applies only to clang.
When wrapping `clang` and using a `gccForLibs` whose `libgcc` is in
its own output (rather than the `lib` output), this commit will adds
`-L${gccForLibs.libgcc}/lib` to `cc-ldflags`.
If that flag is not added, `firefox` will fail to compile because it
invokes `clang-wrapper` with `-fuse-ld=lld` and passes `-lgcc_s` to
`lld`, but does not tell `lld` where to find `libgcc_s.so`. In that
situation, firefox will fail to link.
Without this change $target-cpp is used unwrapped and is missing
standard header search paths among other things).
Example failure:
$ nix build -f. -L pkgsStatic.netbsd.compat
...
> checking how to run the C preprocessor... x86_64-unknown-linux-musl-cpp
> configure: error: in `/build/cvs-export/tools/compat':
> configure: error: C preprocessor "x86_64-unknown-linux-musl-cpp" fails sanity check
> See `config.log' for more details
cc-wrapper has essentially two separate codepaths: the `gccForLibs`
codepath, used only by non-gcc (i.e. clang) compilers, and the
"other" codepath.
This PR allows non-clang compilers to opt-in to the `gccForLibs`
codepath (off by default). To allow this, a new parameter
`ccForLibs` is exposed, since it would be extremely confusing for
gcc to be able to use `gccForLibs` but not do so by default.
This reverts commit 8c80bd08b7
("build-support/cc-wrapper: pass in non-existent --sysroot= to untangle
from libc").
This change was good in spirit: we caught a few genuine problems with
`scons` based packages (`godot`, `fluxus`) and unexpected `-idirafter`
includes in various boot loadres (`ipxe`, wimboot`):
https://github.com/NixOS/nixpkgs/pull/210004#issuecomment-1407162693
Unfortunately `--sysroot=` also has a negative impact on libary search
order for DT_NEEDED libraries and RUNPATHs of linked libraries. This
unexpectedly broke `dmd`, `d-seams`, `llvmPackages_rocm.compiler-rt`).
An interesting case of unexpected breakage is `usbmuxd2` where the bug
exposed incomplete library move on `libstdc++fs` in `gcc`.
The library breakage is very non-intuitive (on top of already unusual
layout of `cc-wrapper` driver). Let's revert this change for now.
Once it lands we can undo `--sysroot=/` workarounds merged for
`staging-next`.
After https://github.com/NixOS/nixpkgs/pull/210004 `usbmuxd2` started
failing to build as:
usbmuxd2-unstable> .../ld: cannot find -lstdc++fs: No such file or directory
usbmuxd2-unstable> clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
This started happening because #210004 exposed a long-standing bug in
`gcc` derivation: `cc.lib` is missing `libstdc++fs` library:
$ find $(nix-build --no-link -A stdenv.cc.cc.lib) | fgrep libstdc | unnix
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++fs.la
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.la
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.so.6.0.29
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.so
/<<NIX>>/gcc-11.3.0-lib/lib/libstdc++.so.6
It was not moved from `cc.out` output:
$ find $(nix-build --no-link -A stdenv.cc.cc) | fgrep libstdc | unnix
/<<NIX>>/gcc-11.3.0/lib/libstdc++.a
/<<NIX>>/gcc-11.3.0/lib/libstdc++fs.a
This change adds `cc` library lookup path back to `staging-next` until
`gcc` is fixed.`
FreeBSD doesn't use LLVM's cxxabi implementation, for backwards
compatibility reasons. Software expects the libcxxrt API when
building on FreeBSD. This fixes the build of
pkgsCross.x86_64-freebsd.boost.
Gcc does not allow `-march=` on PowerPC:
https://gcc.gnu.org/onlinedocs/gcc-12.2.0/gcc/RS_002f6000-and-PowerPC-Options.html#RS_002f6000-and-PowerPC-Options
Instead, `-mcpu=` should be used to set the minimum instruction set
and `-mtune=` is used to optimize instruction scheduling for a
specific processor. Both flags take the same set of valid values,
which includes `native`.
This commit causes `isGccArchSupported` to return `false` for PowerPC
targets so we never pass an `-march=` flag, since that will always be
rejected by gcc.
I would like to add an extra `gcc` build step during linux bootstrap
(https://github.com/NixOS/nixpkgs/issues/208412). This makes it early
bootstrap compiler linked and targeted against `bootstrapTools` `glibc`
including it's headers.
Without this change `gcc`'s spec files always prefer `bootstrapTools` `glibc`
for header search path (passed in as --with-native-system-header-dir=). We'can't
override it with:
- `-I` option as it gets stacked before gcc-specific headers, we need to keep
glibc headers after gcc as gcc cleans namespace up for C standard by using
#include_next and by undefining system macros.
- `-idirafter` option as it gets appended after existing `glibc`-includes
This `--sysroot=/nix/store/does/not/exist` hack allows us to remove existing
`glibc` headers and add new ones with `-idirafter`.
We use `cc-cflags-before` instead of `libc-cflags` to allow user to define
their own `--sysroot=` (like `firefox` does).
To keep it working prerequisite cross-symlink in gcc.libs is required:
https://github.com/NixOS/nixpkgs/pull/209153
Switches that gnatmake needs to pass to gcc must be given as
"-cargs <gcc_switches>" after at least the files to compile (see the
gnatmake docs for all the subtleties). This commit makes that happen,
in a way that triggers rebuilds only for things that actually depend on
GNAT, and not the other compilers contained in GCC.
