Reverts #162607 / 1748887ff2.
Reason for revert: This change caused llvm-config{,-native} to be unable
to find static archives bundled with LLVM, as has been [reported]. Ever
since #152944 using moveToOutput in LLVM is _evil_ because llvm-config
obtains it knowledge about the installation locations from the CMake
configure step.
Consequently a change like #162607 will need to be implemented by making
LLVM itself install the static archives to the correct location or by
adding yet another patch which updates llvm-config's knowledge of the
location. The latter is not desireable in my opinion, though, since it
is just asking for this sort of trouble: Before #152944 we had an
outputs.patch that did this sort of things which broke spectacularly in
edge cases.
Fixes#148117.
[reported]: https://github.com/NixOS/nixpkgs/issues/148117#issuecomment-1158245576
lld package provides an unwrapped lld. It doesn't always work on NixOS
(eg, it doesn't set rpath), and so dosen't always work.
What one should be using instead is the `lld` from
`llvmPackages.bintools` package. This super counterintutive.
One incremental step we can take here is to clarify that the `lld`
package is unrwapped -- right now, it looks like 100% legit thing one
should be using!
Reduces closure size by ~240MiB (down to ~100MiB) for
LLVM 13, the others are similar.
Having those archives in the lib output makes no sense
as they are no runtime dependencies. Removing them
alltogether is also not an option because the dynamic
libraries offer only the C API while many users of
libllvm require the C++ API. Those users must have an
dependency on libllvm.dev anyway and will find those
files for linking.
libcxxClang still depends on cc wrapper's gccForLibs for libgcc which is
not available when useLLVM is set. In such cases we need to switch to
clangUseLLVM and (try) to use compiler-rt instead.
Resolves#153759: pkgsLLVM.llvmPackages.stdenv now correctly
clangUseLLVM as cc, allowing compilation to work as expected.
llvmPackages_*.clang should check the default compiler for the package
set it is targeting (targetPackages.stdenv.cc) instead of the compiler
that has been used to build it (stdenv.cc) in order to get some sense of
whether to use libc++ or libstdc++.
Since we are now inspecting targetPackages in the llvmPackages.clang
attribute, we need to avoid using it in the cross stdenv — which just
forces us to explicitly request libcxxClang for darwin instead of
relying on the clang attribute to pick it for us.
We also need to do something similar for targetPackages.stdenv.cc: Here
the llvmPackages.clang logic would work as we want (inspect
targetPackages.stdenv.cc and if it doesn't exist, make the choice based
on stdenv.cc), but it gets locked in a cycle with the previous package.
We can easily break this, however: We know that the previous set had
clang and the next one doesn't exist, so we'd choose libcxxClang any day
of the week.
To make the value of doCheck in the argument attribute set accurate we
also need to include the condition for cross compilation which normally
is added by stdenv.mkDerivation.
LLVM's build system creates NATIVE/bin/llvm-config by reexecuting cmake
with entirely new flags. Problematically, the `CMAKE_INSTALL_*` flags
are not inherited, causing llvm-config-native to return wrong
installation paths, e. g. CMAKE_INSTALL_LIBDIR would default to `lib64`
on x86_64-linux. Previously this was masked by outputs.patch which
replaced ActiveLibDir with a string passed in from Nix, however
`--cmakedir` for example would turn out to be wrong always, breaking
cross-compilation of e. g. lld.
Additionally LLVM_ENABLE_RTTI needs to be repassed, as it is used to
determine if RTTI is available. Passing LLVM_LINK_LLVM_DYLIB is crucial
if we are building LLVM non-statically: It influences the --shared-mode
flag (which should indicate that -lLLVM is enough to link all
components) and makes --link-shared work in the first place,
i. e. llvm-config-native believes the built shared libs don't exist
unless we repass this flag.
Passing LLVM_LINK_LLVM_DYLIB=ON, however, makes the native build produce
a full libLLVM.so which is something we don't want, so we introduce a
patch which forces llvm-config to link statically against the LLVM
components it needs.
In https://gcc.gnu.org/PR103598 we found out that gcc-12
changed __PRETTY_FUNCTION__ slightly and broke llvm-12 (and older)
tests that rely on exact type match. llvm-13 already removed the
qualified names from the expected output.
This change changes expected output to avoid llvm:: namespace prefix.
This is enough to get tests pass on x86_64 again.
Tested against this week's gcc-12 and against gcc-10.
Since both static and shared libs are installed to the same `lib`
output, we override the ActiveLibDir unconditionally.
Fixes `llvm-config-native --link-static --libs`
Without the fix build fails on gcc-12 as:
/build/llvm/lib/Target/NVPTX/NVPTXAsmPrinter.cpp:
In member function 'std::string llvm::NVPTXAsmPrinter::getPTXFundamentalTypeStr(...':
/build/llvm/lib/Target/NVPTX/NVPTXAsmPrinter.cpp:1319:10:
error: use of deleted function 'std::__cxx11::basic_string<...>; std::nullptr_t = std::nullptr_t]'
1319 | return nullptr;
| ^~~~~~~
The patch is present in upstream releases since llvm-12.
Commit 199b7c50 "compiler-rt: remove <cyclades.h> from libsanitizer"
broke conditional conditional musl patches.
The change has a few effects:
- pkgsStatic.llvmPackages_{5,6,7}.compiler-rt: fix build on musl after cyclades backport
- pkgsStatic.llvmPackages_{{5..13},git}.compiler-rt: drop incomplete musl patches as
sanitizers are disabled anyway and require more upstream porting.
the fix to extendDerivation in #140051 unwittingly worsened eval performance by
quite a bit. set elements alone needed over 1GB extra after the change, which
seems disproportionate to how small it was. if we flip the logic used to
determine which outputs to install around and keep a "this one exactly" flag in
the specific outputs instead of a "all of them" in the root we can avoid most
of that cost.
linux-headers-5.13 removed <cyclades.h> along with device support.
Backport a single https://reviews.llvm.org/D102059 upstream change to
fix compiler-rt build.
Use local patches as there is a whitespace change compared to upstream.
When using GNU binutils, clang passes the LLVMgold.so plugin to the
linker for certain operations that require special support in the linker
like doing link time optimization (LTO). When passing the plugin to the
linker's command line, clang assumes that llvm and itself are installed
in the same prefix and thus `/path/to/clang/bin/../lib/LLVMgold.so` is
the plugin.
Since we install clang and llvm to separate store paths, this assumption
does not hold. When clang-unwrapped only had a single output, we worked
around this issue by symlinking `$out/lib/LLVMgold.so` to
`${llvm}/lib/LLVMgold.so`. However since we split all llvm packages into
multiple outputs clang's `$out` no longer has a lib directory and clang
can't discover clangs lib output on its own. As a result LTO was broken.
Instead of introducing yet another hack and having a symlink to
LLVMgold.so in `$out/lib` (despite having `$lib/lib` as well), we patch
clang to use a hard coded path to `${libllvm.lib}/lib` for discovering
`LLVMgold.so`.
Resolves#123361.
This is in an effort to fix the following build failure shown by
chromium:
clang++: error: no such file or directory: '/nix/store/fhd89wrmkx6nflzjk0d6waz70bk3zc4i-clang-wrapper-12.0.0/resource-root/share/cfi_blacklist.txt'
As it turns out a change introduced via the gnu-install-dirs.patch
caused `add_compiler_rt_resource_file` to install resource files to
$dev/include (FULL_INCLUDEDIR) instead of $out/share (FULL_DATADIR)
which in turn meant that the clang wrappers we had didn't link those
files to its resource root at all.
Alternative fix to this would have been to link
compiler-rt.dev/include/*.txt to the wrappers resource-root/share as
well, but since this was handled inconsistently across the patch anyways
(the dfsan list is installed correctly), opt to handle this
consistently within the patch.
llvmPackages_{5,6} install the resource files to a completely different
location and need separate investigation.
This will begin the process of breaking up the `useLLVM` monolith. That
is good in general, but I hope will be good for NetBSD and Darwin in
particular.
Co-authored-by: sterni <sternenseemann@systemli.org>
The bintools argument received a wrapped version of tools.bintools which
is already wrapped. Wrapped bintools twice leads to users of lldClang
being unable to find the tools which are not wrapped like ar.
The main thing was using `llvm_meta` in all versions.
Secondarily:
- libunwindx7: Forgot to split outputs
- libcxx{,abi} 12: Forgot to apply output-splitting patches.
- simplify `useLLVM` stdenv-switching logic.
- openmp always gets its own directory
- Introduce `preLibcCrossHeaders` to bootstrap libgcc and compiler-rt
the same way.
- Organize LLVM bintools as `bintools{-unwrapped,,NoLibc}` for
consistency with GNU Binutils and Apple's cctools.
- Do Android changes for all `llvmPackages` for consistency.
- Improve the way the default GCC and LLVM versions are selected.
In 7869d16545 I got rid of the symlinking
by forcing `COMPILER_RT_OS_DIR` to always be the empty string. I thought
this was good because it just make compiler-rt be installed in a normal
way.
However, various LLVM tools expect the `COMPILER_RT_OS_DIR` to be set
normally, and fail to find things when they aren't in the expected lib
subdir.
Maybe it would be best to patch that too in the long term, but for now
we just undo this change.
Before, clang was able to find some headers with a relative path to the
`-B` flag pointing near the unwrapped clang binary. But with multiple
outputs that doesn't work, so we use a "resource directory" as it done
later in the bootstrap.
Also begin to start work on cross compilation, though that will have to
be finished later.
The patches are based on the first version of
https://reviews.llvm.org/D99484. It's very annoying to do the
back-porting but the review has uncovered nothing super major so I'm
fine sticking with what I've got.
Beyond making the outputs work, I also strove to re-sync the packages,
as they have been drifting pointlessly apart for some time.
----
Other misc notes, highly incomplete
- lvm-config-native and llvm-config are put in `dev` because they are
tools just for build time.
- Clang no longer has an lld dep. That was introduced in
db29857eb3, but if clang needs help
finding lld when it is used we should just pass it flags / put in the
resource dir. Providing it at build time increases critical path
length for no good reason.
----
A note on `nativeCC`:
`stdenv` takes tools from the previous stage, so:
1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.stdenv.cc`: `(?0, ?1, x)`
while:
1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.targetPackages`: `(x, x, ?2)`
3. `pkgsBuildBuild.targetPackages.stdenv.cc`: `(?1, x, x)`
In 486e12ad68 cmake flags were added matching
later compilers use of libunwind for `useLLVM = true`. Unfortunately, `useLLVM`
on Darwin was not something tested before, and so the other compilers led us
astray: one of the new flags tried to make libunwind be used when it wasn't a
dep.
This is now fixed with more conditional code, but I hope things can perhaps be
made simpler with more insight into why libunwind is skipped. Perhaps it is
included in libSystem?
Finally, I moved the definition of `cmakeFlags` to match the order in the other
llvm versions.
CC @sternenseemann and @thefloweringash
This flag was introduced for clang 9, but we use it in the `lldClang`
wrapper for `llvmPackages` 7, 8 and 9. For this purpose the patch was
backported for `llvmPackages_8.clang`, but not for `llvmPackages_7.clang`
which has been done in this commit.
`lldClang` is mostly used when cross compiling and
`stdenv.hostPlatform.useLLVM` is true. Most likely this problem wasn't
noticed since `useLLVM` with `llvmPackages_7` was broken for other
reasons as well and all cross targets (like `wasi32`) which have
`useLLVM` at the moment use `llvmPackages_8`.
With this change tests.cross.llvm.hello.{musl64, …} works again.
This reverts commit 76b54c75b3 and brings
llvmPackages_7.libcxxabi in line with what the other llvmPackages
sets are doing again (with llvmPackages_7 being the sole outlier).
This also fixes an evaluation error of llvmPackages_7.libcxxabi if
stdenv.hostPlatform.useLLVM is true as the nonexistant libunwind
argument would be overridden.
This patches are included from libcxx and libcxxabi when
stdenv.hostPlatform.isMusl. After #117433 the patchs to that patch
wasn't adjusted for the new structure, likely because it doesn't come up
during normal eval. This fixes (among other attribute paths):
* pkgsMusl.llvmPackages_12.libcxxabi
* pkgsMusl.llvmPackages_12.libcxx
* pkgsMusl.llvmPackages_11.libcxxabi
* pkgsMusl.llvmPackages_11.libcxx
* pkgsMusl.llvmPackages_10.libcxxabi
* pkgsMusl.llvmPackages_10.libcxx
* pkgsMusl.llvmPackages_9.libcxxabi
* pkgsMusl.llvmPackages_9.libcxx
* pkgsMusl.llvmPackages_8.libcxxabi
* pkgsMusl.llvmPackages_8.libcxx
* pkgsMusl.llvmPackages_7.libcxxabi
* pkgsMusl.llvmPackages_7.libcxx
* pkgsMusl.llvmPackages_6.libcxxabi
* pkgsMusl.llvmPackages_6.libcxx
* pkgsMusl.llvmPackages_5.libcxxabi
* pkgsMusl.llvmPackages_5.libcxx
Only evaluation was tested, not compilation though.
We can use use `stdenv.hostPlatform.isStatic` instead, and move the
logic per package. The least opionated benefit of this is that it makes
it much easier to replace packages with modified ones, as there is no
longer any issue of overlay order.
CC @FRidh @matthewbauer
Build the llvm support libraries (libcxx, libcxxabi) from scratch
without using the existing llvm libraries. This is the same spirit and
similar implementation as the "useLLVM" bootstrap in llvm package
sets. Critically it avoids having libcxxabi provided by the cc-wrapper
when building libcxx, which otherwise results in two libcxxabi
instances.
$ otool -L /nix/store/vd4vvgs9xngqbjzpg3qc41wl6jh42s9i-libc++-7.1.0/lib/libc++.dylib
/nix/store/vd4vvgs9xngqbjzpg3qc41wl6jh42s9i-libc++-7.1.0/lib/libc++.dylib:
/nix/store/vd4vvgs9xngqbjzpg3qc41wl6jh42s9i-libc++-7.1.0/lib/libc++.1.0.dylib (compatibility version 1.0.0, current version 1.0.0)
/nix/store/gmpwk5fyp3iasppqrrdpswxvid6kcp8r-libc++abi-7.1.0/lib/libc++abi.dylib (compatibility version 1.0.0, current version 1.0.0)
/nix/store/3hn7azynqgp2pm5gpdg45gpq0ia72skg-libc++abi-7.1.0/lib/libc++abi.dylib (compatibility version 1.0.0, current version 1.0.0)
/nix/store/1nq94scbxs6bk7pimqhvz76q6cfmbv97-Libsystem-osx-10.12.6/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
Additionally move some utilities (clang, binutils, coreutils, gnugrep)
to the stage layers so they can be replaced before the final
stdenv. This should cause most of stage4 to be built from the
toolchain assembled as of stage3 instead of the bootstrap toolchain.
This reverts commit c778945806.
I believe this is exactly what brings the staging branch into
the right shape after the last merge from master (through staging-next);
otherwise part of staging changes would be lost
(due to being already reachable from master but reverted).
This is needed for cross-compiling for LLVM.
After https://github.com/NixOS/nixpkgs/pull/94088, we still need some
of these, so I’ve whitelisted those that are in binutils.
/cc @DavidTruby
These llvm-prefixed utilities are not drop-in replacements for the utilities
with similar names, they are specifically for operating on LLVM IR files.
Symlinking these without the prefix causes incompatibilities with tools that
expect diff, as and others to behave normally.
I hate the thing too even though I made it, and rather just get rid of
it. But we can't do that yet. In the meantime, this brings us more
inline with autoconf and will make it slightly easier for me to write a
pkg-config wrapper, which we need.
Naive concatenation of $LD_LIBRARY_PATH can result in an empty
colon-delimited segment; this tells glibc to load libraries from the
current directory, which is definitely wrong, and may be a security
vulnerability if the current directory is untrusted. (See #67234, for
example.) Fix this throughout the tree.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
