Port of 6d0c87602f ("llvmPackages_15.llvm: add in a missing check
dep").
I did not include the sysctl nativeCheckInputs change from that
commit, as it looks like it was included by mistake and was supposed
to be in c7231c0b6d ("llvmPackages_15.llvm: run the tests on macOS").
I've also included the doCheck default from that commit (without the
change to run on non-Linux), as 6d0c87602f just set it to true.
Port of 404ef6b6c9 ("llvmPackages_15.clang: drop the C++ std version
`-DCMAKE_CXX_FLAG`"). The commit mentioned as removing the need for
the override[1] first appeared in LLVM 10.
[1]: 2724d9e129
Port of 2a58596dd2 ("llvmPackages_15.llvm: enable polly by default").
It should be ported all the back to LLVM 12, but I haven't done that
here because going further back than 14 requires fixing patches, and
I'm focused on my goal of getting the delta between LLVM 15 and LLVM
git under control — it's going to be hard enough to accomplish that.
compiler-rt has accumulated several regressions that prevent it from building
on ARMv6. It is important to note that there are two major versions of ARMv6:
base ARMv6 and ARMv6K. ARMv6K includes several important new instructions,
such as non-word size atomic operations (ldrexd, strexd, etc.) and the yield
instruction. Most ARMv6 CPUs actually implement ARMv6K, including all those used
in Raspberry Pis, but nixpkgs' "raspberryPi" platform targets base ARMv6.
compiler-rt versions 8-14 fail to build on ARMv6 and ARMv6K. compiler-rt 15 (not
yet in nixpkgs) builds on ARMv6K but not ARMv6. This patch fixes versions 9-14
on both ARMv6 variants. The patches don't apply cleanly to version 8, and I
figured it wasn't worth carrying another version of the patches for such an old
version.
A total of five patches are required to get compiler-rt building on ARMv6:
* armv6-mcr-dmb.patch: use `mcr` to provide the equivalent of `dmb` on ARMv6.
Included in LLVM 15.
* armv6-sync-ops-no-thumb.patch: prevent certain atomic operation functions
from using Thumb mode. Included in LLVM 15.
* armv6-no-ldrexd-strexd.patch: don't use ldrexd or strexd, which are not
available in base ARMv6. Submitted upstream by me.
* armv6-scudo-no-yield.patch: use nop instead of yield on ARMv6 in standalone
scudo. Required by versions >=13, since they enable standalone scudo.
Submitted upstream by me.
* armv6-scudo-libatomic.patch: link standlone scudo to libatomic on ARMv6 (and
any other platforms that need it). Not yet submitted because the backport is
a bit different from the upstream version and I need to test it.
This was added in 00839fea22 ("llvmPackages_15.compiler-rt: fixes for Darwin"),
which says:
> this introduces a codesigning related patch that we can drop once
> #195107 goes through
The referenced PR ("sigtool: 0.1.2 → 0.1.3") has now been merged, so
I'm dropping the patch as instructed.
The effect of `.out // { outputSpecified = false; }` in these cases
is to select the default output explicitly, but then make the
selection implicit until `overrideAttrs` is called. Previously
`overrideAttrs` would not preserve output selection, masking the
apparently unnecessary behavior of this workaround.
For `libllvm-polly`, this logic does not apply, as it does not
select the default output.
The `outputSpecified` workaround was introduced in
https://github.com/NixOS/nixpkgs/pull/122554
and was perhaps rushed because of a release deadline, and expected
delays from mass rebuilds.
The change in `overrideAttrs` behavior was added in
5b2f597b11 / https://github.com/NixOS/nixpkgs/pull/211685
and the problem was discovered in https://github.com/NixOS/nixpkgs/pull/218537,
which may contain further information.
Port of 44165d3657 ("llvmPackages_{14, git}.clang: add nostdlibinc flag")
to Clang 15. It was originally thought this wasn't needed[1], but it is,
to fix expressions like the following:
with import ./. {};
llvmPackages_15.libcxxStdenv.mkDerivation {
name = "libcxx-stdenv-c++-test";
dontUnpack = true;
input = ''
#include <cstdlib>
int main() {
std::abort();
return 0;
}
'';
passAsFile = [ "input" ];
installPhase = ''
$CXX -c -o $out -x c++ $inputPath
'';
}
[1]: https://github.com/NixOS/nixpkgs/pull/194634#issue-1398202534
This reverts commit 386aba3115.
As I understand it from reading
<https://llvm.org/docs/DeveloperPolicy.html#copyright-license-and-patents>,
the structure of LLVM licensing is as follows:
- They're in the process of relicensing to Apache-2.0 WITH LLVM-exception,
but they haven't got permission to relicense all the code yet.
This means that some of the code can be used under the new license,
but not all of it, and it's difficult to know which is which. This
license is therefore probably not useful yet, until the relicensing
effort is commit.
- While the relicensing effort is ongoing, code being contributed to
LLVM has to have permission to be used under the old and new
licensing schemes. Since the new licensing scheme can't be used
for all code yet, it only makes sense to use LLVM's code under the
old licensing scheme at the moment.
- The old licensing scheme is that code for the LLVM components we
care about is all available under the NCSA license, and some
components are optionally available under a different license,
usually the MIT license, instead.
So I think we should go back to just listing NCSA, or NCSA/MIT, and
forget about the new license until it actually becomes useful,
i.e. LLVM's relicensing effort is complete.
LLVM-exception only makes sense when used with the Apache 2.0 license,
so let's combine them, so it's not possible to forget one of them like
happened with llvm_15.
with structuredAttrs lists will be bash arrays which cannot be exported
which will be a issue with some patches and some wrappers like cc-wrapper
this makes it clearer that NIX_CFLAGS_COMPILE must be a string as lists
in env cause a eval failure
I got the plugin API support at least once incorrect. Instead of
copying the deifnition let's consolidate it within binutils itself.
While at it forward-ported changes to llvm_{13,14,15}.
The change is a no-op from rebuild perspective.
this is in preparation for the next commit which exposes the release
information and monorepo source as overridable args (which makes it
easier for users to use their own LLVM in nixpkgs)
this commit only adds a check in the `llvm` package's postConfigure that
makes sure the LLVM source provided matches the version we were given;
the actual machinery (functionally just a cosmetic change; causes no
rebuilds) is in the next commit
The two scenarios described within where splicing doesn't handle
selecting the right package for us are observable in the following
(nix repl session):
```
> np = import <nixpkgs> { system = "x86_64-linux"; crossSystem = { config = "aarch64-linux"; }; }
> np.__splicedPackages.hello ? __spliced
true
> np.__splicedPackages.python3Packages.psutil ? __spliced
true
> np.__splicedPackages.python3.pkgs.psutil ? __spliced
false
> (np.__splicedPackages.python3.withPackages (ps: with ps; [psutil])) ? __spliced
false
```
See: #211340
Details within but ultimately there isn't a satisfying resolution for
any of the three test failures we were seeing and all three deserve
further exploration.
For the `sw_vers` macOS version issue in particular, it's possible to
observe the nixpkgs provided `CoreFoundation` vs system `CoreFoundation`
for `x86_64` and `aarch64` like so (on a host running macOS `13.0.1`):
```console
$ nix-shell -p darwin.DarwinTools --system aarch64-darwin --command "sw_vers"
ProductName: macOS
ProductVersion: 13.0.1
BuildVersion: 22A400
$ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command "sw_vers"
ProductName: Mac OS X
ProductVersion: 10.16
BuildVersion: 22A400
```
Where `/System/Library/CoreServices/SystemVersion.plist` has:
```console
$ cat /System/Library/CoreServices/SystemVersion.plist | grep ProductVersion
-A 1
<key>ProductVersion</key>
<string>13.0.1</string>
```
Further:
```console
$ nix-shell -p darwin.DarwinTools --system aarch64-darwin --command 'otool -L $(which sw_vers)'
/nix/store/nb2q33ak2zif49ndcpc6m823z0vhmy8y-DarwinTools-1/bin/sw_vers:
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1770.255.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)
$ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command 'otool -L $(which sw_vers)'
/nix/store/88v4kjvgwl71byfpvd0baviiq7l5appc-DarwinTools-1/bin/sw_vers:
@rpath/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1454.90.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.60.2)
```
For the `x86_64` `sw_vers` binary we can see rpath:
```console
$ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command 'otool -l $(which sw_vers)' | grep LC_RPATH -A 2 -B 1
Load command 13
cmd LC_RPATH
cmdsize 120
path /nix/store/zvr4wypbgskhhw9cawfn7mmxfa75nh8f-swift-corefoundation-unstable-2018-09-14/Library/Frameworks (offset 12)
```
And we can confirm that the nixpkgs provided `CoreFoundation` is what
ultimately gets loaded:
```console
$ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command 'DYLD_PRINT_LIBRARIES=1 sw_vers'
dyld[16215]: <no uuid> /nix/store/88v4kjvgwl71byfpvd0baviiq7l5appc-DarwinTools-1/bin/sw_vers
dyld[16215]: <no uuid> /nix/store/zvr4wypbgskhhw9cawfn7mmxfa75nh8f-swift-corefoundation-unstable-2018-09-14/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
dyld[16215]: <no uuid> /nix/store/xd2a4xh8kdwq0j67hzgw720npdw5hzkk-ICU-66108/lib/libicucore.A.dylib
<snipped>
```
```bash
nix-diff \
$(nix path-info nixpkgs#legacyPackages.aarch64-darwin.darwin.DarwinTools --derivation) \
$(nix path-info nixpkgs#legacyPackages.x86_64-darwin.darwin.DarwinTools --derivation)
```
doesn't show any _obvious_ discrepancies
there are a few parts to this:
- adding darwin specific check deps
- working around referencing LLVM dylibs during the checkPhase in a
way that supports darwin
+ previously we just set `$LD_LIBRARY_PATH` and/or made some
strategic symlinks
+ now we have LLVM's `lit` config set the appropriate env vars as
needed (as is done for other LLVM subprojects)
+ in retrospect switching to `installCheckPhase` might have been the
better move..
- patching `lit` to deal with `$DYLD_LIBRARY_PATH` being purged for
new "protected" processes
more details within.
as detailed within, adding `asm/ptrace.h` leads to `asm/ptrace-abi.h`
being included which defines preprocessor symbols that clash with
identifiers used in the LLVM headers (`FS` and `CS` only defined on
i686)
This restores this check to what it originally was in #196909 (see:
https://github.com/NixOS/nixpkgs/pull/196909#discussion_r1000536298) and
lets `compiler-rt` eval successfully when trying to compile the
`llvmPackages_15` set for mingw targets (i.e. a platform that *is* GNU
but does *not* use glibc).
---
It's not clear to me what the `haveLibc` check is doing here (platforms
that seem to use glibc like `x86_64-linux` and have
`plat.libc == "glibc"` have `haveLibc = false` because `stdenv.cc.libc`
is `null`).
libc++ has switched to using `__attribute__((using_if_exists))` to handle
incomplete libc implementations; see: a9c9183ca4
These essentially require a modern C++ compiler (clang gained support in
LLVM 13: 369c648399,
gcc appears to not have support yet: https://gcc.gnu.org/bugzilla//show_bug.cgi?id=105584).
Previously this was not an issue for us (despite the transition happening
around LLVM 13) but something about the changes to the libc++/libc++-abi
build has made it so that on platforms with incomplete libc impls (i.e.
Darwin is missing `quick_exit`/`at_quick_exit`) we error during the `libcxx-abi`
build when the stdenv's (older, not supporting `using_if_exists`) compiler
tries to import libc symbols that aren't present.
The libc++ docs suggest we use a modern compiler to build libc++ anyways
(https://releases.llvm.org/15.0.0/projects/libcxx/docs/index.html#platform-and-compiler-support)
so this commit uses stdenv's containing the package set's clang to build
libcxx/libcxx-abi.
This is similar to how libc++ bootstrapping builds (https://releases.llvm.org/15.0.0/projects/libcxx/docs/BuildingLibcxx.html#bootstrapping-build)
work.
None of the patches required any touch-up; the only change of note is:
- due to changes in the libc++/libc++abi build
(https://reviews.llvm.org/D120719 and https://reviews.llvm.org/D131037)
we have to add an extra build option to the libc++ header only
build that sidesteps bits of the libc++ build config that assume
libc++-abi is present in the build:
4f827318e3/libcxx/src/CMakeLists.txt (L255-L256)
Rather than maintaining a precise set of build options that let us dodge
referencing libc++-abi variables in the libc++ header only build, we set
`LIBCXX_CXX_ABI` to `none`, as suggested by @lovesegfault.
More discussion about this here: https://github.com/NixOS/nixpkgs/pull/194634#discussion_r990267037
Co-authored-by: Bernardo Meurer <bernardo@meurer.org>
FreeBSD doesn't use LLVM's cxxabi implementation, for backwards
compatibility reasons. Software expects the libcxxrt API when
building on FreeBSD. This fixes the build of
pkgsCross.x86_64-freebsd.boost.
The exception for FreeBSD was added in 0afe9d1f70 ("freebsd packages:
Init at 13.1"), but it seems to have been erroneous, as e.g. ncurses
fails to build:
x86_64-unknown-freebsd13-clang++ -o demo ../obj_s/demo.o -L../lib -lncurses++w -L../lib -lformw -lmenuw -lpanelw -lncursesw -lutil -DHAVE_CONFIG_H -DBUILDING_NCURSES_CXX -I../c++ -I. -I../include -DNDEBUG -O2 -fPIC -DPIC
x86_64-unknown-freebsd13-ld: error: undefined symbol: _Unwind_Resume
>>> referenced by demo.cc
>>> ../obj_s/demo.o:(TestApplication::run())
>>> referenced by demo.cc
>>> ../obj_s/demo.o:(_GLOBAL__sub_I_demo.cc)
>>> referenced by demo.cc
>>> ../obj_s/demo.o:(NCursesUserItem<UserData>::NCursesUserItem(char const*, char const*, UserData const*))
>>> referenced 46 more times
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
This is fixed by undoing the change, adding -lunwind on FreeBSD.
As a first stop towards getting a bit more organized for #171047, add a
maintainer team and add myself and John Ericson as new members. Michael
Raskin asked to be removed.
A second step could be creating a github team additionally.
checkInputs used to be added to nativeBuildInputs. Now we have
nativeCheckInputs to do that instead. Doing this treewide change allows
to keep hashes identical to before the introduction of
nativeCheckInputs.
In https://github.com/NixOS/nixpkgs/pull/211126 I simplified `binutils`
and `libbfd` derivations to follow upstream binutils build system
closer. As a result of `./configure --target=wasm32-unknown-wasi`
`binutils` does not install plugin headers by default.
This causes `pkgsCross.wasi32.llvm_12` (used by `firefox`) to fail the
build as:
[ 81%] Building CXX object tools/gold/CMakeFiles/LLVMgold.dir/gold-plugin.cpp.o
/build/llvm/tools/gold/gold-plugin.cpp:38:10: fatal error:
plugin-api.h: No such file or directory
38 | #include <plugin-api.h>
| ^~~~~~~~~~~~~~
The change accomodates this constraint to disable plugin support for
`wasi` targets.
"llvm-as is an LLVM IR -> LLVM bitcode assembler not a system
assembler"[1], and therefore should not be linked as "as".
The "as" symlink was removed in 46e5ea5af6 ("llvm*: remove symlinks
to llvm-diff, llvm-as and associated LLVM IR utilities."), but that
was partially reverted by b331c72f03 ("llvm: setup some symlinks for
compatibility with binutils"), which restored a bunch of symlinks that
were incorrectly removed, but also incorrectly restored "as". This
was pointed out[2] at the time but apparently never fixed.
[1]: https://github.com/NixOS/nixpkgs/issues/93523#issue-661663683
[2]: b331c72f03 (commitcomment-40981705)
We were missing symlinks for some programs e.g. strings, which caused
e.g. pkgsLLVM.x264 to fail to build.
Here, I have filled in all the symlinks that LLVM would create if
built with the LLVM_INSTALL_BINUTILS_SYMLINKS option. Where an
existing symlink's target has changed, it's to avoid a double
indirection e.g. strip -> llvm-strip -> llvm-objcopy has because just
strip -> llvm-objcopy.
There's also the related problem that we are creating a as -> llvm-as
symlink, which doesn't make sense, but I'll remove that in a
subsequent commit so that if it somehow breaks something it's easy to
revert just that change.
Fixes: https://github.com/NixOS/nixpkgs/pull/210983
Done with the help of https://github.com/Mindavi/nixpkgs-mark-broken
Tool is still WIP but this is one of the first results.
I manually audited the results and removed some results that were not valid.
Note that some of these packages maybe should have more constrained platforms set
instead of broken set, but I think not being perfectly correct is better than
just keep trying to build all these things and never succeeding.
Some observations:
- Some darwin builds require XCode tools
- aarch64-linux builds sometimes suffer from using gcc9
- gcc9 is getting older and misses some new libraries/features
- Sometimes tools try to do system detection or expect some explicit settings for
platforms that are not x86_64-linux
hip: use finalAttrs
rocm-llvm: use finalAttrs
clang-ocl: use finalAttrs
composable_kernel: use finalAttrs
hipcub: use finalAttrs
hipsparse: use finalAttrs
miopengemm: use finalAttrs
rccl: use finalAttrs
rocblas: use finalAttrs
rocclr: use finalAttrs
rocfft: use finalAttrs
rocm-comgr: use finalAttrs
rocm-device-libs: use finalAttrs
rocm-opencl-runtime: use finalAttrs
rocm-runtime: use finalAttrs
rocm-thunk: use finalAttrs
rocmlir: use finalAttrs
rocprim: use finalAttrs
rocsparse: use finalAttrs
rocthrust: use finalAttrs
rocm-cmake: use finalAttrs
rocminfo: use finalAttrs
rocm-smi: use finalAttrs
rocclr: change maintainers to rocm team + old
rocm-comgr: change maintainers to rocm team + old
rocm-device-libs: change maintainers to rocm team + old
rocm-opencl-runtime: change maintainers to rocm team + old
rocm-runtime: change maintainers to rocm team + old
rocm-thunk: change maintainers to rocm team + old
rocminfo: change maintainers to rocm team + old
rocm-smi: change maintainers to rocm team + old
rocm-llvm: change maintainers to rocm team + old
hip: change maintainers to rocm team + old
Same adjustment as made for libc++abi in #185766, for the same reason:
the unamended dylib links to the libc++abi in the build stdenv, which
is the wrong version.
Tested on Darwin with LLVM 14 stdenv, but the phase is added to all
versions, including 11 - so this will cause a mass rebuild.
See: https://github.com/NixOS/nixpkgs/pull/185766
This reverts commit 2000ae5429.
This was intended as a fix for i686 cross compilation, but it has had
other unforeseen consequences, such as breaking rust-bindgen.
Always set `SRCTOP`, set it with abs path
llvmPackages: Bump minimum version for FreeBSD
llvmPackages_*, libgcc, compiler_rt: Hack in enough libs that one can compiler C
freebsd.compat: Rename some things to work around cc-wrapper change
0bea4a194f / #191724 in particular
Since the progress bar results in large output that is broken due to the use of
ncurses and we already use the flag that gives verbose output on test failures,
let's just disable the progress bar.
lit (LLVM Integrated Tester) [0] by default uses as many threads as the build host
has cores, ignoring the user's core settings for the build.
This passes the configured core count on to lit, along with LLVM's default
settings for it which we otherwise override in the process [1].
[0]: https://www.llvm.org/docs/CommandGuide/lit.html
[1]: 329fda39c5/llvm/CMakeLists.txt (L559-L565)
The missing xcrun meant builtins were missing from darwin. This
apparently wasn't an issue until now, but is in projects using
`@available` checks. (The ARM64 hack was apparently the previous
solution to fixing broken SDK detection.)
The new LLVM commit is just before the LLVM 15 fork off.
The readme describing upgrade process so it is easier for others to do.
Co-Authored-By: Dylan Green <Dylan.Green@Obsidian.Systems>
Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
This is the supported way rocm is tested.
It makes packaging in nix a *lot* easier (see the code size).
An important change is the dontLink detection in the clang/clang++
wrapper script: When compiling with --cuda-device-only,
the linker must not be set, otherwise e.g. the blender kernels fail to
compile.
It is possible to both be bare metal and have a libc (newlib).
This libc doesn't provide very much --- not enough for CMake to think
the C toolchain works. We therefore adjust our logic so we hit the "bare
metal" case with or without libc.
The "use LLVM" bootstrap is intentionally not affected.
GCC 10 sets -fno-common by default. This broke some packages, so
when moving to GCC 10 we initially disabled this behavior. This
commit reverts that, bringing us closer to the standard and
upstream.
Co-authored-by: Sergei Trofimovich <slyich@gmail.com>
Reverts #162607 / 1748887ff2.
Reason for revert: This change caused llvm-config{,-native} to be unable
to find static archives bundled with LLVM, as has been [reported]. Ever
since #152944 using moveToOutput in LLVM is _evil_ because llvm-config
obtains it knowledge about the installation locations from the CMake
configure step.
Consequently a change like #162607 will need to be implemented by making
LLVM itself install the static archives to the correct location or by
adding yet another patch which updates llvm-config's knowledge of the
location. The latter is not desireable in my opinion, though, since it
is just asking for this sort of trouble: Before #152944 we had an
outputs.patch that did this sort of things which broke spectacularly in
edge cases.
Fixes#148117.
[reported]: https://github.com/NixOS/nixpkgs/issues/148117#issuecomment-1158245576
