with structuredAttrs lists will be bash arrays which cannot be exported
which will be a issue with some patches and some wrappers like cc-wrapper
this makes it clearer that NIX_CFLAGS_COMPILE must be a string as lists
in env cause a eval failure
I got the plugin API support at least once incorrect. Instead of
copying the deifnition let's consolidate it within binutils itself.
While at it forward-ported changes to llvm_{13,14,15}.
The change is a no-op from rebuild perspective.
this is in preparation for the next commit which exposes the release
information and monorepo source as overridable args (which makes it
easier for users to use their own LLVM in nixpkgs)
this commit only adds a check in the `llvm` package's postConfigure that
makes sure the LLVM source provided matches the version we were given;
the actual machinery (functionally just a cosmetic change; causes no
rebuilds) is in the next commit
The two scenarios described within where splicing doesn't handle
selecting the right package for us are observable in the following
(nix repl session):
```
> np = import <nixpkgs> { system = "x86_64-linux"; crossSystem = { config = "aarch64-linux"; }; }
> np.__splicedPackages.hello ? __spliced
true
> np.__splicedPackages.python3Packages.psutil ? __spliced
true
> np.__splicedPackages.python3.pkgs.psutil ? __spliced
false
> (np.__splicedPackages.python3.withPackages (ps: with ps; [psutil])) ? __spliced
false
```
See: #211340
Details within but ultimately there isn't a satisfying resolution for
any of the three test failures we were seeing and all three deserve
further exploration.
For the `sw_vers` macOS version issue in particular, it's possible to
observe the nixpkgs provided `CoreFoundation` vs system `CoreFoundation`
for `x86_64` and `aarch64` like so (on a host running macOS `13.0.1`):
```console
$ nix-shell -p darwin.DarwinTools --system aarch64-darwin --command "sw_vers"
ProductName: macOS
ProductVersion: 13.0.1
BuildVersion: 22A400
$ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command "sw_vers"
ProductName: Mac OS X
ProductVersion: 10.16
BuildVersion: 22A400
```
Where `/System/Library/CoreServices/SystemVersion.plist` has:
```console
$ cat /System/Library/CoreServices/SystemVersion.plist | grep ProductVersion
-A 1
<key>ProductVersion</key>
<string>13.0.1</string>
```
Further:
```console
$ nix-shell -p darwin.DarwinTools --system aarch64-darwin --command 'otool -L $(which sw_vers)'
/nix/store/nb2q33ak2zif49ndcpc6m823z0vhmy8y-DarwinTools-1/bin/sw_vers:
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1770.255.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)
$ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command 'otool -L $(which sw_vers)'
/nix/store/88v4kjvgwl71byfpvd0baviiq7l5appc-DarwinTools-1/bin/sw_vers:
@rpath/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1454.90.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1238.60.2)
```
For the `x86_64` `sw_vers` binary we can see rpath:
```console
$ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command 'otool -l $(which sw_vers)' | grep LC_RPATH -A 2 -B 1
Load command 13
cmd LC_RPATH
cmdsize 120
path /nix/store/zvr4wypbgskhhw9cawfn7mmxfa75nh8f-swift-corefoundation-unstable-2018-09-14/Library/Frameworks (offset 12)
```
And we can confirm that the nixpkgs provided `CoreFoundation` is what
ultimately gets loaded:
```console
$ nix-shell -p darwin.DarwinTools --system x86_64-darwin --command 'DYLD_PRINT_LIBRARIES=1 sw_vers'
dyld[16215]: <no uuid> /nix/store/88v4kjvgwl71byfpvd0baviiq7l5appc-DarwinTools-1/bin/sw_vers
dyld[16215]: <no uuid> /nix/store/zvr4wypbgskhhw9cawfn7mmxfa75nh8f-swift-corefoundation-unstable-2018-09-14/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
dyld[16215]: <no uuid> /nix/store/xd2a4xh8kdwq0j67hzgw720npdw5hzkk-ICU-66108/lib/libicucore.A.dylib
<snipped>
```
```bash
nix-diff \
$(nix path-info nixpkgs#legacyPackages.aarch64-darwin.darwin.DarwinTools --derivation) \
$(nix path-info nixpkgs#legacyPackages.x86_64-darwin.darwin.DarwinTools --derivation)
```
doesn't show any _obvious_ discrepancies
there are a few parts to this:
- adding darwin specific check deps
- working around referencing LLVM dylibs during the checkPhase in a
way that supports darwin
+ previously we just set `$LD_LIBRARY_PATH` and/or made some
strategic symlinks
+ now we have LLVM's `lit` config set the appropriate env vars as
needed (as is done for other LLVM subprojects)
+ in retrospect switching to `installCheckPhase` might have been the
better move..
- patching `lit` to deal with `$DYLD_LIBRARY_PATH` being purged for
new "protected" processes
more details within.
as detailed within, adding `asm/ptrace.h` leads to `asm/ptrace-abi.h`
being included which defines preprocessor symbols that clash with
identifiers used in the LLVM headers (`FS` and `CS` only defined on
i686)
This restores this check to what it originally was in #196909 (see:
https://github.com/NixOS/nixpkgs/pull/196909#discussion_r1000536298) and
lets `compiler-rt` eval successfully when trying to compile the
`llvmPackages_15` set for mingw targets (i.e. a platform that *is* GNU
but does *not* use glibc).
---
It's not clear to me what the `haveLibc` check is doing here (platforms
that seem to use glibc like `x86_64-linux` and have
`plat.libc == "glibc"` have `haveLibc = false` because `stdenv.cc.libc`
is `null`).
libc++ has switched to using `__attribute__((using_if_exists))` to handle
incomplete libc implementations; see: a9c9183ca4
These essentially require a modern C++ compiler (clang gained support in
LLVM 13: 369c648399,
gcc appears to not have support yet: https://gcc.gnu.org/bugzilla//show_bug.cgi?id=105584).
Previously this was not an issue for us (despite the transition happening
around LLVM 13) but something about the changes to the libc++/libc++-abi
build has made it so that on platforms with incomplete libc impls (i.e.
Darwin is missing `quick_exit`/`at_quick_exit`) we error during the `libcxx-abi`
build when the stdenv's (older, not supporting `using_if_exists`) compiler
tries to import libc symbols that aren't present.
The libc++ docs suggest we use a modern compiler to build libc++ anyways
(https://releases.llvm.org/15.0.0/projects/libcxx/docs/index.html#platform-and-compiler-support)
so this commit uses stdenv's containing the package set's clang to build
libcxx/libcxx-abi.
This is similar to how libc++ bootstrapping builds (https://releases.llvm.org/15.0.0/projects/libcxx/docs/BuildingLibcxx.html#bootstrapping-build)
work.
None of the patches required any touch-up; the only change of note is:
- due to changes in the libc++/libc++abi build
(https://reviews.llvm.org/D120719 and https://reviews.llvm.org/D131037)
we have to add an extra build option to the libc++ header only
build that sidesteps bits of the libc++ build config that assume
libc++-abi is present in the build:
4f827318e3/libcxx/src/CMakeLists.txt (L255-L256)
Rather than maintaining a precise set of build options that let us dodge
referencing libc++-abi variables in the libc++ header only build, we set
`LIBCXX_CXX_ABI` to `none`, as suggested by @lovesegfault.
More discussion about this here: https://github.com/NixOS/nixpkgs/pull/194634#discussion_r990267037
Co-authored-by: Bernardo Meurer <bernardo@meurer.org>
FreeBSD doesn't use LLVM's cxxabi implementation, for backwards
compatibility reasons. Software expects the libcxxrt API when
building on FreeBSD. This fixes the build of
pkgsCross.x86_64-freebsd.boost.
The exception for FreeBSD was added in 0afe9d1f70 ("freebsd packages:
Init at 13.1"), but it seems to have been erroneous, as e.g. ncurses
fails to build:
x86_64-unknown-freebsd13-clang++ -o demo ../obj_s/demo.o -L../lib -lncurses++w -L../lib -lformw -lmenuw -lpanelw -lncursesw -lutil -DHAVE_CONFIG_H -DBUILDING_NCURSES_CXX -I../c++ -I. -I../include -DNDEBUG -O2 -fPIC -DPIC
x86_64-unknown-freebsd13-ld: error: undefined symbol: _Unwind_Resume
>>> referenced by demo.cc
>>> ../obj_s/demo.o:(TestApplication::run())
>>> referenced by demo.cc
>>> ../obj_s/demo.o:(_GLOBAL__sub_I_demo.cc)
>>> referenced by demo.cc
>>> ../obj_s/demo.o:(NCursesUserItem<UserData>::NCursesUserItem(char const*, char const*, UserData const*))
>>> referenced 46 more times
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
This is fixed by undoing the change, adding -lunwind on FreeBSD.
As a first stop towards getting a bit more organized for #171047, add a
maintainer team and add myself and John Ericson as new members. Michael
Raskin asked to be removed.
A second step could be creating a github team additionally.
checkInputs used to be added to nativeBuildInputs. Now we have
nativeCheckInputs to do that instead. Doing this treewide change allows
to keep hashes identical to before the introduction of
nativeCheckInputs.
In https://github.com/NixOS/nixpkgs/pull/211126 I simplified `binutils`
and `libbfd` derivations to follow upstream binutils build system
closer. As a result of `./configure --target=wasm32-unknown-wasi`
`binutils` does not install plugin headers by default.
This causes `pkgsCross.wasi32.llvm_12` (used by `firefox`) to fail the
build as:
[ 81%] Building CXX object tools/gold/CMakeFiles/LLVMgold.dir/gold-plugin.cpp.o
/build/llvm/tools/gold/gold-plugin.cpp:38:10: fatal error:
plugin-api.h: No such file or directory
38 | #include <plugin-api.h>
| ^~~~~~~~~~~~~~
The change accomodates this constraint to disable plugin support for
`wasi` targets.
"llvm-as is an LLVM IR -> LLVM bitcode assembler not a system
assembler"[1], and therefore should not be linked as "as".
The "as" symlink was removed in 46e5ea5af6 ("llvm*: remove symlinks
to llvm-diff, llvm-as and associated LLVM IR utilities."), but that
was partially reverted by b331c72f03 ("llvm: setup some symlinks for
compatibility with binutils"), which restored a bunch of symlinks that
were incorrectly removed, but also incorrectly restored "as". This
was pointed out[2] at the time but apparently never fixed.
[1]: https://github.com/NixOS/nixpkgs/issues/93523#issue-661663683
[2]: b331c72f03 (commitcomment-40981705)
We were missing symlinks for some programs e.g. strings, which caused
e.g. pkgsLLVM.x264 to fail to build.
Here, I have filled in all the symlinks that LLVM would create if
built with the LLVM_INSTALL_BINUTILS_SYMLINKS option. Where an
existing symlink's target has changed, it's to avoid a double
indirection e.g. strip -> llvm-strip -> llvm-objcopy has because just
strip -> llvm-objcopy.
There's also the related problem that we are creating a as -> llvm-as
symlink, which doesn't make sense, but I'll remove that in a
subsequent commit so that if it somehow breaks something it's easy to
revert just that change.
Fixes: https://github.com/NixOS/nixpkgs/pull/210983
Done with the help of https://github.com/Mindavi/nixpkgs-mark-broken
Tool is still WIP but this is one of the first results.
I manually audited the results and removed some results that were not valid.
Note that some of these packages maybe should have more constrained platforms set
instead of broken set, but I think not being perfectly correct is better than
just keep trying to build all these things and never succeeding.
Some observations:
- Some darwin builds require XCode tools
- aarch64-linux builds sometimes suffer from using gcc9
- gcc9 is getting older and misses some new libraries/features
- Sometimes tools try to do system detection or expect some explicit settings for
platforms that are not x86_64-linux
hip: use finalAttrs
rocm-llvm: use finalAttrs
clang-ocl: use finalAttrs
composable_kernel: use finalAttrs
hipcub: use finalAttrs
hipsparse: use finalAttrs
miopengemm: use finalAttrs
rccl: use finalAttrs
rocblas: use finalAttrs
rocclr: use finalAttrs
rocfft: use finalAttrs
rocm-comgr: use finalAttrs
rocm-device-libs: use finalAttrs
rocm-opencl-runtime: use finalAttrs
rocm-runtime: use finalAttrs
rocm-thunk: use finalAttrs
rocmlir: use finalAttrs
rocprim: use finalAttrs
rocsparse: use finalAttrs
rocthrust: use finalAttrs
rocm-cmake: use finalAttrs
rocminfo: use finalAttrs
rocm-smi: use finalAttrs
rocclr: change maintainers to rocm team + old
rocm-comgr: change maintainers to rocm team + old
rocm-device-libs: change maintainers to rocm team + old
rocm-opencl-runtime: change maintainers to rocm team + old
rocm-runtime: change maintainers to rocm team + old
rocm-thunk: change maintainers to rocm team + old
rocminfo: change maintainers to rocm team + old
rocm-smi: change maintainers to rocm team + old
rocm-llvm: change maintainers to rocm team + old
hip: change maintainers to rocm team + old
Same adjustment as made for libc++abi in #185766, for the same reason:
the unamended dylib links to the libc++abi in the build stdenv, which
is the wrong version.
Tested on Darwin with LLVM 14 stdenv, but the phase is added to all
versions, including 11 - so this will cause a mass rebuild.
See: https://github.com/NixOS/nixpkgs/pull/185766
This reverts commit 2000ae5429.
This was intended as a fix for i686 cross compilation, but it has had
other unforeseen consequences, such as breaking rust-bindgen.
Always set `SRCTOP`, set it with abs path
llvmPackages: Bump minimum version for FreeBSD
llvmPackages_*, libgcc, compiler_rt: Hack in enough libs that one can compiler C
freebsd.compat: Rename some things to work around cc-wrapper change
0bea4a194f / #191724 in particular
Since the progress bar results in large output that is broken due to the use of
ncurses and we already use the flag that gives verbose output on test failures,
let's just disable the progress bar.
lit (LLVM Integrated Tester) [0] by default uses as many threads as the build host
has cores, ignoring the user's core settings for the build.
This passes the configured core count on to lit, along with LLVM's default
settings for it which we otherwise override in the process [1].
[0]: https://www.llvm.org/docs/CommandGuide/lit.html
[1]: 329fda39c5/llvm/CMakeLists.txt (L559-L565)
The missing xcrun meant builtins were missing from darwin. This
apparently wasn't an issue until now, but is in projects using
`@available` checks. (The ARM64 hack was apparently the previous
solution to fixing broken SDK detection.)
The new LLVM commit is just before the LLVM 15 fork off.
The readme describing upgrade process so it is easier for others to do.
Co-Authored-By: Dylan Green <Dylan.Green@Obsidian.Systems>
Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>
This is the supported way rocm is tested.
It makes packaging in nix a *lot* easier (see the code size).
An important change is the dontLink detection in the clang/clang++
wrapper script: When compiling with --cuda-device-only,
the linker must not be set, otherwise e.g. the blender kernels fail to
compile.
It is possible to both be bare metal and have a libc (newlib).
This libc doesn't provide very much --- not enough for CMake to think
the C toolchain works. We therefore adjust our logic so we hit the "bare
metal" case with or without libc.
The "use LLVM" bootstrap is intentionally not affected.
GCC 10 sets -fno-common by default. This broke some packages, so
when moving to GCC 10 we initially disabled this behavior. This
commit reverts that, bringing us closer to the standard and
upstream.
Co-authored-by: Sergei Trofimovich <slyich@gmail.com>
Reverts #162607 / 1748887ff2.
Reason for revert: This change caused llvm-config{,-native} to be unable
to find static archives bundled with LLVM, as has been [reported]. Ever
since #152944 using moveToOutput in LLVM is _evil_ because llvm-config
obtains it knowledge about the installation locations from the CMake
configure step.
Consequently a change like #162607 will need to be implemented by making
LLVM itself install the static archives to the correct location or by
adding yet another patch which updates llvm-config's knowledge of the
location. The latter is not desireable in my opinion, though, since it
is just asking for this sort of trouble: Before #152944 we had an
outputs.patch that did this sort of things which broke spectacularly in
edge cases.
Fixes#148117.
[reported]: https://github.com/NixOS/nixpkgs/issues/148117#issuecomment-1158245576
Fixes#166833.
The build creates a symlink for this assuming it's present,
so be sure it's there when filtering the source for clang.
Alternatively we could use LLVM_EXTERNAL_CLANG_TOOLS_EXTRA_SOURCE_DIR.
lld package provides an unwrapped lld. It doesn't always work on NixOS
(eg, it doesn't set rpath), and so dosen't always work.
What one should be using instead is the `lld` from
`llvmPackages.bintools` package. This super counterintutive.
One incremental step we can take here is to clarify that the `lld`
package is unrwapped -- right now, it looks like 100% legit thing one
should be using!
