The new Darwin SDK pattern relies on an effectively empty, stub libc
implementation. The actual libSystem to be linked is located dynamically
based on the active SDK for the target. Independent build, host, and
target SDKs are all supported by Darwin.
The stub libSystem contains empty `include` and `lib` folders to avoid
warnings from wrappers that add those paths unconditionally, which can
turn into errors when a package is building with warnings-as-errors.
While it would be nice if a fallback libc could be provided, SDK headers
are not compatible between framework versions. Providing a fallback
risks mixing headers from different frameworks, which can result in hard
to diagnose errors involving semicolons or other punctuation.
Packages propagated by the SDK need to use a stdenv that does not
propagate anything. Otherwise, an infinite recursion will result when
building those packages.
For consistency, all source releases should use the bootstrapStdenv.
Update darwin.binutils based on darwin.cctools-llvm. It works essentially in the same way, simplifying the derivation. The support for providing both assemblers is handled in a separate commit.
Additionally, unlike cctools-llvm, the lists of tools are provided as passthru attributes: `cctools_cmds` and `llvm_cmds`.
- Drop the patches in favor of patching up the xcodeproj files, which
should make updates in the future easier (no more patch conflicts);
- Switch to building `MoltenVKPackaging.xcodeproj` instead of building
the projects individually;
- Link `libMoltenVK.dylib` manually, which is needed for MoltenVK 1.2.8
due to xcbuild not being able to build the dylib in the xcodeproj;
- Add support for enabling private API usage and default it to `true`.
This will be a new feature in MoltenVK 1.2.8;
- Use darwin.apple_sdk.libs.simd instead of symlinking from the SDK;
- Filter out rc and beta releases in the update script; and
- Support static builds of MoltenVK.
This patch switches the CoreFoundation on x86_64-darwin from the open
source swift-corelibs-foundation (CF) to the system CoreFoundation.
This change was motivated by failures building packages for the current
staging-next cycle #263535 due to an apparent incompatibility with the
rpath-based approach to choosing CF or CoreFoundation and macOS 14. This
error often manifests as a crash with an Illegal Instruction.
For example, building aws-sdk-cpp for building Nix will fail this way.
https://hydra.nixos.org/build/239459417/nixlog/1
Application Specific Information:
CF objects must have a non-zero isa
Error Formulating Crash Report:
PC register does not match crashing frame (0x0 vs 0x7FF8094DD640)
Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0 CoreFoundation 0x7ff8094dd640 CF_IS_OBJC.cold.1 + 14
1 CoreFoundation 0x7ff8094501d0 CF_IS_OBJC + 60
2 CoreFoundation 0x7ff8093155e8 CFRelease + 40
3 ??? 0x10c7a2c61 s_aws_secure_transport_ctx_destroy + 65
4 ??? 0x10c87ba32 aws_ref_count_release + 34
5 ??? 0x10c7b7adb aws_tls_connection_options_clean_up + 27
6 ??? 0x10c596db4 Aws::Crt::Io::TlsConnectionOptions::~TlsConnectionOptions() + 20
7 ??? 0x10c2d249c Aws::CleanupCrt() + 92
8 ??? 0x10c2d1ff0 Aws::ShutdownAPI(Aws::SDKOptions const&) + 64
9 ??? 0x102d9bc6f main + 335
10 dyld 0x202f333a6 start + 1942
According to a [post][1] on the Apple developer forums, hardening was
added to CoreFoundation, and this particular message occurs when you
attempt to release an object it does not recognize as a valid CF object.
(Thank you to @lilyinstarlight for finding this post).
When I switched aws-sdk-cpp to link against CoreFoundation instead of
CF, the error went away. Somehow both libraries were being used. To
prevent dependent packages from linking the wrong CoreFoundation, it
would need to be added as a propagated build input.
Note that there are other issues related to mixing CF and CoreFoundation
frameworks. #264503 fixes an issue with abseil-cpp where it propagates
CF, causing issues when using a different SDK version. Mixing versions
can also cause crashes with Python when a shared object is loaded that
is linked to the “wrong” CoreFoundation.
`NIX_COREFOUNDATION_RPATH` is supposed to make sure the right
CoreFoundation is being used, but it does not appear to be enough on
macOS 14 (presumably due to the hardening). While it is possible to
propagate CoreFoundation manually, the cleaner solution is to make it
the default. CF remains available as `darwin.swift-corelibs-foundation`.
[1]: https://developer.apple.com/forums/thread/739355
Deeply-curried functions are pretty error-prone in untyped languages
like Nix. This is a particularly bad case because
`top-level/splice.nix` *also* declares a makeScopeWithSplicing, but
it takes *two fewer arguments*.
Let's switch to attrset-passing form, to provide some minimal level
of sanity-checking.
cctools-llvm is a replacement for cctools that replaces as much of cctools with equivalents from LLVM that it can reasonably do. This was motivated by wanting to reduce dependencies on cctools, which are updated infrequently by upstream.
To provide a motivating example, the version of `strip` included in cctools cannot properly strip the archives in compiler-rt in LLVM 15. Paths are left to bootstrap tools, resulting in failed requisites checks in the final stdenv build. Since `strip` needs replaced, the opportunity was taken to replace other provided they are functional replacements.
Note: This has to be done in cctools (or some equivalent) because some derivations (noteably LLVM) use the bintools of the stdenv directly instead of going through the wrapper.
The following tools from LLVM are not used in this derivation:
* LLD - not fully compatible with ld64 yet and potentially too big of a change;
* libtool - not a drop-in replacement yet because it does not support linker passthrough, which is needed by xcbuild;
* lipo - crashes when running the LLVM test suite;
* install_name_tool - fails when trying to build swift-corefoundation; and.
* randlib - not completely a drop-in replacement, so leaving it out for now.
If other incompatabilities are found, the tools can be reverted or made conditional. For example, cctools `strip` is preferred on older versions of LLVM (which lack the compiler-rt issue) or when cctools itself is a new enough version because `llvm-strip` on LLVM 11 produces files that older verions of `codesign_allocate` cannot process correctly.
One final caveat/note: Some tools are not duplicated or linked from cctools-port. The names of the tools and which ones were linked was determined based on what is provided upstream in Xcode and is installed on macOS system.
For reasons explained in the commit contents, in order to build the
native gnat package for x86_64-darwin, the native gnatboot package for
x86_64-darwin must have access to both the Clang integrated assembler
and the cctools GNU assembler for that platform. This commit creates a
package with both of those assemblers that x86_64-darwin gnatboot can
then be wrapped with.
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
- Use the 11.0 SDK instead of the 10.12 one on x86_64-darwin;
- Use `NIX_CFLAGS_COMPILE` and `NIX_LDFLAGS` to pass flags to the
compiler instead of patching the Xcode project files; and
- Use xcbuild to build the project.
- Introduce `preLibcCrossHeaders` to bootstrap libgcc and compiler-rt
the same way.
- Organize LLVM bintools as `bintools{-unwrapped,,NoLibc}` for
consistency with GNU Binutils and Apple's cctools.
- Do Android changes for all `llvmPackages` for consistency.
- Improve the way the default GCC and LLVM versions are selected.
I am taking the non-invasive parts of #110914 to hopefully help out with #111988.
In particular:
- Use `lib.makeScopeWithSplicing` to make the `darwin` package set have
a proper `callPackage`.
- Adjust Darwin `stdenv`'s overlays keeping things from the previous
stage to not stick around too much.
- Expose `binutilsNoLibc` / `darwin.binutilsNoLibc` to hopefully get us
closer to a unified LLVM and GCC bootstrap.
Without this change `as` on `aarch64-darwin` is defunct:
```
$ /nix/store/g11xba8ch20zndaf32hfnap46z47b5k3-cctools-port-949.0.1/bin/as -v
/nix/store/g11xba8ch20zndaf32hfnap46z47b5k3-cctools-port-949.0.1/bin/as: assembler (/nix/store/g11xba8ch20zndaf32hfnap46z47b5k3-cctools-port-949.0.1/libexec/as/arm64/as or /nix/store/g11xba8ch20zndaf32hfnap46z47b5k3-cctools-port-949.0.1/libexec/as/arm64/as) for architecture arm64 not installed
Installed assemblers are:
/nix/store/g11xba8ch20zndaf32hfnap46z47b5k3-cctools-port-949.0.1/libexec/as/ppc64/as for architecture ppc64
/nix/store/g11xba8ch20zndaf32hfnap46z47b5k3-cctools-port-949.0.1/libexec/as/x86_64/as for architecture x86_64
/nix/store/g11xba8ch20zndaf32hfnap46z47b5k3-cctools-port-949.0.1/libexec/as/ppc/as for architecture ppc
/nix/store/g11xba8ch20zndaf32hfnap46z47b5k3-cctools-port-949.0.1/libexec/as/i386/as for architecture i386
/nix/store/g11xba8ch20zndaf32hfnap46z47b5k3-cctools-port-949.0.1/libexec/as/arm/as for architecture arm
```
Apple defaults to Clang in all cases from what I see, but nixpkgs prefers `cctools`: #51881.
The only remaining use-case for cf-private are symbols that are not
available in the opensource build. This generally solved the problem
because of it's setup-hook.
CoreFoundation is included by the stdenv, moving the decision of what
version should be used there makes it possible to override it entirely
rather then prepending flags like cf-private does which can be
unreliable.
Lots of stuff has gotten moved around. Many security libraries have been merged
into the Security monorepo. I’ve cleared them out for now, we will
need to modify Security to build them!
This also moves some things around to more clearly separate
bootstrapping the stdenv from everything else. We want the “normal”
mode to be the non-bootstrapped version. When you ask for “Security”,
you want the actual built software, not a crippled one.
- Add TARGET_OS_OSX to darwin.libSystem. Looks like something
introduced in 10.12. TARGET_OS_MAC is only set when building for
desktop (iOS will have TARGET_OS_MAC set)
- Bump darwin.dtrace
- Bump darwin.libpthread
- Remove SmartCardServices, libsecurity*, etc.
- Install some more headers for darling.
We were previously using a dummy wrapper for dsymutil. This meant that
debug symbols were not getting generated when dsymutil was otherwise
available. This should fix that issue & provide a real dsymutil from
llvm.
Fixes#52148.
Undefined symbols for architecture x86_64:
"_OBJC_CLASS_$_NSMutableArray", referenced from:
objc-class-ref in trash-571f39.o
"_OBJC_CLASS_$_NSURL", referenced from:
objc-class-ref in trash-571f39.o
"_OBJC_CLASS_$_NSUserDefaults", referenced from:
objc-class-ref in trash-571f39.o
objc-class-ref in HGCLIUtils-31f3b3.o
ld: symbol(s) not found for architecture x86_64
