While it would be nice if this could be split, there are too many
changes as part of the cleanup and improvements, including:
- Refactoring all propagated packages into functions that can be used to
ensure that packages are propagated only at the expected stages;
- Using a sanity-checking merge function to ensure that packages are
only propagated by one of the above functions;
- Reducing the number of Python builds during the bootstrap to one;
- Removing the extra sysctl stage;
- Using the LLVM bootstrap to build LLVM, clang, libc++, etc;
- Propagating llvmPackages_<version> in the final stdenv, so that
packages needing that version specifically don’t have to rebuild it;
- Bootstrapping with the new Darwin SDK; and
- Reducing the overall number of paths build during a bootstrap by ~33%.
Using the bootstrap stdenv avoids an infinite recursion from xcbuild
depending on zlib depending on xcrun from xcbuild, which is propagated
by the non-bootstrap stdenv.
macOS ships with several stubs in `/usr/bin` that invoke `xcrun` to run
the tools from the active SDK. When `/usr/bin` is in `PATH`, this will
cause `xcrun` from xcbuild to invoke itself over and over. Filtering
`/usr/bin` from `xcrun`’s search path prevents this from happening.
xcbuild determines the SDK dynamically, so overriding the `sdkVer` no
longer works. If packages want to change the SDK, they need to add one
of the SDK packages to their inputs.
Take advantage of the new Darwin SDKs to dynamically determine SDK
information such as path, version, and binaries (via `xcrun --find`).
This is accomplished by relying on the existance of `DEVELOPER_DIR`,
which the SDK will set up in nixpkgs.
Using the bootstrap stdenv avoids an infinite recursion from xcbuild
depending on ncurses depending on xcrun from xcbuild, which is
propagated by the non-bootstrap stdenv.
When using the LLVM bootstrap to build LLVM and its libraries,
overriding just LLVM to disable tests during the first build doesn’t
seem to work. This stage name should remain stable, so check for it and
disable tests when building in it.
When compiler-rt targets Darwin, it is built with `-target`, which
causes clang to try to invoke `ld` without a target prefix (e.g., it
will try to exec `ld` instead of `x86_64-apple-darwin-ld` on a
cross-build to x86_64-darwin). Specifying `--ld-path` overrides that
behavior, allowing it to find the appropriate cross-linker.
The first build of compiler-rt in the LLVM bootstrap is build without
libc++ being available, which causes support for the `-g` flag to be
detected incorrectly on Darwin. Overriding the check by specifying that
it’s usable allows the first build of compiler-rt to succeed.
compiler-rt supports specifying the SDK path and version, so do that to
avoid needing to include `xcrun` as a native build input, which
simplifies the bootstrap.
Instead of using overrides in the stdenv bootstrap, Darwin will be
relying on the LLVM bootstrap to build compiler-rt. The only special
handling it needs is to use a stdenv with a bootstrap SDK instead of the
default one (to avoid infinite recursions).
While the Darwin stdenv bootstrap sets up its own clang wrappers and
doesn’t provide these wrappers in the final stdenv, it does use them
indirectly via the LLVM bootstrap to build LLVM and its libraries.
Note on using the system libunwind: It is possible to build and use the
LLVM libunwind on Darwin, but using the system by default one ensures
everything is using the same unwinder.
Using the bootstrap stdenv avoids an infinite recursion from xcbuild
depending on libxml2 depending on xcrun from xcbuild, which is
propagated by the non-bootstrap stdenv.
Using the bootstrap stdenv avoids an infinite recursion from xcbuild
depending on libpng depending on xcrun from xcbuild, which is propagated
by the non-bootstrap stdenv.
With this workaround enabled, CMake looks for SDK headers and libraries
in `${SDKROOT}/var/empty`, which ensures it will not find them.
Disabling the workaround in just this case allows CMake’s SDK support to
work with the SDKs in nixpkgs while still not including other, impure
paths in its search paths.
Without this change, all Darwin platforms mangle to the same suffix
salt. That is normally not an issue because build = host should mean a
non-cross build, but it causes issues on Darwin with static builds
because `DEVELOPER_DIR_FOR_BUILD` and `DEVELOPER_DIR` will refer to
different SDKs but mangle to the same `DEVELOPER_DIR` with suffix salt.
The fix is to mangle static builds differently from non-static ones on
Darwin, which allows building for a static Darwin target on a
same-architecture Darwin host. This fix is applied only to Dariwn
because the issue does not appear to affect other platforms.
It is unusual to invoke a wrapped compiler without setting
`DEVELOPER_DIR`, but it can happen when a user adds a compiler to their
installed packages or when a package intentionally invokes the compiler
without an environment (such as the GHC binary packages).
Some compilers may know to check these paths when `SDKROOT` is set, but
it’s not assumed they do. `SDKROOT` is instead derived from the
`DEVELOPER_DIR`, and `NIX_CFLAGS_COMPILE` is set up with the sysroot and
necessary framework and include search paths.
This is a new packaging of the Darwin SDK. Instead of splitting
libraries and frameworks into separate packages, it provides a single
package for the whole SDK.
# Features
- Vendored files are removed from the SDK. There are 50+ different
packages that are vendored by upstream (depending on the version);
- Components that are built in nixpkgs (either from upstream or from the
source releases) are also removed. If they need to be included by
default, they are propagated;
- A single SDK pattern is used to package all SDKs, and scripts are
provided to aid updating the SDK version and its source release
versions. This makes adding new SDKs much easier;
- SDK overrides are handled by adding the SDK version you require. If
multiple SDKs are present, only the newest is used. It is possible to
have different SDKs for each of build, host, and target platforms;
- Private headers are no longer provided by default unless you use the
SDK’s `privateFrameworksHook` to add them. It does the right thing
when multiple SDKs are in your inputs;
- Source releases for the SDK version are available via a passthru
`sourceRelease` function. This is mostly useful for getting private
headers for building source releases in the darwin attrset; and
- The same versions of propagated components are used on both platforms
(e.g., the same libresult, libiconv, etc).
See `pkgs/by-name/ap/apple-sdk/README.md` for details on how the SDK
derivation is structured and how to update it.
libsbuf is required by some of the source release updates that will
be done. Unfortunately, it is only available on macOS 14 and newer, and
there is no source release available currently.
This is a port of libsbuf from FreeBSD, which appears to be the origin
of the header provided in the 14.x SDK. It provides the same ABI as the
system dylib and same API as the the SDK header while being available on
all supported deployment targets in nixpkgs.
Note: This package is not based on libsbuf from the FreeBSD package set
in nixpkgs because: it doesn’t build on Darwin, and using it would pull
many FreeBSD packages into the Darwin bootstrap, which is undesirable.
