The old stdenv adapters were subtly wrong in two ways:
- `overrideAttrs` leaked the original, unoverridden `mkDerivation`.
- `stdenv.override` would throw away any manually-set `mkDerivation`
from a stdenv reverting to the original.
Now, `mkDerivation` is controlled (nearly directly) via an argument, and
always correctly closes over the final ("self") stdenv. This means the
adapters can work entirely via `.override` without any manual `stdenv //
...`, and both those issues are fixed.
Note hashes are changed, because stdenvs no previously overridden like
`stdenvNoCC` and `crossLibcStdenv` now are. I had to add some
`dontDisableStatic = true` accordingly. The flip side however is that
since the overrides compose, we no longer need to override anything but
the default `stdenv` from which all the others are created.
When we "fix" libtool, we empty out its system library path to avoid
it discovering libraries in e.g. /usr when the sandbox is disabled.
But this also means that the checks libtool does to make sure it can
find the libraries its supposed to be linking to won't work. On Linux
and Darwin, this isn't a problem, because libtool doesn't actually
perform any checks, but it is on at least NetBSD and Cygwin[1].
So, we force libtool not to do these checks on any platform, bringing
the more exotic platforms into line with the existing behaviour on
Linux and Darwin.
Without this change, lots of library packages produce warnings like
this in their build output on the platforms with checks by default:
*** Warning: linker path does not have real file for library -lz.
*** I have the capability to make that library automatically link in when
*** you link to this library. But I can only do this if you have a
*** shared version of the library, which you do not appear to have
*** because I did check the linker path looking for a file starting
*** with libz but no candidates were found. (...for regex pattern test)
*** The inter-library dependencies that have been dropped here will be
*** automatically added whenever a program is linked with this library
*** or is declared to -dlopen it.
And dependent packages break because libtool doesn't link their
transitive dependencies. So making this change fixes _lots_ of
packages on those platforms.
[1]: https://git.savannah.gnu.org/cgit/libtool.git/tree/m4/libtool.m4?id=544fc0e2c2a03129a540aebef41ad32bfb5c06b8#n3445
At least for now. Such changes are risky (we have very many packages),
and apparently it needs more testing/review without blocking other
changes.
This reverts the whole range 4d0e3984918^..8752c327377,
except for one commit that got reverted in 6f239d7309 already.
(that MR didn't even get its merge commit)
... for x86_64-darwin (into staging-next)
It wouldn't bootstrap otherwise.
Unfortunately we still haven't managed to get the tarballs
on the proper URLs, but GitHub should be reliable enough
and surely almost noone will bootstrap themselves anyway.
This is not the "correct" way to check if a variable is non null in
bash. There is already an instance of the "right" way to do it in
setup.sh. Bash is "generous" enough to accept the original input though.
I couldn't find the relevant shellcheck.
This confused the hell out of me, as I didn't spot the
> The following flags are disabled by default ...
when reading about `pie`, because that sentence was hidden in the
previous hardening flag's section.
Also explain that `pie` hardening is on by default on musl.
Only bash 4+ works in setup.sh. To make sure this is obvious, we can
check BASH_VERSINFO to get the major version number of Bash.
While Bash 3 is pretty rare, it still comes stock in macOS.
We *could* provide a warning here for non-Bash shells, but it’s not
always clear whether they will work or not. Zsh should have no trouble
while busybox sh, fish, or any others. There’s no great way to detect
what feature set the shell supports.
Fixes#71625
This enables the bootstrap stdenv test to specify the actual llvm
of the newly generated build instread of assuming it's the same version
as the current stdenv.
With removeUnknownConfigureFlags, it's impossible to express a package
that needs --enable-static, but will not accept --disable-shared,
without overriding the result of removeUnknownConfigureFlags _again_
in pkgs/top-level/static.nix.
It would be much better (and more in line with the rest of Nixpkgs) if
we encoded changes needed for static builds in package definitions
themselves, rather than in an ever-expanding list in static.nix. This
is especially true when doing it in static.nix is going to require
multiple overrides to express what could be expressed with stdenv
options.
So as a step in that direction, and to fix the problem described
above, here I replace removeUnknownConfigureFlags with a new stdenv
option, dontAddStaticConfigureFlags. With this mechanism, a package
that needs one but not both of the flags just needs to set
dontAddStaticConfigureFlags and then set up configureFlags manually
based on stdenv.hostPlatform.isStatic.
Changes to llvmPackages have caused the `libclang-cpp*.dylib` files to
be included in the `clang-unwrapped.lib` output. So we no longer need to
copy them from libclang.
`TargetConditionals.h` was missing several definitions, like
`TARGET_OS_TV` that are part of SDK 10.12 at least. And one that doesn't
seem to occur in any SDK afaict, `TARGET_OS_EMBEDDED_OTHER`.
I added the definitions from SDK 10.12 verbatim and defined
`TARGET_OS_EMBEDDED_OTHER` to be equal to `0`.
This is a modified version of a patch to avoid a stdenv rebuild.
I was having a hard time testing new bootstrapFiles because
`make-bootstrap-tools.nix` imports `pkgspath` but does not pass anything
but the current system.
This is merely for convenience and I'm not entirely certain it's a
sensible thing to do, maybe generating new bootstrapFiles while
overriding the current bootstrapFiles isn't something you're supposed to
do?
The rpath structure for the bootstrap tools was reworked to minimize
the amount of rewriting required on unpack, but the test was not
updated to match the different structure.
Additionally [1] builds that use the bootstrap version of libc++
cannot find libc++abi if the reference includes the "lib"
component (ie, libc++ refers to libc++abi with
@rpath/lib/libc++abi.dylib).
[1] https://logs.nix.samueldr.com/nix-darwin/2021-05-18#4993282
Test failure observed on Hydra: https://hydra.nixos.org/build/143130126
This will begin the process of breaking up the `useLLVM` monolith. That
is good in general, but I hope will be good for NetBSD and Darwin in
particular.
Co-authored-by: sterni <sternenseemann@systemli.org>
If things build fine with `stdenvNoCC`, let them use that. If tools
might be prefixed, prepare for that, either by directly splicing or just
using the env vars provided by the wrapper setup-hooks.
Co-authored-by: Dmitry Kalinkin <dmitry.kalinkin@gmail.com>
I am taking the non-invasive parts of #110914 to hopefully help out with #111988.
In particular:
- Use `lib.makeScopeWithSplicing` to make the `darwin` package set have
a proper `callPackage`.
- Adjust Darwin `stdenv`'s overlays keeping things from the previous
stage to not stick around too much.
- Expose `binutilsNoLibc` / `darwin.binutilsNoLibc` to hopefully get us
closer to a unified LLVM and GCC bootstrap.
Also begin to start work on cross compilation, though that will have to
be finished later.
The patches are based on the first version of
https://reviews.llvm.org/D99484. It's very annoying to do the
back-porting but the review has uncovered nothing super major so I'm
fine sticking with what I've got.
Beyond making the outputs work, I also strove to re-sync the packages,
as they have been drifting pointlessly apart for some time.
----
Other misc notes, highly incomplete
- lvm-config-native and llvm-config are put in `dev` because they are
tools just for build time.
- Clang no longer has an lld dep. That was introduced in
db29857eb3, but if clang needs help
finding lld when it is used we should just pass it flags / put in the
resource dir. Providing it at build time increases critical path
length for no good reason.
----
A note on `nativeCC`:
`stdenv` takes tools from the previous stage, so:
1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.stdenv.cc`: `(?0, ?1, x)`
while:
1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.targetPackages`: `(x, x, ?2)`
3. `pkgsBuildBuild.targetPackages.stdenv.cc`: `(?1, x, x)`
Patch every `derivation` call in the bootsrap process to add it a
conditional `__contentAddressed` parameter.
That way, passing `contentAddressedByDefault` means that the entire
build closure of a system can be content addressed
Adding the hostSuffix to the end of the derivation's name is problematic
since some stuff, including user facing programs like nix-env rely on
the behavior of parseDrvName instead of pname and version.
builtins.parseDrvName currently thinks that the cross compilation target
added via hostSuffix is part of the version. This has the practical
consequence for example that nix-env would think a cross compiled
derivation would be an updated version of a native derivation of the
same package and version — breaking user's profiles.
We can easily prevent this by moving the hostSuffix in between pname and
version. In case name is passed to mkDerivation this is of course not
possible and we are forced to fall back to the old behavior.
This change could serve as a replacement for the migitation we
introduced with the -static appendix to pname in order to avoid
confusion between nix and nixStatic as outlined in the comment added
with this commit.
Support `mainProgram` as an attribute of `meta` for packages.
This is an attribute used by [`nix
run`](https://nixos.org/manual/nix/unstable/command-ref/new-cli/nix3-run.html#description)
to customize the main program of a package.
For example, `pkgs.neovim` provides a `/bin/nvim` executable which users
would (almost certainly) prefer `nix run` to execute instead of a
non-existing `/bin/neovim`.
Signed-off-by: Ana Hobden <operator@hoverbear.org>
`hasCC` was getting overridden in the cross bootstrapping (for GHCJS),
which preventing the default logic from re-triggering for `stdenvNoCC`.
Also remove `stdenv.noCC` which is obseleted by `stdenv.hasCC`.
Add a config field `contentAddressedByDefault` and an associated
environment variable `NIXPKGS_CA_BY_DEFAULT` to make every nixpkgs
derivation content-addressed by default
This was removed in e29b0da9c7, because
it was felt it was ambiguous whether isBSD should remove Darwin.
I think it should be reintroduced. Packages sometimes have their own
concepts of "is BSD" e.g. Lua, and these almost never include Darwin,
so let's keep Darwin excluded.
Without a way to say "is this BSD", one has to list all flavours of
BSD seperately, even though fundamentally they're still extremely
similar. I don't want to have to write the following!
stdenv.isFreeBSD || stdenv.isNetBSD || stdenv.isOpenBSD || stdenv.isDragonFlyBSD
Additionally, we've had stdenv.hostPlatform.isBSD this whole time, and
it hasn't hurt anything.
Unify the logic for constructing the name from pname and version and
modifying the name in case a host suffix needs to appended. This allows
us to modify the construction of name from pname and version without
having to duplicate it in two places.
4e9dc46dea re-enabled hardening for Musl,
which is good.
Though static builds for ARM fail in various ways
- cross armv7l static does not build
- cross aarch64 static produces segfaulting dynamically linked binaries
- native aarch64 static also produces segfaulting dynamically linked binaries
It seems that for native x86_64-linux, static builds are fine though.
This works around the issue by removing PIE from the hardening flags,
keeping all other hardening flags. This is an improvement (I think) from
before 4e9dc46d.
Fixes#114953
* stdenv/check-meta: change to allowlist and blocklist
* Update pkgs/stdenv/generic/check-meta.nix
Co-authored-by: Graham Christensen <graham@grahamc.com>
Since the deprecation is fairly recent, we should warn by default.
Also fix the wording of the comment: stdenv.lib will be removed for the 21.11
release, not just deprecated (as it already is deprecated).
The `platform` field is pointless nesting: it's just stuff that happens
to be defined together, and that should be an implementation detail.
This instead makes `linux-kernel` and `gcc` top level fields in platform
configs. They join `rustc` there [all are optional], which was put there
and not in `platform` in anticipation of a change like this.
`linux-kernel.arch` in particular also becomes `linuxArch`, to match the
other `*Arch`es.
The next step after is this to combine the *specific* machines from
`lib.systems.platforms` with `lib.systems.examples`, keeping just the
"multiplatform" ones for defaulting.
Increase schedulingPriority of the bootstrap tools to unblock the
nixpkgs-unstable channel.
The channel is repeatedly blocked by the makeBootstrapTools job for
aarch64. The cause is lack of resources.
By increasing the priority, it should become the first job Hydra would
build, allowing the channel to advance quicker. Of course, it does mean
that while the channel advances, nothing else has been built.
This should be a temporary solution until we have more capacity for
aarch64.
