I realized what rhelmot did in 61202561d9
(specify what packages just need `stdenvNoLibc`) is definitely the right
approach for this, and adjusted NetBSD and OpenBSD to likewise use it.
With that change, we don't need these confusing and ugly `*bsdCross`
package sets at all!
We can get rid of a lot more libc-related `*Cross`, and I will do so
soon, but this is the first step.
(adapted from commit 51f1ecaa59)
For a long time, we've had `crossLibcStdenv`, `*Cross` libc attributes,
and `*bsdCross` pre-libc package sets. This was always bad because
having "cross" things is "not declarative": the naming doesn't reflect
what packages *need* but rather how we *provide* something. This is
ugly, and creates needless friction between cross and native building.
Now, almost all of these `*Cross` attributes are gone: just these are
kept:
- Glibc's and Musl's are kept, because those packages are widely used
and I didn't want to risk changing the native builds of those at this
time.
- generic `libcCross`, `theadsCross`, and friends, because these relate
to the convolulted GCC bootstrap which still needs to be redone.
The BSD and obscure Linux or freestnanding libcs have conversely all
been made to use a new `stdenvNoLibc`, which is like the old
`crossLibcStdenv` except:
1. It usable for native and cross alike
2. It named according to what it *is* ("a standard environment without
libc but with a C compiler"), rather than some non-compositional
jargon ("the stdenv used for building libc when cross compiling",
yuck).
I should have done this change long ago, but I was stymied because of
"infinite recursions". The problem was that in too many cases we are
overriding `stdenv` to *remove* things we don't need, and this risks
cyles since those more minimal stdenvs are used to build things in the
more maximal stdenvs.
The solution is to pass `stage.nix` `stdenvNoCC`, so we can override to
*build up* rather than *tear down*. For now, the full `stdenv` is also
passed, so I don't need to change the native bootstraps, but I can see
this changing as we make things more uniform and clean those up.
Finally, the BSDs also had to be cleaned up, since they have a few
pre-libc dependencies, demanding a systematic approach. I realized what
rhelmot did in 61202561d9 (specify what
packages just need `stdenvNoLibc`) is definitely the right approach for
this, and adjusted NetBSD and OpenBSD to likewise use it.
The big `fetchCVS` is slow, but a one-time cost. Everything else is much
faster, and not having to manage a gazillion `version` and `sha256`
fields is much easier.
This brings NetBSD in line with how we do FreeBSD and OpenBSD.
These headers conflict with the unwind headers from libunwind
provided by libgcc_eh in `freebsd.libc`.
Upstream FreeBSD does not use these headers in any capacity,
and they cause some incompatibilities since libcxxrt unwind.h
requires _GNU_SOURCE for some functions, while libunwind does not.
Previously, an attribute named isStatic did this, but this was lost in a
refactor. Revive it and rename it to noLibc to be more clear about its
intended use.
netbsd can no longer compile under FreeBSD native early bootstrap
stdenv, so switch to coreutils. This only involves discarding the -l
flag. The -l flag causes a symlink instead of a copy to be installed, so
it is safe to discard during bootstrap.
This package will be necessary down the line for binary patching
applications built for non-nix FreeBSD. It is additionally being
considered for inclusion in FreeBSD libcxx, both cross and native.
The old patches are kept because the version is actually configurable.
In the future both versions may be exposed.
Co-Auhtored-By: Artemis Tosini <me@artem.ist>
Co-Authored-by: John Ericson <John.Ericson@Obsidian.Systems>
* Extend libc
Include non-libc core libraries in the libc package. Many of these
mirror libraries present in glibc on linux, such as libgcc, libraries
used for iconv, and libraries used for reading kernel info (libkvm,
libprocstat, libmemstat).
Without this many packages outside the freebsd tree would need to be
modified to include standard dependencies which would already be on
the system for other packages.
* Mark FreeBSD as using LLVM
* Update default LLVM version FreeBSD
* Use patch monolith
The patchesRoot system combined with the fact that each derivation
will Request specific names of patches makes it very annoying to use
other FreeBSD source trees with nixpkgs. This new system allows
providing one Or more entire trees of patches whose contents will be
dynamically Parsed and only the relevant patches will be applied for
any one Derivation.
With this commit, the following knobs are available for specifying the
FreeBSD source:
- overriding `freebsd.versionInfo`, for picking another official
supported FreeBSD release.
- overriding `freebsd.source` for specifying a specific unpatched
FreeBSD source tree.
- overriding `freebsd.patches`, for specifying the patches to apply.
Co-Authored-by: Audrey Dutcher <audrey@rhelmot.io>
Co-Authored-by: John Ericson <John.Ericson@Obsidian.Systems>
This creates multiple package sets for different versions. The other
versions don't yet work, but that will be fixed in subsequent PRs.
Push versionData into package set so that it can be overridden
Co-Authored-By: Audrey Dutcher <audrey@rhelmot.io>
Co-Authored-By: Artemis Tosini <me@artem.ist>
We want to know the minor versions.
We want to know the patch versions too, where possible. We get those
either from the branch when it is the form `RELEASE-p<patch>`, or from
the tag when its in the form `release/<major>.<minor>.<patch>`.
There are a number of packages that we ought to be able to autocall, but
cannot because we need to add manual arguments just to avoid splicing.
This sucks but is the right call for now --- the conclusion should be
not that auto-calling is bad, but that splicing is bad.
This tries to do nothing but move things around; hashes are almost
unchanged. @rhelmot then has more changes to do on top of this, which
will be easier to review since code will be modified in place rather
than moved around and modified at the same time.
Without the change the eval fails as:
$ nix build --no-link -f. netbsd.libcurses
...
… while evaluating attribute 'NIX_CFLAGS_COMPILE' of derivation 'libcurses-netbsd-9.2'
error: value is a string while a list was expected
with structuredAttrs lists will be bash arrays which cannot be exported
which will be a issue with some patches and some wrappers like cc-wrapper
this makes it clearer that NIX_CFLAGS_COMPILE must be a string as lists
in env cause a eval failure
FreeBSD implements Linux's evdev API, but doesn't come with headers
for it. Instead, the Linux headers are just modified to be suitable
for FreeBSD, via a port called evdev-proto. I don't want to copy the
complicated sed expressions from the port into Nixpkgs, so instead we
just build and install the port inside a Nix derivation.
Always set `SRCTOP`, set it with abs path
llvmPackages: Bump minimum version for FreeBSD
llvmPackages_*, libgcc, compiler_rt: Hack in enough libs that one can compiler C
freebsd.compat: Rename some things to work around cc-wrapper change
0bea4a194f / #191724 in particular
See https://github.com/NixOS/nixpkgs/pull/82131 for the rest of the
changes for FreeBSD. This is PRed separately because it is a macOS
moderate rebuild so we target staging.
The main change is that we CD to the path we're building *after*
applying patches, so we can patch other parts of the tree (from
`extraPaths`) as needed.
Another change is that `netbsd.install` no longer depends on `fts`,
which it evidently no longer needs.
When building glib statically, a Meson check would fail, because the
check would interpret any warning as failure, and it would see the
warning that the musl sys/cdefs.h emits about the file being
deprecated.
In pkgsStatic, /all/ build inputs are propagated. This means that
netbsd.compat was propagated, along with its setup hook, which broke
static glib builds because glib defines a function with the same name
as one in nbtool_config.h, and nbtool_config.h was being automatically
included in every C file processed by the compiler, in any transitive
dependent of netbsd.compat's setup hook.
To fix this, rather than forcing nbtool_config.h to be included for
_every_ C file in a derivation that depends on netbsd.compat, modify
the NetBSD-specific mkDerivation to detect files that need the header,
and patch it in there where appropriate. That way, only files that
are part of NetBSD will be affected, not all transitive dependents.
This expression is not properly spliced, so we need to
manually specify buildPackages so that mandoc will not
be built for the host platform
Fixes cross-compilation of netbsd.* packages that depend
on mandoc in nativeBuildInputs, such as `getent`.
* Fails because it's missing ps2pdf. In the beginning I fixed this by
providing ghostscript as a nativeBuildInput, but the PDF created for
man0 doesn't appear to be installed, so we can just patch out the call
to the tool.
* We need to disable MKRUMP or provide sys/rump/share/man in sourceRoot,
opted for the former since it's simpler at the moment, but the latter
is also possible with a custom unpackPhase.
Using extraPaths in NetBSD packages now requires rsync, but the rsync
dependency wasn't added to all the packages using extraPaths that
override nativeBuildInputs, so they'd just fail immediately.
Fixes: 75db7f8eb0 ("netbsd: Use rsync to speed up source merging")
Reverts d43df749ac
NetBSD makefiles strip local symbols from libs using `OBJCOPY?=objcopy`,
which is missing on macOS. GNU objcopy appears to succeed but produces
broken .a libs which do not link into dependers.
(As this issue does not fail the netbsd.compat build,
downstream netbsd.install is added to passthru.tests.)
Since `OBJCOPY` is only used for stripping, we can:
* skip stripping with the hacky `OBJCOPY=echo`
* use cctools strip, which is invoked in the same way
The latter is obviously preferable if it works.
Indeed, locals are stripped, although it doesn't affect size much.
Comparison:
`OBJCOPY=echo`:
```
$ du -b result/lib/*.a
347784 result/lib/libnbcompat.a
357120 result/lib/libnbcompat_p.a
```
`OBJCOPY=${cctools}/bin/strip`:
```
$ du -b result/lib/*.a
347008 result/lib/libnbcompat.a
357120 result/lib/libnbcompat_p.a
```
Not to netbsd, where it isn't needed, but elsewhere.
A few things going on here:
- Make compat use the "regular" not "host" makefile infra. This,
however, makes more assumptions that the toolchain is BSD-like, and
so we need to compensate for them with the likes of:
- `LORDER=...` and `TSORT=...`
- Move `export INSTALL_*` to install's setup hook so they don't interfere
with coreutils install
- Don't use `DESTDIR` for installing include files, instead set `INCSDIR`.
This is more proper, but doesn't work when `INCSDIR` is set multiple
times, unfortunately, as CLI defs override all other assignments. So
instead set `INCSDIR0` on the CLI, and do some `INCSDIR =
${INCSDIR0}/...` in the relevant packages.
- `INCSDIR` is set just in the NetBSD setup hook because FreeBSD uses
`INCLUDEDIR`.
I plan on doing the sources for FreeBSD differently. Indeed we might
want to change this for NetBSD too eventually.
In any event, the way we manage sources is not intrinsically the same
across BSDs so it makes sense to pull this out.
We need netbsdCross.ld_elf_so to be the dynamic linker in cross
netbsd's bintools, but netbsdCross doesn't have a libc in stdenv. So
instead, use netbsdCross.libc for netbsdCross.ld_elf_so.
Notes:
- compat: Needs a header from common merged with it's wrapper.
- librt: Needs the jemalloc sources.
- libc: install libc_pic.a
This was the behaviour on NetBSD 8.0, and ld.elf_so (in its current
configuration) requires it. We could also have disabled PIC in
ld.elf_so.
- sys: fix build
We use a more recent version of binutils than NetBSD 9.1 does, so we
need to backport a patch from CURRENT.
Co-authored-by: John Ericson <John.Ericson@Obsidian.Systems>
"common" is just a lot of shared code, not a component in and of
itself. There's no Makefile, so if we try to build it Make will go up
a directory and try to build all of NetBSD.
This is another one of those packages that's just a rebuild of a
certain part of libc...
Tested by building pkgsCross.x86_64-netbsd.netbsd.librt on
x86_64-linux.
libc's postPatch was entirely fixes for librt, so move that to librt
and inherit it in libc.
82c231d17e ("netbsd: Generalize builder to any-bsd setup hook")
missed libterminfo when it added bsdSetupHook to every other
package. (I checked it didn't miss anything else.)
It also didn't change a NETBSDSRCDIR to BSDSRCDIR, but in fairness
that line was added about half an hour before the setupHook change was
merged in a16384e118.
Fixes: 82c231d17e ("netbsd: Generalize builder to any-bsd setup hook")
Tested building pkgsCross.x86_64-netbsd.netbsd.libm from x86_64-linux.
At some point we should probably set SHLIBINSTALLDIR in the setupHook,
but I think I'd like to get everything working first, and then make
changes that affect all the builds like that. It's easier to spot
regressions when you know _everything_ worked before.
- No splicing makes everything less finnicky.
- Normal bootstrapping matches e.g. linux where kernel headers are also
`stdenvNoCC` but part of this stage.
Before, we were only building the headers, firmware, and bootloader.
CONFIG could be overridden to use another pre-defined kernel, but
there's no way to pass a custom kernel configuration yet.
Tested booting the built kernel in a NetBSD VM.
For some reason fetchcvs was taken from the cross package set when
cross compiling netbsd.* instead of from buildPackages. I suspect that
this was no issue previously because it just could be substituted as it
only ever produced fixed output derivations. However since
00a2084a40 the evaluation of fetchcvs
would depend on `stdenv.cc` (via its openssh dependency). Since
netbsd.libc is required to evaluate stdenv.cc this caused a infinite
recursion when staging the cross stdenv for pkgsCross.amd64-netbsd. The
fix for this is pretty easy: Just take fetchcvs from buildPackages
explicitly instead of hoping for mkDerivation to get it right.
Resolves#119323.
