BTF is a new, lightweight debug information format tailored specifically
for the needs of eBPF, allowing eBPF programs to be portable across
various kernel versions, configurations and distributions. This is used
by bpftrace and lots of new eBPF-based tooling to avoid a dependency
on LLVM on the host.
BTF debug information is enabled on all major distributions: Fedora 31+,
RHEL 8.2+, Ubuntu 20.10, Debian 11 and ArchLinux all have enabled it.
Enabling BTF debug information requires adding two new dependencies to
the kernel build: Python3 and pahole. Those will be used to generate the
BTF debugging information.
This is just for practicity, as it allows users of buildLinux to pass
along extra flags they need in the kernel's make invocation. This makes,
for example, supporting LLVM _much_ easier, and could enable us in the
future to provide clang-built kernels.
There are many scripts in `scripts/` which may be called by the build,
depending on how the user chooses to configure the kernel. For example,
`scripts/jobserver-exec` is called whenever the kernel is being built
with LLVM tooling, and without this patch that build will fail due to
the broken shebang.
This patch makes us fix _all_ scripts, as well as add a dependency on
python3Minimal, since a lot of the aforementioned scripts are written in
Python3 instead of shell.
In order to have our linux builds be reproducible we patch the
`Makefile` to use `--build-id=none` as opposed to the default
`--build-id=sha1`. The way we've been doing this, however, caused the
flag to be mangled, and being set to `--build-id=none=sha1`. While bfd
seems to parse this normally, lld will loudly complain that the flag
does not exist:
```
linux> LD .tmp_vmlinux.kallsyms1
linux> ld.lld: error: unknown --build-id style: none=sha1
```
With this change the flag is now correctly set to `--build-id=none`.
Xen is now enabled unconditionally on kernels that support it, so the
xen_dom0 feature doesn't do anything. The isXen attribute will now
produce a deprecation warning and unconditionally return true.
Passing in a custom value for isXen is no longer supported.
The `platform` field is pointless nesting: it's just stuff that happens
to be defined together, and that should be an implementation detail.
This instead makes `linux-kernel` and `gcc` top level fields in platform
configs. They join `rustc` there [all are optional], which was put there
and not in `platform` in anticipation of a change like this.
`linux-kernel.arch` in particular also becomes `linuxArch`, to match the
other `*Arch`es.
The next step after is this to combine the *specific* machines from
`lib.systems.platforms` with `lib.systems.examples`, keeping just the
"multiplatform" ones for defaulting.
Don't include an NT_GNU_BUILD_ID (that is randomly generated
at build time).
This improves the kernel reproducibility: when also disabling
the MOUDLE_SIG and SECURITY_LOCKDOWN_LSM options the build
is bit-by-bit reproducible.
utillinux depends on systemd because:
* uuidd supports socket activation
* lslogins can show recent journal entries
* fstrim comes with a service file (and we use this in NixOS)
* logger can write journal entries
(See https://www.openembedded.org/pipermail/openembedded-core/2015-February/102069.html)
systemd doesn't depend on utillinux but on utillinuxMinimal which is a
version of utillinux without these features to avoid cyclic
dependencies.
With this change, the linux kernel (of which i don't fully understand
why it would depend on util-linux in the first place, but this was added in
https://github.com/NixOS/nixpkgs/pull/32137/files without too much
explanation) depends on the minimal version of util-linux too.
This makes it that every time we change build flags in systemd
the linux kernel doesn't have to wastefully rebuild.
Go beyond the obvious setup hooks now, with a bit of sed, with a skipped case:
- cc-wrapper's `dontlink`, because it already is handled.
Also, in nix files escaping was manually added.
EMP
In 5.2 kernel a new mechanism was introduced which embeds the kernel
headers in the kernel image and exposes them in procfs for simplified
use by userland tools.
It was introduced in
43d8ce9d65
and later modified a bit in
f7b101d330
The archive containing the header files had nondeterminism through the
header files metadata - specifically `mtime`, but I also decided to
normalize some other aspects just in case.
In our default setup we currently compile this as a module, so to expose
the headers to test the functionality `kheaders` needs to be loaded.
See https://lkml.org/lkml/2019/10/4/1036 and
https://git.kernel.org/pub/scm/linux/kernel/git/masahiroy/linux-kbuild.git/commit/?h=fixes&id=2cc99c9cdc8fde5e92e34f9655829449cebd3e00
I commented out the documentation part of the patch to make it cleanly apply to
5.2 and 5.3, see remark in the patch itself.
81d4e65891 automatically bumped
linux_testing to 5.2.0-rc1, but the 5.2 merge window included a new
feature adding compressed headers for compiled kernels into
/proc/kheaders.tar.xz
See 43d8ce9d65
This feature requires 'cpio' to now be included in nativeBuildInputs
since it's used to construct that archive.
This wasn't caught by Hydra since we turn off build of linuxPackages,
but ideally we should at least build the kernel in the future
(linux_testing itself.)
Signed-off-by: Austin Seipp <aseipp@pobox.com>
Some packages don’t work correctly with pie. Here I disable it for:
- busybox
- linux kernel
- kexectools
I also get rid of the Musl conditional for disabling pie in GCC and
Binutils. Some day we might want to enable PIE without Musl and it
will be useful to have the *just* work with our compiler and linkers.
Since commit f620b1b693, the build directory is located inside the
source directory. Thus, the `cp -dpR` copies gigabytes worth of .o files
only to be deleted later on when we trim all non-essential files from
`$dev/lib/modules/${modDirVersion}/source/` thus causing a significant
amount of wasted I/O and peak disk usage.
As `cp` doesn't come with a `--exclude` flag, use rsync. And throw out
the Documentation folder while at it.
Instead of using a string to describe kernel config, use a nix
attribute set, then converted to a string.
- allows to override the config, aka convert 'yes' into 'modules' or
vice-versa
- while for now merging different configs is still crude (last spec wins),
at least there should be only one CONFIG_XYZ value compared to the current string
config where the first defined would be used and others ignored.
[initial idea by copumpkin in 2016, a major rebase to 2018 by teto]
The required argument 'hostPlatform' was missing from linuxPackages_custom's
call to linuxManualConfig.
In order to prevent this in the future, this commit adds
linuxPackages_custom_tinyconfig_kernel so linuxPackages_custom gets tested.
This also adds linuxConfig, to derivate default linux configurations
via make defconfig, make tinyconfig, etc.
Closes#38034.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
