modules-closure.sh seems to consider everything under
/lib/firmware to be a kernel module,
this change adds a special handling of `edid` directory,
which does not contain kernel modules
fixes#279739
Closes#302291Closes#301536
The following things have changed:
* For 5.7+: ZSWAP compressor uses zstd now.
* For 5.11+: ZRAM compressor uses zstd now.
* For 5.13+: kernel modules are compressed with zstd instead of xz.
* For 5.19+: support zstd-compressed firmware.
The modules-closure functionality needed explicit support for copying
over `.zst` files. Also, the VM image builder used busybox's `insmod`
before which doesn't support zstd. Switched to `kmod` and added xz/zstd
as dependencies for it, similar to how it's done for the actual stage1
in d33e52b253. The use of `kmod` here
doesn't seem to be such a big deal since it's only a build-time
dependency.
A handful of kernel modules use glob patterns to express their
firmware dependencies. (`git grep 'MODULE_FIRMWARE.*\*'`)
Previously, we weren't handling these patterns. Now, we are.
This should be a significant disk space saving for most NixOS
installations. This method is a bit more complicated than doing it in
the postInstall for the firmware derivations, but this way it's
automatic, so each firmware package doesn't have to separately
implement its compression.
Currently, only xz compression is supported, but it's likely that
future versions of Linux will additionally support zstd, so I've
written the code in such a way that it would be very easy to implement
zstd compression for those kernels when they arrive, falling back to
xz for older (current) kernels.
I chose the highest possible level of compression (xz -9) because even
at this level, decompression time is negligible. Here's how long it took
to decompress every firmware file my laptop uses:
i915/kbl_dmc_ver1_04.bin 2ms
regulatory.db 4ms
regulatory.db.p7s 3ms
iwlwifi-7265D-29.ucode 62ms
9d71-GOOGLE-EVEMAX-0-tplg.bin 22ms
intel/dsp_fw_kbl.bin 65ms
dsp_lib_dsm_core_spt_release.bin 6ms
intel/ibt-hw-37.8.10-fw-22.50.19.14.f.bseq 7ms
And since booting NixOS is a parallel process, it's unlikely (but
difficult to measure) that the time to user interaction was held up at
all by most of these.
Fixes (partially?) #148197
Since fdf32154fc, we no longer allow
missing modules in the initrd. Unfortunately since before this commit,
the modules-closure script would also fail on missing firmware, which
is a very common case (e.g. xhci-pci.ko.xz lists renesas_usb_fw.mem as
dependent firmware). Fix this by only issuing a warning instead.
Before this commit, the firmware information would be loaded from the
currently running kernel, not from the kernel to be loaded.
This commit ensures the correct kernel version and modules are read.
After a recent upgrade of modinfo, its output is now incorrect for
builtin modules. This commit filters out the output until a fix is made
available upstream
The previous code discarded entire dependency trees if the first entry in the dependency list compiled by `modprobe --show-depends` is a builtin and otherwise handled its output in a rather hackish way.
depmod looks for files modules.order and modules.builtin which are
generated at kernel build time but were previously not passed to
the modules-shrunk derivation
Pull request #38470 added support for running/building kernels without
modules. This got merged in 38e04bbf29 but
unfortunately while this works perfectly on kernels without modules it
also makes sure that *every* kernel gets no modules.
So all of our VM tests fail since that merge with something like this:
machine# loading module loop...
machine# modprobe: FATAL: Module loop not found in directory /lib/modules/4.14.33
machine# loading module vfat...
machine# modprobe: FATAL: Module vfat not found in directory /lib/modules/4.14.33
machine# loading module nls_cp437...
machine# modprobe: FATAL: Module nls_cp437 not found in directory /lib/modules/4.14.33
machine# loading module nls_iso8859-1...
machine# modprobe: FATAL: Module nls_iso8859-1 not found in directory /lib/modules/4.14.33
machine# loading module fuse...
machine# modprobe: FATAL: Module fuse not found in directory /lib/modules/4.14.33
machine# loading module dm_mod...
machine# modprobe: FATAL: Module dm_mod not found in directory /lib/modules/4.14.33
I shortly tested this against the "misc" VM test and the test is working
again.
In the long term (and I currently don't have time for this) it would be
better to also have a VM test which tests a kernel without modules.
Signed-off-by: aszlig <aszlig@nix.build>
Cc: @roberth, @7c6f434c
There were conflicting patches of pkgs/os-specific/linux/module-init-tools.
Apparently, the expression was updated independently in both branches. I've
resolved the conflict by preferring the patches from stdenv-updates, because
those patches appeared to be more sophisticated, i.e. they build the manual,
etc.
svn path=/nixpkgs/branches/stdenv-updates/; revision=29680
This comes from:
svn diff ^/nixpkgs/trunk/@18255 ^/nixpkgs/branches/stdenv-updates/ > diff
patch -p0 < diff
and then adding into svn all files new from the patch.
trunk@18255 comes from the last time I updated stdenv-updates from trunk.
svn path=/nixpkgs/stdenv-updates2/; revision=18272
tree under $out into a separate stdenv adapter named keepBuildTree.
* makeModulesClosure: support building an initrd for a kernel that has
been compiled with coverage instrumentation.
svn path=/nixpkgs/trunk/; revision=16916
modules for the initial ramdisk if there were no additional kernel
module packages (such as the NVIDIA driver or AUFS), leading to a
kernel panic in the initrd. This was because in that case modprobe
would print paths referring to the kernel path rather than the
module aggregation path, and then `sed "s^$kernel^$out^"' would
silently fail. Fixed.
* Also, use depmod here rather than doing sed hackery on modules.dep.
* Also, `allowMissing' was broken (missing "$" before the variable
name).
svn path=/nixpkgs/trunk/; revision=15394
