When installing NixOS on a machine with Windows, the "easiest" solution
to dual-boot is re-using the existing EFI System Partition (ESP), which
allows systemd-boot to detect Windows automatically.
However, if there are multiple ESPs, maybe even on multiple disks,
systemd-boot is unable to detect the other OSes, and you either have to
use Grub and os-prober, or do a tedious manual configuration as
described in the wiki:
https://wiki.nixos.org/w/index.php?title=Dual_Booting_NixOS_and_Windows&redirect=no#EFI_with_multiple_disks
This commit automates and documents this properly so only a single line
like
boot.loader.systemd-boot.windows."10".efiDeviceHandle = "HD0c2";
is required.
In the future, we might want to try automatically detecting this
during installation, but finding the correct device handle while the
kernel is running is tricky.
Use the store directory for the devicetree package containing the
desired DTB when installing to the ESP. This allows for more than one
NixOS generation containing differing DTBs to coexist on the same ESP
(similar to how we can have multiple kernels & initrds). This change
removes the assumption that the filepath passed to `copy_from_file` is a
file that lives at the toplevel of a nix output path (which prior to the
systemd-boot DTB support was the case for the kernel and initrd
derivations).
Without sort-keys specified on entries, the entries are sorted only by
file name (in decreasing order, so starting at the end of the alphabet!),
without taking any other fields into account (see
[the boot loader specification reference][1]).
Moreover, entries without a sort-key are always ordered after all
entries with a sort-key, so by not adding a sort-key to the NixOS ones,
we cannot add a sort-key to any other entry while keeping it below the
NixOS entries.
So currently we have options to set the file names for additional entries like
memtest and netbootxyz.
However, as mentioned above, the sorting by file name is not very intuitive and
actually sorts in the opposite order of what is currently mentioned in the option
descriptions.
With this commit, we set a configurable sort-key on all NixOS entries,
and add options for setting the sort-keys for the memtest and netbootxyz
entries.
The sorting by sort-key is more intuitive (it starts at the start of the
alphabet) and also takes into account the machine-id and version for entries
with identical sort-keys.
We use a bootspec extension to store the sort keys, which allows us to
redefine the sort key for individual specialisations without needing any
special casing.
[1]: https://uapi-group.org/specifications/specs/boot_loader_specification/#sorting
This allows us to drop our fsck-look-for-fsck-binary-not-just-in-
sbin.patch, as it was upstreamed.
We also manually backport https://github.com/systemd/systemd/pull/27856 as
it didn't get backported and without it we can't merge this PR as
systemd-boot-builder.py will remain broken and make it impossible to do upgrade
to NixOS 23.05 in some scenarios
Changelog:
```
991158e8b9 (hwdb: update to 2533fdd0fbe71e4a3fa7a2cca9830cd864fb9136, 2023-06-01)
d1087bc599 (test-network: add tests for vlan QoS mapping, 2023-05-24)
7ed7b07a92 (network/vlan: paranoia about type safety, 2023-05-24)
b20bc7c1ff (network/vlan: drop unnecessary restriction for QoS mapping, 2023-05-24)
dbf50f1911 (udev: do not set ID_PATH and by-path symlink for nvmf disks, 2023-05-10)
75d4967502 (journalctl: fix --no-tail handling, 2023-05-04)
f1ea9cd55e (journalctl: use correct variable to check if --since is specified, 2023-05-04)
0227947bab (test/README: fix advice for testsuite debugging, 2023-05-29)
3222272c46 (test-fstab-generator: fix test on systemd with systemd-boot, 2023-05-30)
23b7bf3d01 (home: move the assert back to the intended place, 2023-05-29)
901f0f0ac1 (resolvectl: drop extra colon, 2023-05-28)
5f3ca32d0c (basic/syscall: update syscall list, 2023-05-29)
375e6be16c (tree-wide: Downgrade a few more noisy log messages to trace, 2023-05-27)
3f5f7e5f30 (journal-remote: bump the refcount right after creating the writer object, 2023-05-25)
4810e789ad (man: fix UKI filename suffix in 'tries' description, 2023-05-26)
2e10f8874a (units: Shut down networkd and resolved on switch-root, 2023-05-25)
9dde31ac74 (resolve: avoid memory leak from a partially processed RR, 2023-05-23)
b1663b8333 (sd-journal: avoid double-free, 2023-05-23)
aa48ecb0a6 (core/timer: Always use inactive_exit_timestamp if it is set, 2023-05-23)
ac380e43a4 (core: Do not check child freezability when thawing slice, 2023-05-23)
53bc78d3e0 (tree-wide: Fix false positives on newer gcc, 2023-05-23)
58c1816aa4 (json: correctly handle magic strings when parsing variant strv, 2023-05-23)
fbb2c5ab19 (sysusers: fix argument confusion in error message, 2022-10-13)
e5520ab28f (sysusers: add usual "ret_" prefix, fix messages, 2022-10-13)
286ce2be44 (man: extend description of --boot, 2022-10-09)
7394a75688 (sd-bus: refuse to send messages with an invalid string, 2023-05-19)
ae83e97a51 (core/service: when resetting PID also reset known flag, 2023-05-22)
f0bb967388 (shared: correctly propagate possible allocation errors, 2023-05-21)
318c9d5fec (wait-online: downgrade log level of failure that interface is removed or unmanaged during processing it, 2023-05-22)
1a0f2c5c57 (boot: Read files in small chunks on broken firmware, 2023-01-05)
eeaf884f5b (cryptenroll: update log messages, 2023-05-20)
debce7c184 (test: check if we can use --merge with --follow, 2023-05-19)
3cf401e3e3 (manager: restrict Dump*() to privileged callers or ratelimit, 2023-04-27)
6ca461fe29 (ratelimit: add ratelimit_left helper, 2023-04-28)
604d132fde (journalctl: make --follow work with --merge again, 2023-05-19)
6a4c05c615 (test: make the stress test slightly less stressful on slower machines, 2023-05-19)
a08cb80451 (core/device: downgrade error when units specified in SYSTEMD_WANTS= not found, 2023-05-19)
eb5dad0a72 (unit: add conditions and deps to make oomd.socket and .service consistent, 2023-05-19)
c756ffea57 (oomd: shorten message, 2023-05-18)
a3e5eb5606 (sd-bus,sd-event: allow querying of description even after fork, 2023-05-18)
e91557a1e0 (sd-bus: do not assert if bus description is not set, 2023-05-18)
93b3bd12ac (test: don't mount /sys & /proc if already mounted, 2023-05-18)
c51273941d (nspawn: make the error message less confusing, 2023-05-18)
e85daabd3e (Revert (partially) "man: Clarify when OnFailure= activates after restarts (#7646)", 2023-05-17)
3e286a7b2e (man/tmpfiles: fix off-by-one in example, 2023-05-17)
cb6641bde3 (man: explain allowed values for /sys/power/{disk,state}, 2023-05-17)
65bf6c5a8f (man: say that ProtectClock= also affects reads, 2023-05-17)
13c8807360 (man: fixes for assorted issues reported by the manpage-l10n project, 2023-05-17)
1809fff392 (nspawn: make sure the device type survives when setting device mode, 2023-05-16)
b8ed81660f (nspawn: fix a global-buffer-overflow, 2023-05-15)
756e77b936 (nspawn: fix inverted condition, 2023-05-15)
c7861222ba (nspawn: call json_dispatch() with a correct pointer, 2023-05-15)
6f577f5d92 (nspawn: use the just returned errno in the log message, 2023-05-15)
9a7c6ed568 (nspawn: avoid NULL pointer dereference, 2023-05-16)
17c7b07c67 (nspawn: file system namespace -> mount namespace, 2023-05-15)
b13e836315 (nspawn: fix a typo in an error message, 2023-05-15)
d88225ef44 (busctl: set a description for the bus connection, 2023-05-05)
29115ef32e (man: indicate that the JOB parameter to "systemctl cancel" is optional, 2023-05-16)
051f86ae0e (meson: fix description for link-udev-shared option, 2023-05-16)
85ba46539f (man: use correct name for --bank option, 2023-05-15)
d7e75c7315 (machine,portable: fix a typo in an info message, 2023-05-12)
4d29f741c8 (machine: fix a memory leak when showing multiple machines, 2023-05-12)
e6a719598c (machine: fix a memory leak when showing multiple images, 2023-05-12)
ea221dc685 (fstab-generator: Fix log message, 2023-05-10)
4c3b06f255 (test: test O_CLOEXEC filtering of fdset fill logic, 2023-05-30)
88bf6b5815 (pid1: when taking possession of passed fds check O_CLOEXEC state first, 2023-05-30)
0d8372b450 (repart: Create temporary root directory using var_tmp_dir(), 2023-02-14)
aedfe41cda (cryptenroll: actually allow using multiple "special" strings when wiping, 2023-05-10)
f59ce1aa7b (core: fix use of uninitialized value, 2023-05-04)
3f5db0dbc1 (sd-journal: check .next_entry_array_offset earlier, 2023-05-03)
0baac8e60e (tree-wide: drop _pure_ attribute from non-pure functions, 2023-05-10)
4984f70db5 (dirent: conditionalize dirent assert based on dirent64 existence, 2023-05-10)
5fcbda8b5e (network/tc: rename settings in log messages too, 2023-05-10)
59dccdfddb (sd-bus: bus_message_type_from_string is not pure, 2023-05-10)
133d4ff6d6 (cryptenroll: fix an assertion with weak passwords, 2023-05-09)
c937b8f9de (units: Add CAP_NET_ADMIN condition to systemd-networkd-wait-online@.service as well, 2023-05-07)
60af5019fb (units: add/fix Documentation= about bus interface, 2023-05-09)
53f7e5f18f (core/service: fix error cause in the log, 2023-05-09)
951c27ce14 (shell completion: add timesync-status and show-timesync to zsh completion file (#27574), 2023-05-08)
32831842ba (doc: remove legacy DefaultControlGroup from dbus properties, 2023-05-08)
c31e2fa9c7 (zsh: add service-log-{level,target} completions for systemctl, 2023-05-07)
011a686a23 (test_ukify: fix loop iteration, 2023-04-21)
927d234406 (hwdb: do not include '#' in modalias, 2023-05-06)
b1a7a15ed2 (core: check the unit type more thoroughly when deserializing, 2023-05-04)
154b108513 (shared: refuse fd == INT_MAX, 2023-05-04)
a25605d01d (zsh: remove usage of PREFIX in _systemctl, 2023-05-05)
4be604e75a (basic/audit-util: make a test request before enabling use of audit, 2023-05-02)
4b4285e231 (main: add missing return, 2023-05-05)
ce096b0212 (shared: reject empty attachment path, 2023-05-02)
6027fbf1af (shared: ignore invalid valink socket fd when deserializing, 2023-05-02)
d649128268 (core: fix NULL pointer dereference during deserialization, 2023-05-02)
6ae77d6b99 (boot: Use correct memory type for allocations, 2023-05-02)
de0cbaceb7 (core: check for SERVICE_RELOAD_NOTIFY in manager_dbus_is_running, 2023-05-02)
5ed087fa46 (generators: skip private tmpfs if /tmp does not exist, 2023-04-30)
93143b6d6a (test: replace sleep with timeout, 2023-05-02)
881382685e (test-network: add workaround for bug in iproute2 v6.2.0, 2023-05-02)
abf9e916ad (coredumpctl: add --file/--root/--image to bash completion, 2023-04-25)
dd349a0ede (coredumpctl: fix bash completion matching, 2023-04-25)
120342b62d (test: match all messages with the FILE field, 2023-04-29)
e0da5c9bc6 (test: add tests for "systemctl stop" vs triggering by path unit, 2023-04-29)
c1542a967b (test: create temporary units under /run, 2023-04-29)
03f2a8921e (core/path: do not enqueue new job in .trigger_notify callback, 2023-04-29)
674591e6af (core/path: align table, 2023-04-29)
0413fb7de9 (test: add a couple of tests for systemd-pstore, 2023-04-27)
de41e55c7d (pstore: avoid opening the dmesg.txt file if not requested, 2023-04-28)
37c212dbd7 (pstore: explicitly set the base when converting record ID, 2023-04-28)
daee48adbb (test: dont use anchor char '$' to match a part of a string, 2023-04-27)
53ac14a054 (core/transaction: use hashmap_remove_value() to make not remove job with same ID, 2023-04-26)
0258760397 (resolved: adjust message about credentials, 2023-04-25)
8f19911bc3 (fuzz-journal-remote: fix potential fd-leak, 2023-03-18)
df1e479d4e (fuzz-journal-remote: remove temporary files on exit, 2023-03-18)
0d745e2de3 (hwdb: update to 46b8c3f5b297ac034f2d024c1f3d84ad2c17f410, 2023-04-30)
df9d1d9bb2 (sd-journal: make journal_file_copy_entry() return earlier, 2023-04-26)
3bc2553cfc (sd-journal: copy boot ID, 2023-04-26)
45b045880c (sd-journal: tighten variable scope, 2023-04-26)
3821e3ea07 (journal: Don't try to write garbage if journal entry is corrupted, 2023-04-26)
4eedc4711a (test: add test case of negative match for SYMLINK and TAG, 2023-04-25)
cd795f9abc (udev-rules: fix negative match rule for SYMLINK and TAG, 2023-04-25)
a25e2ef992 (core: fix property getter method for NFileDescriptorStore bus property, 2023-04-12)
eec30e3143 (repart: always take BSD lock when whole block device is opened, 2023-04-13)
50ab96e442 (bootctl: clean up handling of files with no version information, 2023-03-30)
9d97c8d423 (mkosi: disable centos 8 build, 2023-04-26)
c603dae241 (mkosi: disable key check for Fedora builds, 2023-04-26)
724a50fb01 (mkfs-util: do not pass -quiet to mksquashfs, 2023-04-27)
43d194392f (test: use setpriv instead of su for user switch from root, 2023-03-14)
ba683eb48c (test: wrap mkfs.*/mksquashfs/mkswap binaries when running w/ ASan, 2023-03-16)
fdcd1807ff (test: bump the D-Bus related timeouts to 120s, 2023-03-09)
4f8b2abf69 (coredump filter: add mask for 'all' using UINT32_MAX, not UINT64_MAX, 2023-04-26)
021bb972ff (coredump filter: fix stack overflow with =all, 2023-04-26)
3fd444c048 (build(deps): bump github/super-linter from 4.9.7 to 4.10.1, 2023-04-01)
a19396c73b (cryptenroll: fix a memory leak, 2023-03-27)
083ede1482 (test: tell dfuzzer to skip Reexecute(), 2023-04-26)
ae12c1380b (portablectl: add --extension to bash completion, 2023-04-25)
b1ecfe3fe7 (man: /usr/lib/systemd/random-seed -> /usr/lib/systemd/systemd-random-seed, 2023-04-25)
8895ccaaa8 (cryptsetup-fido2: Depend on libcryptsetup, 2023-04-24)
c6e957d02d (test: use idiomatic bash loop iteration, 2023-04-07)
26e181e94e (testsuite-54: drop unnecessary pipe, 2023-04-05)
d2c738341b (testsuite-70: drop unnecessary env, 2023-04-05)
f3abd451dd (test: drop uses of "&& { echo 'unexpected success'; exit 1; }", 2023-04-05)
59243061f6 (man: fix LogControl1 manpage example, 2023-04-24)
04983c2b00 (pam: cache sd-bus separately per module, 2023-04-16)
0045d952b5 (pam_systemd_home: clean up sd-bus when called about something else's user, 2023-04-20)
c50ec75e1e (testsuite-04: remove unnecessary conditional, 2023-04-04)
5a8987794e (man: clarify sd_bus_default, 2023-04-22)
b9af9a320e (man: add working example to LogControl1 manpage, 2023-04-21)
4d2b5338ac (detect-virt: add message at debug level, 2023-04-20)
749a6d9959 (dissect: let's check for crypto_LUKS before fstype allowlist check, 2023-04-20)
1aa6171081 (ratelimit: handle counter overflows somewhat sanely, 2023-04-20)
5ff63b8507 (man: try to make clearer that /var/ is generally not available in /usr/lib/systemd/system-shutdown/ callouts, 2023-04-20)
2be23f69ee (dissect-image: issue BLKFLSBUF before probing an fs at block device offset != 0, 2023-04-20)
7b437659b1 (list: fix double evaluation, 2023-04-20)
ffbb75aa46 (mountpoint-util: check /proc is mounted on failure, 2023-04-17)
14eb49b5eb (test: prefix the transient unit with test- to make coverage runs happy, 2023-04-18)
980954d2cf (kmod-setup: bypass heavy virtio-rng check if we are not running in a VM anyway, 2023-04-18)
567a1a6fd8 (kmod-setup: use STARTSWITH_SET() where appropriate, 2023-04-18)
d37f06f96f (creds: make available to all ExecStartPre= and ExecStart= processes, 2023-04-15)
d15f907b5b (user-util:remove duplicate includes, 2023-04-17)
cedea4cb7e (virt: Further improve detection of EC2 metal instances, 2023-04-13)
826662680b (string-util: add strstrafter(), 2023-04-14)
ac721c88af (test: add a couple of tests with invalid UTF-8 characters, 2023-04-15)
9c8d8719e4 (test: add a simple test for getenv_path_list(), 2023-04-15)
a9c73150ac (test: add a couple of basic sanity tests for the security verb, 2023-04-15)
06a70861bc (test: add a couple of basic sanity tests for timedatectl, 2023-04-15)
def6c37a19 (shared: add a missing include, 2023-04-15)
79e23f618f (test: add tests for uuid/uint64 specifiers, 2023-04-15)
3ee1839c19 (fsck: look for fsck binary not just in /sbin, 2023-04-13)
eab75a8591 (test: stop the test unit when it's not needed anymore, 2023-04-14)
f86ec34958 (Synposis and description of networkctl man page reflecting only part of its functionality (#27264), 2023-04-13)
fffcebc4bb (core/main: fix a typo for --log-target, 2023-04-13)
f152cdabae (test: add some tests for RuntimeMaxSec, 2023-04-13)
999f48558b (scope: do not disable timer event source when state is SCOPE_RUNNING, 2023-04-04)
430861fc96 (Fix cross-reference of manual for LogsDirectory, 2023-04-12)
91953109ec (pid1: fix coredump_filter setting, 2023-04-12)
fa8d33bb37 (Uphold/StopWhenUnneeded/BindsTo: requeue when job finishes, 2023-04-12)
6fc08d8407 (Uphold/StopWhenUnneeded/BindsTo: add retry timer on rate limit, 2023-04-12)
1fb4ae32b0 (man: add util-linux to the package list for Fedora container, 2023-04-12)
841146f243 (man: link to Fedora 37, 2023-04-12)
465edc1230 (systemctl: suppress error for try-* if unit is masked, 2023-04-04)
7102925d1a (ci: drop checkout from release workflow, 2023-04-11)
167c01688f (ci: don't run release wf on `systemd-security`, 2023-04-11)
bda5c892a8 (shell-completion: add --xml-interface option of busctl to the rules, 2023-04-11)
6265430ca9 (busctl: add --xml-interface to the help message, 2023-04-11)
d26fd71d1a (test: update description, 2023-04-11)
35a6460a2f (test: systemd-analyze blame should succeed now, 2023-04-10)
ef10974c66 (analyze: make blame command work even the default target not reached, 2023-04-10)
dc2facf61d (ci: add permissions to make a release, 2023-04-03)
4c65c644d6 (test/test-functions: fix typo in install_suse_systemd(), 2023-04-04)
fca5a45a59 (test: install symlinks with valid targets on SUSE and Debian, 2023-03-24)
d18037b8ff (localed: fix invalid free after shifting pointers using strstrip, 2023-04-07)
93ac024b7e (test: bump the timeout for non-qemu runs to 90s, 2023-04-07)
283b7b4159 (test: enable the systemd-resolved unit in TEST-75, 2023-04-07)
6179141124 (man/systemd-mount: Clearify documentation about --bind-device, 2023-04-05)
b2e1dabbeb (resolve: change DNS_PACKET_UNICAST_SIZE_LARGE_MAX to 1232 (#27171), 2023-04-07)
16dc17d68c (man: netdev: Clarify wireguard IPv6 endpoint format, 2023-04-07)
0558c490a6 (test: use kbd-mode-map we ship in TEST-73-LOCALE, 2023-04-05)
64ef6ccd4f (ci: do one build with no tpm/p11kit/fido2, 2023-04-04)
018461aaf0 (man: mention -o option for systemd-journal-remote, 2023-04-05)
31c7f6d0d1 (manager: remove transient unit directory during startup, 2023-04-04)
49c6965946 (core: a more informative error when SetProperties/StartTransientUnit fails, 2023-04-02)
649e335bc1 (journald: fix log message, 2023-04-04)
eda7bf237f (Added unit test for strv_env_name_is_valid() function listed in env-util.c (#27100), 2023-04-02)
0430078cfb (man: restore description of ConditionControlGroupController=v1|v2, 2023-03-31)
0d9c2c270b (test: set ReadWritePaths= for test-.services when built w/ coverage, 2023-03-31)
384fec2622 (core: skip deps on oomd if v2 or memory unavailable, 2023-03-31)
2950b4ebf6 (test: fixed negative checks in TEST-70-TPM2. Use in-line error handling rather than redirections. Follow up on #27020, 2023-03-30)
786649c904 (test: make make_addresses() actually return the addresses, 2023-03-30)
5e3ac73017 (coverage: add a wrapper for execveat(), 2023-03-30)
8b1cc644c5 (man: add example for sd_bus_call_method, 2023-03-30)
382e53977c (man: further shorten print-unit-path example, 2023-03-29)
960f05945c (man: link up new online coredump docs from man page, 2023-03-30)
edfca36727 (tree-wide: reset optind to 0 when GNU extensions in optstring are used, 2023-03-21)
91ff21962d (test-kernel-install: several cleanups, 2023-03-28)
9943f2af3d (units: let's establish the coredump socket before writting core_pattern sysctl, 2023-03-29)
dbb1b9c2c8 (test: do not remove state directory on failure, 2023-03-29)
29cfb05183 (test: fix shellcheck warnings in test-sysusers.sh, 2023-03-29)
18afac6e90 (man: fix shellcheck warning for html.in, 2023-03-29)
4629419038 (added more test cases, 2023-03-27)
05ae9e276c (test: fix regexp in testsuite-74.mount.sh, 2023-03-28)
295012f7fa (test: drop extraneous bracket in testsuite-74.mount.sh, 2023-03-28)
ff7040b193 (busctl: also assume --full if not writing to terminal, 2023-03-28)
00977a8e74 (busctl: use size_t for set size, 2023-03-28)
802fded9a5 (busctl: do not truncate property values when --full, 2023-03-28)
e400a62a92 (oomd: add inline comments with param names, 2023-03-21)
4067ec52f4 (test: add more testcases for rm_rf(), 2023-03-19)
201830df21 (rm-rf: also chmod() directory if it cannot be opened, 2023-03-19)
d91f7eb0fb (rm-rf: mask file mode with 07777 when passed to chmod(), 2023-03-19)
80417f90b0 (rm-rf: fix errno handling, 2023-03-18)
```
Co-authored-by: Arian van Putten <arian.vanputten@gmail.com>
The update test patches the systemd-boot binary to report a known
version then tests that this is the version updated from. The previous
patch would also search the kernel and initrd binaries, which would
cause sed to write out a temporary file that might cause the disk
to run out of space and the test to fail.
Only attempt to patch binaries which contain systemd-boot (usually
`BOOT<arch>.EFI` and `systemd-boot<arch>.efi` to avoid this problem.
As a bonus, this reduces test time by 20-30%.
At some point many months ago, the systemd-boot update script stopped
outputting parentheses around the version being upgraded from, causing
the test to fail. Remove the parentheses from the expected message to
fix the test.
Some specialisations (such as those which affect various boot-time
attributes) cannot be switched to at runtime. This allows picking the
specialisation at boot time.
