This is done for sd-images only here, but should probably also be done
for dvd-images.
The --invariant arg should be a better way of making mkfs.vfat deterministic.
The previous version of invoking faketime was building fine and reproducible
when I was compiling an sdimage for aarch64 under emulation.
It was however still logging errors:
ERROR: ld.so: object '/nix/store/1c2cp2709kmvby8ql2n9946v7l52nn50-libfaketime-0.9.9/lib/libfaketime.so.1' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/nix/store/1c2cp2709kmvby8ql2n9946v7l52nn50-libfaketime-0.9.9/lib/libfaketime.so.1' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
The logged errors were presumably inaccurate somehow as calling
faketime was required for reproducibility, even though the log makes it
looks like it failed.
Within #193485 (and the previous changes) the internal structure of the
testing driver was changed. Since then, `makeTest` returns the
attributes for the VM test(s) (including `driverInteractive`) inside a
sub-attribute called `test`, so without this change running
`nixos-build-vms` would fail like this:
error: attribute 'driverInteractive' missing
Move already implemented functionality to the upper level so
it could be used in a more generic way.
Signed-off-by: Ivan Nikolaenko <ivan.nikolaenko@unikie.com>
this mostly means marking options that use markdown already
appropriately and making a few adjustments so they still render
correctly. notable for nftables we have to transform the md links
because the manpage would not render them correctly otherwise.
This is image media, so use the override level designed for it. As
detailed in the definition for mkImageMediaOverride:
> image media profiles can be derived by inclusion into host config,
> hence needing to override host config, but do allow user to mkForce
our xslt already replaces double line breaks with a paragraph close and
reopen. not using explicit para tags lets nix-doc-munge convert more
descriptions losslessly.
only whitespace changes to generated documents, except for two
strongswan options gaining paragraph two breaks they arguably should've
had anyway.
The `nixos-rebuild` tool calls `get-version-suffix` to figure out the
git revision of the nixpkgs directory if there is a .git.
https://nvd.nist.gov/vuln/detail/CVE-2022-24765 made git throw an
error if the .git search logic is not turned off and a user
tries to access a `.git` directory they don’t own (otherwise a
different user could trick them into setting arbitrary git config).
So from now on we should always explicitely set `--git-dir`, which
turns this search logic (and thus the security check) off.
The substr solution assumed a newline to be present.
The new solution will not remove the newline if it goes missing in the future.
Apparently this is idiomatic perl.
Thanks pennae for the suggestion!
`nixos/modules/installer/kexec/kexec-boot.nix` doesn't contain any
custom NixOS config, other than importing `netboot-minimal.nix` (which
imports `netboot-base.nix`, which imports `netboot.nix`.
`netboot.nix` really is just describing a self-contained system config,
running entirely off kernel and initrd, so we might as well move the
kexec script generation there as well.
`netboot.nix` already contains some `system.build` attributes.
Provide a `system.build.kexecTree` attribute (and `kexecScript` for
composability).
Previously, `makeInitrd` added the whole closure of the squashfs
derivation to initrd.
This closure contains the squashfs.img and some store paths which are
still referenced by the compressed squashfs.img.
These extra store paths are unused in stage 1.
With `makeInitrdNG` only the squashfs.img is added to the initrd.
(`makeInitrdNG` only resolves shared library references instead of the
whole closure).
This shrinks the netboot ramdisk by ~6% for a minimal system and
significantly decreases the size of the uncompressed root filesystem
in stage 1.
Very confusingly, the `isPowerPC` predicate in
`lib/systems/inspect.nix` does *not* match `powerpc64le`!
This is because `isPowerPC` is defined as
isPowerPC = { cpu = cpuTypes.powerpc; };
Where `cpuTypes.powerpc` is:
{ bits = 32; significantByte = bigEndian; family = "power"; };
This means that the `isPowerPC` predicate actually only matches the
subset of machines marketed under this name which happen to be 32-bit
and running in big-endian mode which is equivalent to:
with stdenv.hostPlatform; isPower && isBigEndian && is32bit
This seems like a sharp edge that people could easily cut themselves
on. In fact, that has already happened: in
`linux/kernel/common-config.nix` there is a test which will always
fail:
(stdenv.hostPlatform.isPowerPC && stdenv.hostPlatform.is64bit)
A more subtle case of the strict isPowerPC being used instead of the
moreg general isPower accidentally are the GHC expressions:
Update pkgs/development/compilers/ghc/8.10.7.nix
Update pkgs/development/compilers/ghc/8.8.4.nix
Update pkgs/development/compilers/ghc/9.2.2.nix
Update pkgs/development/compilers/ghc/9.0.2.nix
Update pkgs/development/compilers/ghc/head.nix
Since the remaining legitimate use sites of isPowerPC are so few, remove
the isPowerPC predicate completely. The alternative expression above is
noted in the release notes as an alternative.
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
Installing Firefox is a good example for a package that could be
installed as a user, since it is a graphical one.
Also use thunderbird as a second example.
Currently we're still using scripted networking by default. A problem
with scripted networking is that having `useDHCP` on potentially
non-existing interfaces (e.g. an ethernet interface for USB tethering)
can cause the boot to hang.
Closes#107908
This includes disabling some features in the initrd by default, this is
only done when the new initrd is used. Namely, ext and bcache are
disabled by default. bcache gets an own enable option while ext is
detected like any other filesystem.
UEFI firmware does not have to be able to read ISO9660 filesystems, so
the El Torito mechanism provides a way to specify an embedded FAT32
image which contains files the UEFI firmware itself must be able to
read, such as UEFI executables. Once GRUB starts and reads its
configuration, it can access the ISO9660 filesystem to load other files.
This change removes the unused kernel, initrd, and GRUB font files from
the El Torito image, but keeps the GRUB configuration and UEFI
executables. These files have been present since EFI support was
originally introduced in commit 097c656. Other distribution ISOs, such
as Ubuntu 20.04, Fedora 35, and Windows 10 work this way too. This saves
24MiB on x86_64 and 61MiB on aarch64 ISOs.
This module exposes a config.system.build.kexecBoot attribute,
which returns a directory with kernel, initrd and a shell script
running the necessary kexec commands.
It's meant to be scp'ed to a machine with working ssh and kexec binary
installed.
This is useful for (cloud) providers where you can't boot a custom image, but
get some Debian or Ubuntu installation.
Not entirely sure when it got broken this time, but when creating a VM
network with `nixos-build-vms(8)`, there are should be the following scripts:
* `$out/bin/nixos-test-driver` which drops into an interactive shell to
interactively perform test steps.
* `$out/bin/nixos-run-vms` which non-interactively starts the VMs from
the network so that one can manually play around in the VM.
The latter also starts an interactive shell for a while now which means
that it does the exact same thing as `nixos-test-driver` which is not
its purpose.
The live image is primarily used for installation so we should make
link to manual as well as other useful tools front and center,
instead of having them buried in the app drawer.
The default GNOME apps can still be found there when the ISO
is used for demonstration purposes.
The `nix.*` options, apart from options for setting up the
daemon itself, currently provide a lot of setting mappings
for the Nix daemon configuration. The scope of the mapping yields
convience, but the line where an option is considered essential
is blurry. For instance, the `extra-sandbox-paths` mapping is
provided without its primary consumer, and the corresponding
`sandbox-paths` option is also not mapped.
The current system increases the maintenance burden as maintainers have to
closely follow upstream changes. In this case, there are two state versions
of Nix which have to be maintained collectively, with different options
avaliable.
This commit aims to following the standard outlined in RFC 42[1] to
implement a structural setting pattern. The Nix configuration is encoded
at its core as key-value pairs which maps nicely to attribute sets, making
it feasible to express in the Nix language itself. Some existing options are
kept such as `buildMachines` and `registry` which present a simplified interface
to managing the respective settings. The interface is exposed as `nix.settings`.
Legacy configurations are mapped to their corresponding options under `nix.settings`
for backwards compatibility.
Various options settings in other nixos modules and relevant tests have been
updated to use structural setting for consistency.
The generation and validation of the configration file has been modified to
use `writeTextFile` instead of `runCommand` for clarity. Note that validation
is now mandatory as strict checking of options has been pushed down to the
derivation level due to freeformType consuming unmatched options. Furthermore,
validation can not occur when cross-compiling due to current limitations.
A new option `publicHostKey` was added to the `buildMachines`
submodule corresponding to the base64 encoded public host key settings
exposed in the builder syntax. The build machine generation was subsequently
rewritten to use `concatStringsSep` for better performance by grouping
concatenations.
[1] - https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md
since fc614c37c6 nixos needs access to its
own path (<nixpkgs/nixos>) to evaluate a system with documentation.
since documentation is enabled by default almost all systems need such
access, including the installer tests. nixos-install however does not
ensure that a channel exists in the target store before evaluating the
system in that store, which can lead to `path is not valid` errors.
`mktemp` tries to use the `TMPDIR` from `nixos-install` outside of the
`chroot` instead of `/tmp` inside the `chroot` and fails. For some
reason the `TMPDIR` is being passed through the `chroot` call.
I haven't tested if other environment variables are being passed through
that shouldn't be.
This commit encapsulates the involved domain into classes and
defines explicit and typed arguments where untyped dicts where used.
It preserves backwards compatibility through legacy wrappers.
so the underlaying use case of the preceding commit is so
generic, that we gain a lot in reasoning to give it an
appropriate name.
As the comment states:
image media needs to override host config short of mkForce
https://github.com/NixOS/nixpkgs/pull/131760 was made to avo
a speicific configuration conflict that errored out for multiple definitions of "/" when the installer where overlayed
on any existing host configuration.
---
Problem 1: It turns out that in also other mountpoints can coflict.
Solution 1: use `mkOverride 60` for all mountpoints (even for the ones unlikely causing confilct for consistency sake)
---
Problem 2: It turns out that on an installation media for a fresh machine (before formatting), we usually don't have any devices yet formatted. However defining for example `fileSystems.<nme>.device = "/dev/disk/by-label/...", in newer versions of nixos, seems to make the system startup fail. Similarily waiting for a non-existent swap device does not make the startup fail, but has a 1:30 min timeout.
Solution 2: For an installation medium, soft-override ("unless users know what they are doing") the entire `fileSystems` and `swapDevices` definitions.
installer media can be used on top of existing host configs. In such
scenarions, root fs types will already be defined.
Before this change, this will inevitably lead to the following error:
```console
error: The option `fileSystems./.fsType' has conflicting definition values:
- In `/nix/store/2nl5cl4mf6vnldpbxhrbzfh0n8rsv9fm-source/DevOS/os/hardware/common.nix': "ext4"
- In `/nix/store/jbch90yqx6gg1h3fq30jjj2b6h6jfjgs-source/nixos/modules/installer/cd-dvd/iso-image.nix': "tmpfs"
```
With this patch, the installers will override those values according to
their own local requirement.
Use `mkOverride 60` so that conscientious overriding specially targeted
at the installer, e.g. with `mkForce` is still straight forward.
Different boards using u-boot SPL require to write to different
locations. Sometimes, the 8MiB gap isn't sufficient - rk3399 boards
write to 0x16384 for example, which is at 8MiB, thus overriding the
fat32 partition with the SPL.
This should help in rare hardware-specific situations where the root is
not automatically detected properly.
We search using a marker file. This should help some weird UEFI setups
where the root is set to `(hd0,msdos2)` by default.
Defaulting to `(hd0)` by looking for the ESP **will break themeing**. It
is unclear why, but files in `(hd0,msdos2)` are not all present as they
should be.
This also fixes an issue introduced with cb5c4fcd3c
where rEFInd stopped booting in many cases. This is because it ended up
using (hd0) rather than using the `search` which was happening
beforehand, which in turn uses (hd0,msdos2), which is the ESP.
Putting back the `search` here fixes that.
This technically changes nothing. In practice `$root` is always the
"CWD", whether searched for automatically or not.
But this serves to announce we are relying on `$root`... I guess...
The value of du output depends on the underlying file system, and thus is not fully deterministic. This workaround rounds up the disk usage size to the nearest multiple of 1MB, to increase the probability that two du output values on two different file systems fall within the same 1MB window. Note that this workaround won't make du output 100% reproducible, but will increase the probability of getting deterministic builds across different file systems.
mcopy file globbing is non-deterministic with respect to the underlying file
system. As a result, the current mcopy approach is less likely to reproduce
efi.img on different machines. We replace mcopy file globbing with
fixed-order mmd and mcopy operations for better determinism. We also use
faketime on mmd for the same reason. We use faketime, mmd, and mcopy
directly, becase they are already in PATH.
Thank misuzu@ for the feedback.
This reverts commit 6f6b2cdc98.
Version wasn't updated, and apparently a patch didn't apply. Let's do
this upgrade properly, in a PR, but for now I'm reverting so we don't
have a broken nix package in master.
Looks like GRUB has issues loading EFI binaries from (cd0), which is
what would be used in e.g. qemu with OVMF with `-cdrom`. Apparently also
what is used with AArch64 + U-Boot USB.
The serial output (but it's named console, not serial actually) causes
issues on U-Boot's EFI, at the very least.
This is inspired by OpenSUSE's approach:
* https://build.opensuse.org/package/view_file/Base:System/grub2/grub2-SUSE-Add-the-t-hotkey.patch
Where they add a hidden menu entry, which can be used to force the
console output.
The `echo` will be visible on the serial terminal (grub "console"),
while the graphical interface is shown. Note that input in the serial
terminal (grub "console") will continue controlling the graphical
interface. Useful if you have an SBC connectedinto an HDMI monitor, but
no keyboard connected to it.
This is supeer useful to allow the normal sd-image code to be used by
someone who wants to setup multiple partitions with a sd-image.
Currently I'm manually copying the sd-image file and modifying it
instead.
Since 03eaa48 added perl.withPackages, there is a canonical way to
create a perl interpreter from a list of libraries, for use in script
shebangs or generic build inputs. This method is declarative (what we
are doing is clear), produces short shebangs[1] and needs not to wrap
existing scripts.
Unfortunately there are a few exceptions that I've found:
1. Scripts that are calling perl with the -T switch. This makes perl
ignore PERL5LIB, which is what perl.withPackages is using to inform
the interpreter of the library paths.
2. Perl packages that depends on libraries in their own path. This
is not possible because perl.withPackages works at build time. The
workaround is to add `-I $out/${perl.libPrefix}` to the shebang.
In all other cases I propose to switch to perl.withPackages.
[1]: https://lwn.net/Articles/779997/
It was introduced in c10fe14 but removed in c4f910f.
It remained such that people with older generations in their boot
entries could still boot those. Given that the parameter hasn't had any
use in quite some years, it seems safe to remove now.
Fixes#60184
The `platform` field is pointless nesting: it's just stuff that happens
to be defined together, and that should be an implementation detail.
This instead makes `linux-kernel` and `gcc` top level fields in platform
configs. They join `rustc` there [all are optional], which was put there
and not in `platform` in anticipation of a change like this.
`linux-kernel.arch` in particular also becomes `linuxArch`, to match the
other `*Arch`es.
The next step after is this to combine the *specific* machines from
`lib.systems.platforms` with `lib.systems.examples`, keeping just the
"multiplatform" ones for defaulting.
Minimal ISO:
1m21 -> 2m25
625M -> 617M
Plasma5 ISO:
2m45 -> 5m18
1.4G -> 1.3G
Decompression speed stays about the same. It's just a few seconds for the whole
image anyways and, with that kind of speed, you're going to be bottlenecked by
IO long before the CPU.
It's been 8.5 years since NixOS used mingetty, but the option was
never renamed (despite the file definining the module being renamed in
9f5051b76c ("Rename mingetty module to agetty")).
I've chosen to rename it to services.getty here, rather than
services.agetty, because getty is implemantation-neutral and also the
name of the unit that is generated.
As per the in-line comment, this is where distros should configure it.
Not via kernel command line parameters.
As found by looking at the implementation, while exploring the cause of
a bug on the Raspberry Pi 4, it was found that `cma=` on the command
line parameters will overwrite the values a device tree will have
configured for a given platform.
With this, the more recent 5.4 vendor kernel boots just fine on the
Raspberry Pi 4 using our common configuration.
This includes setting up everything for the mainline Raspberry Pi 4
image.
In fact, the only difference left in the Raspberry Pi 4-specific image
is the kernel from the vendor.
Prior to this commit, installation over serial console would requiring
manually having to modify the kernel modeline, as described in
https://github.com/NixOS/nixpkgs/issues/58198 .
This is unnecessarily fiddly, so this commit adds a syslinux boot
entry that has serial enabled.
GRUB already has a serial console entry:
2c07a0800a/nixos/modules/installer/cd-dvd/iso-image.nix (L311-L317)
Why 115200 bps? This is already used in other places, e.g. https://github.com/NixOS/nixpkgs/pull/58196
I tested this change by building the image, booting the image, and
observing the boot process over serial:
$ cd nixos/
$ nix-build -A config.system.build.isoImage -I nixos-config=modules/installer/cd-dvd/installation-cd-minimal.nix default.nix
$ sudo cp /nix/store/arcl702c3z8xlndlvnfplq9yhixjvs9k-nixos-20.09pre-git-x86_64-linux.iso/iso/nixos-20.09pre-git-x86_64-linux.iso /dev/sdb
$ picocom -b 115200 /dev/ttyUSB0
This option can be set to disable installer tools like nixos-rebuild,
nixos-install, and nixos-generate-config (as well as more). This is
nice when a system is not expected to be rebuild or reconfigure itself
such as in a stateless PXE setup, as well as other embedded scenarios.
Note, that the system can still be updated, but it must either get
nixos-rebuild from another source, or, for embedded systems, be
upgraded by another machine like:
nix copy "$system" --to "ssh://root@<host>" && ssh "root@<host>"
"nix-env -p /nix/var/nix/profiles/system --set $system && $system/bin/switch-to-configuration switch".
Along with other options, this allows removing Perl from a closure.
For example:
{
boot.enableContainers = false;
environment.defaultPackages = [];
system.disableInstallerTools = true;
}
should not include Perl.
