Not entirely sure when it got broken this time, but when creating a VM
network with `nixos-build-vms(8)`, there are should be the following scripts:
* `$out/bin/nixos-test-driver` which drops into an interactive shell to
interactively perform test steps.
* `$out/bin/nixos-run-vms` which non-interactively starts the VMs from
the network so that one can manually play around in the VM.
The latter also starts an interactive shell for a while now which means
that it does the exact same thing as `nixos-test-driver` which is not
its purpose.
The live image is primarily used for installation so we should make
link to manual as well as other useful tools front and center,
instead of having them buried in the app drawer.
The default GNOME apps can still be found there when the ISO
is used for demonstration purposes.
The `nix.*` options, apart from options for setting up the
daemon itself, currently provide a lot of setting mappings
for the Nix daemon configuration. The scope of the mapping yields
convience, but the line where an option is considered essential
is blurry. For instance, the `extra-sandbox-paths` mapping is
provided without its primary consumer, and the corresponding
`sandbox-paths` option is also not mapped.
The current system increases the maintenance burden as maintainers have to
closely follow upstream changes. In this case, there are two state versions
of Nix which have to be maintained collectively, with different options
avaliable.
This commit aims to following the standard outlined in RFC 42[1] to
implement a structural setting pattern. The Nix configuration is encoded
at its core as key-value pairs which maps nicely to attribute sets, making
it feasible to express in the Nix language itself. Some existing options are
kept such as `buildMachines` and `registry` which present a simplified interface
to managing the respective settings. The interface is exposed as `nix.settings`.
Legacy configurations are mapped to their corresponding options under `nix.settings`
for backwards compatibility.
Various options settings in other nixos modules and relevant tests have been
updated to use structural setting for consistency.
The generation and validation of the configration file has been modified to
use `writeTextFile` instead of `runCommand` for clarity. Note that validation
is now mandatory as strict checking of options has been pushed down to the
derivation level due to freeformType consuming unmatched options. Furthermore,
validation can not occur when cross-compiling due to current limitations.
A new option `publicHostKey` was added to the `buildMachines`
submodule corresponding to the base64 encoded public host key settings
exposed in the builder syntax. The build machine generation was subsequently
rewritten to use `concatStringsSep` for better performance by grouping
concatenations.
[1] - https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md
since fc614c37c6 nixos needs access to its
own path (<nixpkgs/nixos>) to evaluate a system with documentation.
since documentation is enabled by default almost all systems need such
access, including the installer tests. nixos-install however does not
ensure that a channel exists in the target store before evaluating the
system in that store, which can lead to `path is not valid` errors.
`mktemp` tries to use the `TMPDIR` from `nixos-install` outside of the
`chroot` instead of `/tmp` inside the `chroot` and fails. For some
reason the `TMPDIR` is being passed through the `chroot` call.
I haven't tested if other environment variables are being passed through
that shouldn't be.
This commit encapsulates the involved domain into classes and
defines explicit and typed arguments where untyped dicts where used.
It preserves backwards compatibility through legacy wrappers.
so the underlaying use case of the preceding commit is so
generic, that we gain a lot in reasoning to give it an
appropriate name.
As the comment states:
image media needs to override host config short of mkForce
https://github.com/NixOS/nixpkgs/pull/131760 was made to avo
a speicific configuration conflict that errored out for multiple definitions of "/" when the installer where overlayed
on any existing host configuration.
---
Problem 1: It turns out that in also other mountpoints can coflict.
Solution 1: use `mkOverride 60` for all mountpoints (even for the ones unlikely causing confilct for consistency sake)
---
Problem 2: It turns out that on an installation media for a fresh machine (before formatting), we usually don't have any devices yet formatted. However defining for example `fileSystems.<nme>.device = "/dev/disk/by-label/...", in newer versions of nixos, seems to make the system startup fail. Similarily waiting for a non-existent swap device does not make the startup fail, but has a 1:30 min timeout.
Solution 2: For an installation medium, soft-override ("unless users know what they are doing") the entire `fileSystems` and `swapDevices` definitions.
installer media can be used on top of existing host configs. In such
scenarions, root fs types will already be defined.
Before this change, this will inevitably lead to the following error:
```console
error: The option `fileSystems./.fsType' has conflicting definition values:
- In `/nix/store/2nl5cl4mf6vnldpbxhrbzfh0n8rsv9fm-source/DevOS/os/hardware/common.nix': "ext4"
- In `/nix/store/jbch90yqx6gg1h3fq30jjj2b6h6jfjgs-source/nixos/modules/installer/cd-dvd/iso-image.nix': "tmpfs"
```
With this patch, the installers will override those values according to
their own local requirement.
Use `mkOverride 60` so that conscientious overriding specially targeted
at the installer, e.g. with `mkForce` is still straight forward.
Different boards using u-boot SPL require to write to different
locations. Sometimes, the 8MiB gap isn't sufficient - rk3399 boards
write to 0x16384 for example, which is at 8MiB, thus overriding the
fat32 partition with the SPL.
This should help in rare hardware-specific situations where the root is
not automatically detected properly.
We search using a marker file. This should help some weird UEFI setups
where the root is set to `(hd0,msdos2)` by default.
Defaulting to `(hd0)` by looking for the ESP **will break themeing**. It
is unclear why, but files in `(hd0,msdos2)` are not all present as they
should be.
This also fixes an issue introduced with cb5c4fcd3c
where rEFInd stopped booting in many cases. This is because it ended up
using (hd0) rather than using the `search` which was happening
beforehand, which in turn uses (hd0,msdos2), which is the ESP.
Putting back the `search` here fixes that.
This technically changes nothing. In practice `$root` is always the
"CWD", whether searched for automatically or not.
But this serves to announce we are relying on `$root`... I guess...
The value of du output depends on the underlying file system, and thus is not fully deterministic. This workaround rounds up the disk usage size to the nearest multiple of 1MB, to increase the probability that two du output values on two different file systems fall within the same 1MB window. Note that this workaround won't make du output 100% reproducible, but will increase the probability of getting deterministic builds across different file systems.
mcopy file globbing is non-deterministic with respect to the underlying file
system. As a result, the current mcopy approach is less likely to reproduce
efi.img on different machines. We replace mcopy file globbing with
fixed-order mmd and mcopy operations for better determinism. We also use
faketime on mmd for the same reason. We use faketime, mmd, and mcopy
directly, becase they are already in PATH.
Thank misuzu@ for the feedback.
This reverts commit 6f6b2cdc98.
Version wasn't updated, and apparently a patch didn't apply. Let's do
this upgrade properly, in a PR, but for now I'm reverting so we don't
have a broken nix package in master.
Looks like GRUB has issues loading EFI binaries from (cd0), which is
what would be used in e.g. qemu with OVMF with `-cdrom`. Apparently also
what is used with AArch64 + U-Boot USB.
The serial output (but it's named console, not serial actually) causes
issues on U-Boot's EFI, at the very least.
This is inspired by OpenSUSE's approach:
* https://build.opensuse.org/package/view_file/Base:System/grub2/grub2-SUSE-Add-the-t-hotkey.patch
Where they add a hidden menu entry, which can be used to force the
console output.
The `echo` will be visible on the serial terminal (grub "console"),
while the graphical interface is shown. Note that input in the serial
terminal (grub "console") will continue controlling the graphical
interface. Useful if you have an SBC connectedinto an HDMI monitor, but
no keyboard connected to it.
This is supeer useful to allow the normal sd-image code to be used by
someone who wants to setup multiple partitions with a sd-image.
Currently I'm manually copying the sd-image file and modifying it
instead.
Since 03eaa48 added perl.withPackages, there is a canonical way to
create a perl interpreter from a list of libraries, for use in script
shebangs or generic build inputs. This method is declarative (what we
are doing is clear), produces short shebangs[1] and needs not to wrap
existing scripts.
Unfortunately there are a few exceptions that I've found:
1. Scripts that are calling perl with the -T switch. This makes perl
ignore PERL5LIB, which is what perl.withPackages is using to inform
the interpreter of the library paths.
2. Perl packages that depends on libraries in their own path. This
is not possible because perl.withPackages works at build time. The
workaround is to add `-I $out/${perl.libPrefix}` to the shebang.
In all other cases I propose to switch to perl.withPackages.
[1]: https://lwn.net/Articles/779997/
It was introduced in c10fe14 but removed in c4f910f.
It remained such that people with older generations in their boot
entries could still boot those. Given that the parameter hasn't had any
use in quite some years, it seems safe to remove now.
Fixes#60184
The `platform` field is pointless nesting: it's just stuff that happens
to be defined together, and that should be an implementation detail.
This instead makes `linux-kernel` and `gcc` top level fields in platform
configs. They join `rustc` there [all are optional], which was put there
and not in `platform` in anticipation of a change like this.
`linux-kernel.arch` in particular also becomes `linuxArch`, to match the
other `*Arch`es.
The next step after is this to combine the *specific* machines from
`lib.systems.platforms` with `lib.systems.examples`, keeping just the
"multiplatform" ones for defaulting.
Minimal ISO:
1m21 -> 2m25
625M -> 617M
Plasma5 ISO:
2m45 -> 5m18
1.4G -> 1.3G
Decompression speed stays about the same. It's just a few seconds for the whole
image anyways and, with that kind of speed, you're going to be bottlenecked by
IO long before the CPU.
It's been 8.5 years since NixOS used mingetty, but the option was
never renamed (despite the file definining the module being renamed in
9f5051b76c ("Rename mingetty module to agetty")).
I've chosen to rename it to services.getty here, rather than
services.agetty, because getty is implemantation-neutral and also the
name of the unit that is generated.
As per the in-line comment, this is where distros should configure it.
Not via kernel command line parameters.
As found by looking at the implementation, while exploring the cause of
a bug on the Raspberry Pi 4, it was found that `cma=` on the command
line parameters will overwrite the values a device tree will have
configured for a given platform.
With this, the more recent 5.4 vendor kernel boots just fine on the
Raspberry Pi 4 using our common configuration.
This includes setting up everything for the mainline Raspberry Pi 4
image.
In fact, the only difference left in the Raspberry Pi 4-specific image
is the kernel from the vendor.
Prior to this commit, installation over serial console would requiring
manually having to modify the kernel modeline, as described in
https://github.com/NixOS/nixpkgs/issues/58198 .
This is unnecessarily fiddly, so this commit adds a syslinux boot
entry that has serial enabled.
GRUB already has a serial console entry:
2c07a0800a/nixos/modules/installer/cd-dvd/iso-image.nix (L311-L317)
Why 115200 bps? This is already used in other places, e.g. https://github.com/NixOS/nixpkgs/pull/58196
I tested this change by building the image, booting the image, and
observing the boot process over serial:
$ cd nixos/
$ nix-build -A config.system.build.isoImage -I nixos-config=modules/installer/cd-dvd/installation-cd-minimal.nix default.nix
$ sudo cp /nix/store/arcl702c3z8xlndlvnfplq9yhixjvs9k-nixos-20.09pre-git-x86_64-linux.iso/iso/nixos-20.09pre-git-x86_64-linux.iso /dev/sdb
$ picocom -b 115200 /dev/ttyUSB0
This option can be set to disable installer tools like nixos-rebuild,
nixos-install, and nixos-generate-config (as well as more). This is
nice when a system is not expected to be rebuild or reconfigure itself
such as in a stateless PXE setup, as well as other embedded scenarios.
Note, that the system can still be updated, but it must either get
nixos-rebuild from another source, or, for embedded systems, be
upgraded by another machine like:
nix copy "$system" --to "ssh://root@<host>" && ssh "root@<host>"
"nix-env -p /nix/var/nix/profiles/system --set $system && $system/bin/switch-to-configuration switch".
Along with other options, this allows removing Perl from a closure.
For example:
{
boot.enableContainers = false;
environment.defaultPackages = [];
system.disableInstallerTools = true;
}
should not include Perl.
This reverts commit aab534b894 & uses the
driverInteractive attribute for the test driver instead.
This has the same effect but removes the extra module in the
nixos-build-vms code.
When I test a change e.g. in the module system manually, I usually use
`nixos-build-vms(8)` which also gives me a QEMU window where I can play
around in the freshly built VM.
It seems as this has changed recently when the default package for
non-interactive VM tests using the same framework was switched to
`pkgs.qemu_test` to reduce the closure size. While this is a reasonable
decision for our CI tests, I think that you really want a QEMU window of
the VM by default when using `nixos-build-vms(8)`.
[1] bc2188b083
The only nix version available in the installer should be the version
configure in the module system. If someone needs `nixUnstable` in their
`nixos-install` they should probably set the module option and not just
add it to the closure.
$EDITOR is allowed to contain flags, so it is important to allow the
shell to split this normally. For example, Sublime Text needs to be
passed --wait, since otherwise it will daemonise.
$NIXOS_CONFIG can be set to a directory, in which case the file used
is $NIXOS_CONFIG/default.nix. This updates 'nixos-rebuild edit' to
handle that case correctly.
This allows to perform `dd if= of=$img` after the image is built
which is handy to add e.g. uBoot SPL to the built image.
Instructions for some ARM boards sometimes contain this step
that needs to be performed manually, with this patch it can be
part of the nix file used to built the image.
The nix store more-or-less requires o+rx on all parent directories.
This is primarily because nix runs builders in a uid/gid mapped
user-namespace, and those builders have to be able to operate on the nix
store.
This check is especially helpful because nix does not produce a helpful
error on its own (rather, creating directories and such works, it's not
until 'mount --bind' that it gets an EACCES).
Helps users who run into this opaque error, such as in #67465.
Possibly fixes that issue if bad permissions were the only cause.
This should have been done initially, as otherwise it gets awfully
awkward to boot into new generations by default.
This system-specific image wasn't expected to be long-lived, thus why it
didn't end up being polished much.
Reality shows us we may be stuck with it for a bit longer, so let's make
it easier to use for new users.
'nix build' is an experimental command so we shouldn't use it
yet. (nixos-rebuild also uses 'nix', but only when using flakes, which
are themselves an experimental feature.)
nix build should store it's temporary files on target filesystem.
This should fix 'No space left on device' on systems
with low amount of RAM when there is a need to build something
like Linux kernel
For imports, it is better to use ‘modulesPath’ than rely on <nixpkgs>
being correctly set. Some users may not have <nixpkgs> set correctly.
In addition, when ‘pure-eval=true’, <nixpkgs> is unset.
When trying to build a VM using `nixos-build-vms` with a configuration
that doesn't evaluate, an error "at `<unknown-file>`" is usually shown.
This happens since the `build-vms.nix` creates a VM-network of
NixOS-configurations that are attr-sets or functions and don't contain
any file information. This patch manually adds the `_file`-attribute to
tell the module-system which file contained broken configuration:
```
$ cat vm.nix
{ vm.invalid-option = 1; }
$ nixos-build-vms vm.nix
error: The option `invalid-option' defined in `/home/ma27/Projects/nixpkgs/vm.nix@node-vm' does not exist.
(use '--show-trace' to show detailed location information)
```
we use stdenv.hostPlatform.uname.processor, which I believe is just like
`uname -p`.
Example values:
```
(import <nixpkgs> { system = "x86_64-linux"; }).stdenv.hostPlatform.uname.processor
"x86_64"
(import <nixpkgs> { system = "aarch64-linux"; }).stdenv.hostPlatform.uname.processor
aarch64
(import <nixpkgs> { system = "armv7l-linux"; }).stdenv.hostPlatform.uname.processor
"armv7l"
```
Instead of making the configuration less portable by hard coding the number of
jobs equal to the cores we can also let nix set the same number at runtime.
The volumeID will now be in the format of:
nixos-$EDITON-$RELEASE-$ARCH
an example for the minimal image would look like:
nixos-minimal-20.09-x86-64-linux
Note we're not using wayland default in the graphical media because it
could cause headaches for Nvidia users. But the session is still available
if someone logs out.
In 0945178b3c we decided that Perl-based
VM tests should be deprecated and will be removed between 20.03 and
20.09. So let's switch `nixos-build-vms(8)` to python as well (which is
entirely interactive, so other scripts won't break).
In my experience, the test-driver isn't used most of the time, so this
patch is mainly supposed to get rid of the (probably misleading)
deprecation warning when running `nixos-build-vms`. Apart from that, the
interface for python's test-driver is way nicer.
In some cases, /dev/stderr may not point to a sensible location. For
example, running nixos-enter inside a systemd unit where the unit's
StandardOutput and StandardError are set to be sockets. In these
cases, this line would fail.
Piping to fd2 directly works just as well, even under strange and
twisted executions.
Co-authored-by: Michael Bishop <michael.bishop@iohk.io>
"master" is not a valid SHA-1 commit hash, and it's not even
necessarily the branch used. 'nixos-version --revision' now returns an
error if the commit hash is not known.
In 87a19e9048 I merged staging-next into master using the GitHub gui as intended.
In ac241fb7a5 I merged master into staging-next for the next staging cycle, however, I accidentally pushed it to master.
Thinking this may cause trouble, I reverted it in 0be87c7979. This was however wrong, as it "removed" master.
This reverts commit 0be87c7979.
I merged master into staging-next but accidentally pushed it to master.
This should get us back to 87a19e9048.
This reverts commit ac241fb7a5, reversing
changes made to 76a439239e.
When installing a fresh NixOS system it occasionally happens that you
encounter issues that are rather hard to track down since
`nixos-install(8)` doesn't provide any debugging flags.
This patch adds `-L` to force `nix build` to display the build-log on
stderr and `-v` to increase the log-level of Nix.
1. This makes aggregates of submodules (including the very important
"nixos-option users.users.<username>" case) behave the same way as any
other you-need-to-keep-typing-to-get-to-an-option-leaf (eg:
"nixos-option environment").
Before e0780c5:
$ nixos-option users.users.root
error: At 'root' in path 'users.users.root': Attribute not found
An error occurred while looking for attribute names. Are you sure that 'users.users.root' exists?
After e0780c5 but before this change, this query just printed out a raw
thing, which is behavior that belongs in "nix eval", "nix-instantiate
--eval", or "nix repl <<<":
$ nixos-option users.users.root
{
_module = {
args = { name = "root"; };
check = true;
};
createHome = false;
cryptHomeLuks = null;
description = "System administrator";
...
After this change:
$ nixos-option users.users.root
This attribute set contains:
createHome
cryptHomeLuks
description
extraGroups
group
hashedPassword
...
2. For aggregates of other types (not submodules), print out the option
that contains them rather than printing an error message.
Before:
$ nixos-option environment.shellAliases.l
error: At 'l' in path 'environment.shellAliases.l': Attribute not found
An error occurred while looking for attribute names. Are you sure that 'environment.shellAliases.l' exists?
After:
$ nixos-option environment.shellAliases.l
Note: showing environment.shellAliases instead of environment.shellAliases.l
Value:
{
l = "ls -alh";
ll = "ls -l";
ls = "ls --color=tty";
}
...
When running e.g. `nixos-option users.users.ma27`, the evaluation breaks
since `ma27` is the attribute name in `attrsOf (submodule {})`, but not
a part of the option tree and therefore breaks with the following
errors:
```
error: At 'ma27' in path 'users.users.ma27': Attribute not found
An error occurred while looking for attribute names. Are you sure that 'users.users.ma27' exists?
```
This happens since the option evaluator expects that either the option
exists or the option is a submodule and the "next" token in the
attribute path points to an option (e.g. `users.users.ma27.createHome`).
This patch checks in the `Attribute not found` condition if the attribute-path
actually exists in the config tree. If that's true, a dummy-attrset is created
which contains `{_type = "__nixos-option-submodule-attr";}`, in that case, the
entire entry of the submodule will be displayed.
When 'grafting' '/nix/store/<hash>-loopback.cfg' from disk onto
'/boot/grub/loopback.cfg' on the iso, the parent 'grub' directory does not
exist yet. In this case it is automatically created and inherits its
attributes, including timestamp, from /nix/store.
This is correct/expected/intentional behavior of xorriso, but has the
undesired result of leaking the timestamps of /nix/store into the iso. For
this reason we put the loopback.cfg in a
'/nix/store/<hash>-loopback.cfg/grub/loopback.cfg' instead, so it will inherit
the attributes from the correctly-timestamped
'/nix/store/<hash>-loopback.cfg/grub' directory.
For the same reason we move '/EFI/boot/efi-background.png' down in the list
so it is grafted after its parent '/EFI/boot' directory is created with
the correct timestamp.
fixes#74944
Add --use-remote-sudo option. When set, remote commands will be prefixed
with 'sudo'. This allows using sudo remotely _without_ having to use
sudo locally (when using --build-host/--taget-host).
Automated consumers can use 'sed 1d' or similar to remove this header.
This probably makes this output *easier* to consume correctly. Having
this header show up in consumers' terminal or log output is probably not
useful, but hiding it without hiding all error messages would have been
more troublesome that just stripping it from stdout.
I.e., previously, unsophisticated use would show undesired output:
$ some-other-tool
This attribute set contains:
This attribute set contains:
This attribute set contains:
This attribute set contains:
<Actual some-other-tool output>
The simplest way to hide this undesired output would have been
nixos-option ... 2>/dev/null, which would hide all error messages.
We do not wish to encourage that.
Correct use would have been something like:
nixos-option ... 2> >( grep --line-buffered -v 'This attribute set contains:')
After this change, correct use is simpler:
nixos-option ... | sed 1d
or
nixos-option ... | sed '1/This attribute set contains:/d'
if the caller don't know if this invocation of nixos-option will yield
an attribute listing or an option description.
Switch from convention "appease clang-tidy --checks='*'" to
"References are like non-nullptr pointers". The clang-tidy check
"google-runtime-references" complains about non-const reference
arguments, but this is not a convention used in Nix.
Switch from convention "appease clang-tidy --checks='*'" to
"References are like non-nullptr pointers". The clang-tidy check
"google-runtime-references" complains about non-const reference
arguments, but this is not a convention used in Nix.
