Commit Graph

350 Commits

Author SHA1 Message Date
Janne Heß
bc9b484db8
Merge pull request #232230 from oddlama/fix-activation-template-unit-specializations
Fix detection of changed template unit specializations in switch-to-configuration.pl
2023-08-16 13:46:45 +02:00
Janne Heß
d995da11d8
nixos/switch-to-configuration: Fix unit location in toplevel 2023-08-13 14:26:30 +02:00
Vincent Haupert
fbebce610f nixos/toplevel: move systemBuilder for boot.kernel.enable
This avoids creating a build-time reference on `boot.kernelParams` if
the configuration does not use a kernel, i.e., `boot.kernel.enable` is
set to `false`.
2023-08-04 12:46:03 +02:00
oddlama
d073105d6b
nixos/switch-to-configuration: fix ignoring of template unit specialization dropins 2023-07-23 13:16:58 +02:00
Robert Hensing
3fd4ac8e82
Merge pull request #237040 from roberth/flexible-activation
nixos/system: Support pre-activated images
2023-07-08 16:06:25 +02:00
Robert Hensing
772d6076e8 nixos: Add system.activatable flag for images that are pre-activated 2023-06-28 14:06:28 +02:00
Robert Hensing
9edad17d29 nixos/top-level: Remove unused builder variables
These variables were previously used by the activation script
build commands, but are now embedded into those commands for
to improve reusability for an upcoming addition.
2023-06-28 12:49:34 +02:00
Robert Hensing
7891c8cdaf nixos/activatable-system: Move legacy variables to top-level 2023-06-28 12:48:59 +02:00
Robert Hensing
990b72f6af nixos/activatable-system: Make system builder commands env independent
This way it will be easier to reuse in a different context, such as
a separate build of the activation script by itself (TBD).
2023-06-28 12:42:51 +02:00
Naïm Favier
0179d9f7e6
nixos/top-level: change extraDependencies from package to pathInStore
Allows adding subdirectory flake inputs that aren't top-level store paths.
2023-06-27 11:48:08 +02:00
Wout Mertens
a97fe8990d
switch-to-configuration.pl: fix inverted dry-activate logic for swap 2023-06-14 12:11:45 +02:00
Robert Hensing
193f4fea90 nixos/activatable-system: Make substitutions explicit
This helps with understanding the code.
We might make this not depend on environment variables later.
systemBuilderArgs is a form of global state, which isn't helpful.
2023-06-10 19:15:00 +02:00
Robert Hensing
a16986f1a3 nixos: Move installBootLoader to activation script modules 2023-06-10 19:15:00 +02:00
Robert Hensing
61d43dee5f nixos: Extract module for activation script inclusion into toplevel
Allows omission of this functionality through disabledModules, e.g.
for image building.
2023-06-10 19:15:00 +02:00
Victor Engmark
8deaa732a8 refactor: Split mkdir -m … into mkdir + chmod
As recommended by ShellCheck
<https://www.shellcheck.net/wiki/SC2174>.
2023-06-08 15:55:40 +12:00
Bjørn Forsman
ef85c3fe51 nixos: use passAsFile to avoid "Argument list too long" error
This patch fixes "Argument list too long" build failure when passing a
list of store paths to system.extraDependencies that exceeds Linux'
MAX_ARG_STRLEN limit of 128 KiB. With the shortest possible derivation
names (one byte), the 128 KiB limit is equivalent to about 2850
derivations. With longer derivations names, the limit is hit earlier.
Fix this restriction.
2023-05-19 22:31:31 +02:00
Robert Hensing
5bdf63819b nixos/top-level.nix: Add system.checks
Note that this does not add to the `forbiddenDependenciesRegex`
code because that code check should be unaffected as it only checks
output dependencies, not build dependencies.
Build deps are added after that check, if those are enabled in the
first place.
2023-05-11 21:18:38 +02:00
Jared Baur
67561dcc59
nixos/bootspec: Fix cross for nixos-system systemBuilder
When `nixpkgs.hostPlatform` != `nixpkgs.buildPlatform`, building the
top-level attribute fails since the bootspec portion of the system
builder tries to reference the host platform's `jq`. Change this to
reference the build platform's `jq`.
2023-05-04 20:09:16 -07:00
Raito Bezarius
bc502d0a14 nixos/bootspec: adopt the merged RFC-0125
This removes the feature preview warning, enable by default bootspec,
adds a validation flag to prevent Go to go into build-time closure.

This will break all downstream users of bootspec as those changes are
not backward-compatible.
2023-04-28 18:26:11 -07:00
K900
48b2f723f5 nixos/system: disallow system.copySystemConfiguration in pure eval mode
This is already broken, so let's give it a better error message.
2023-03-29 17:49:30 +03:00
Linus Heckemann
eb45cd5108 nixos/top-level: add includeBuildDependencies option
This option allows adding the build closure of the system to its
runtime closure, enabling fully-offline rebuilds (as long as no new
packages are added).
2023-03-14 11:55:13 +01:00
Victor Fuentes
d3528cdc3d
nixos/version: add config.system.nixos.distroName and config.system.nixos.distroId 2023-01-14 16:19:06 -05:00
Ryan Lahfa
e2e8dfcf44
Merge pull request #207163 from NixOS/rfc0125-improvements
nixos/activation/bootspec: make initrd optional, serialize system, precise extensions' type
2022-12-23 00:43:29 +01:00
Cole Helbling
21f4afd799 nixos/activation/bootspec: fix document output path
The RFC currently stipulates the document will be available at
`$out/boot.json`.
2022-12-22 07:56:09 -08:00
Raito Bezarius
5dd2e60371 nixos/activation/bootspec: make initrd optional, serialize system, precise extensions' type 2022-12-21 23:28:21 +01:00
Jörg Thalheim
668a2b2f33
Merge pull request #172237 from DeterminateSystems/bootspec-rfc
Support external bootloader backends (RFC-0125)
2022-12-17 08:35:53 +00:00
Cole Helbling
5af481f67f nixos/activation/bootspec: fixup improper $out substitution 2022-12-08 13:50:05 -08:00
Cole Helbling
dce9add02b nixos/activation/bootspec: refactor the generator script
We separate the different steps (injecting the toplevel and injecting
the specialisations) so that it's easy to document what each snippet is
actually doing.
2022-12-08 13:50:05 -08:00
Cole Helbling
38e5089814 nixos/activation/bootspec: drop problematic comment, only generate bootspec when bootspec is enabled 2022-12-08 13:50:05 -08:00
Cole Helbling
97f657c742 nixos/activation/bootspec: DocBook -> Markdown, add description for extensions field 2022-12-08 13:50:05 -08:00
Raito Bezarius
680369e504 nixos/activation/bootspec: add some comments to explain the delicate manipulations 2022-12-08 13:50:05 -08:00
Raito Bezarius
11dfbee0a4 nixos/activation/bootspec: add bootspec chapter in NixOS manual 2022-12-08 13:50:05 -08:00
Raito Bezarius
9832e3e9b9 nixos/activation/bootspec: remove SB extension example in Cue schema 2022-12-08 13:50:04 -08:00
Raito Bezarius
348ba1b33c nixos/activation/bootspec: module-ify
This does the following:

* turns bootspec into a NixOS module
* validates bootspecs with Cue
* exposes internal knobs
2022-12-08 13:50:04 -08:00
Raito Bezarius
ee27291b34 nixos/activation/bootspec: fix slurping specialisation bootspecs 2022-12-08 13:50:04 -08:00
K900
e69c37eae9 nixos/activation: don't generate bootspec for containers 2022-12-08 13:50:04 -08:00
Graham Christensen
e9c85d6d0f nixos/activation/bootspec: embed the document into a bootspec subdir 2022-12-08 13:50:04 -08:00
Graham Christensen
6c0e4e892f nixos/activation/bootspec: embed the entire contents of specialisation's bootspecs into the parent
See: https://github.com/NixOS/rfcs/pull/125#discussion_r871222614
2022-12-08 13:50:04 -08:00
Cole Helbling
942dcd238b nixos/activation/bootspec: init bootspec support (RFC-0125) 2022-12-08 13:50:03 -08:00
Artturin
a34d7b67fd nixos/top-level.nix: add forbiddenDependenciesRegex option
useful for making sure that there's no dev outputs in the system
2022-12-02 21:15:24 +02:00
Sandro
9477fa1e44
Merge pull request #197917 from SuperSandro2000/kernel-enable 2022-11-12 21:12:56 +01:00
Sandro Jäckel
182575a60d
nixos/kernel: replace boot.isContainer with boot.kernel.enable 2022-11-09 23:45:33 +01:00
Robert Hensing
9299483604 nixos/top-level.nix: Move configurationName to grub.nix
The configuration-name file is grub specific, so it should not be
in top-level.nix.
2022-11-05 00:25:46 +01:00
Robert Hensing
37fa46a224 nixos/top-level.nix: Remove workaround for #156533 2022-11-05 00:25:46 +01:00
Robert Hensing
0b05ed2c78 nixos/specialisation.nix: Extract module 2022-11-05 00:25:46 +01:00
Robert Hensing
d3ac0938a7 nixos/top-level.nix: Make extensible
(cherry picked from commit 4ec415cff9)
2022-11-05 00:05:26 +01:00
pennae
1d41cff3dc nixos/*: convert straggler options to MD 2022-08-31 17:27:38 +02:00
pennae
f2ea09ecbe nixos/*: convert options with listings
minor rendering changes.
2022-08-31 17:27:36 +02:00
pennae
bd56368848 nixos/*: md-convert hidden plaintext options
most of these are hidden because they're either part of a submodule that
doesn't have its type rendered (eg because the submodule type is used in
an either type) or because they are explicitly hidden. some of them are
merely hidden from nix-doc-munge by how their option is put together.
2022-08-31 16:32:54 +02:00
pennae
9547123258 nixos/*: convert internal option descriptions to MD
we'll have to do it eventually, may as well be now.
2022-08-31 16:32:54 +02:00
pennae
0cfcb5908c nixos/*: <screen> -> <programlisting>
most of the screen tags used in option docs are actually listings of
some sort. nsd had a notable exception where its screen usage was pretty
much a raw markdown block that made most sense to convert into docbook lists.
2022-08-31 16:27:24 +02:00
pennae
51a11254a7 nixos/*: literalDocBook -> literalMD
no change to rendered output
2022-08-27 19:18:29 +02:00
Stig
8c4a3b279b
Merge pull request #186503 from Weathercold/patch-1
nixos/switch-to-configuration: fix units starting with dash not being able to reload
2022-08-21 17:31:52 +02:00
pennae
e4ed177f82 nixos/* eliminate inner whitespace in tags that was missed earlier
nix-doc-munge won't match tags that contain newlines anywhere. most of
these have already been removed, but a few obviously made it through.
2022-08-19 22:40:58 +02:00
Weathercold
8fb63401af
nixos/switch-to-configuration: fix units starting with dash
Fix units starting with `-` being recognized as options.
2022-08-12 18:36:15 -04:00
pennae
50f9b3107a
Merge pull request #185474 from pennae/option-docs-md
nixos/*: more options md conversion
2022-08-12 23:23:26 +02:00
Janne Heß
1120326a5b
Merge pull request #185067 from stigtsp/refactor/stc-no-net-debus
nixos/switch-to-configuration: replace Net::DBus with busctl
2022-08-09 20:28:49 +02:00
pennae
087472b1e5 nixos/*: automatically convert option docs 2022-08-06 20:39:12 +02:00
Stig Palmquist
4c2764c69c
nixos/switch-to-configuration: replace Net::DBus with busctl
Call dbus by using `$cur_systemd/busctl --json=...` and core modules
JSON::PP and IPC::Cmd to slim down dependencies for baseSystem.

perlPackages.NetDBus pulls in quite a few other dependencies, like
XML::Twig, LWP, and HTTP::Daemon. These are not really neccecary for
s-t-c, and some of them have caused issues particularly with cross
builds after updates to perlPackages.
2022-08-04 00:08:26 +02:00
pennae
694d5b19d3 nixos/*: replace </para><para> with double linebreaks
our xslt already replaces double line breaks with a paragraph close and
reopen. not using explicit para tags lets nix-doc-munge convert more
descriptions losslessly.

only whitespace changes to generated documents, except for two
strongswan options gaining paragraph two breaks they arguably should've
had anyway.
2022-08-03 20:39:21 +02:00
pennae
2e751c0772 treewide: automatically md-convert option descriptions
the conversion procedure is simple:

 - find all things that look like options, ie calls to either `mkOption`
   or `lib.mkOption` that take an attrset. remember the attrset as the
   option
 - for all options, find a `description` attribute who's value is not a
   call to `mdDoc` or `lib.mdDoc`
 - textually convert the entire value of the attribute to MD with a few
   simple regexes (the set from mdize-module.sh)
 - if the change produced a change in the manual output, discard
 - if the change kept the manual unchanged, add some text to the
   description to make sure we've actually found an option. if the
   manual changes this time, keep the converted description

this procedure converts 80% of nixos options to markdown. around 2000
options remain to be inspected, but most of those fail the "does not
change the manual output check": currently the MD conversion process
does not faithfully convert docbook tags like <code> and <package>, so
any option using such tags will not be converted at all.
2022-07-30 15:16:34 +02:00
Sandro
b4e7303ddf
nixos/specialisation: fix curly brackets 2022-04-17 05:48:33 +02:00
Janne Heß
0581d31bf9
Merge pull request #167393 from iblech/patch-docs-cc
nixos-rebuild, switch-to-configuration: document and protect against cross compilation subtlety
2022-04-14 12:30:39 +02:00
Janne Heß
ffb320378b
nixos/stage-1-systemd: Fix booting grub tests 2022-04-13 23:02:10 +01:00
Janne Heß
3df2691e6b
nixos/stage-1-systemd: Handover between the systemds directly 2022-04-11 20:04:33 +01:00
Ingo Blechschmidt
c3428419e0 nixos/switch-to-configuration: Provider better error message in cross-compiling situations 2022-04-05 16:52:09 +02:00
Janne Heß
85874efcb0
nixos/switch-to-configuration: Make perlcritic happy 2022-03-20 13:04:23 +01:00
Janne Heß
9c494b5773
nixos/switch-to-configuration: Snake-case all variables
Also add a lot of comments and reorder some assignments
2022-03-20 13:03:39 +01:00
Janne Heß
67f84b4b87
nixos/switch-to-configuration: Snake-case all subroutines and add comments 2022-03-20 13:03:19 +01:00
Janne Heß
23ea9965bb
nixos/switch-to-configuration: Enforce consistent quoting 2022-03-20 13:01:11 +01:00
Janne Heß
bdcd558812
nixos/switch-to-configuration: Get rid of all postfixes and unlesses 2022-03-20 13:01:11 +01:00
Naïm Favier
9bdd2f852c
nixos/switch-to-configuration: fix installBootLoader escaping
Use a quoted heredoc to inject installBootLoader safely into the script,
and restore the previous invocation of `system` with a single argument so
that shell commands keep working.
2022-03-14 02:12:52 +01:00
Janne Heß
461c1c9e86
nixos/switch-to-configuration: Use parenthesis on all calls 2022-03-11 14:05:20 +01:00
Janne Heß
bc58430068
nixos/switch-to-configuration: Fix reloading of stopped services 2022-03-11 14:05:19 +01:00
Janne Heß
3052d3aa50
nixos/switch-to-configuration: Fix restarting by activation script
This bug is so obscure and unlikely that I was honestly not able to
properly write a test for it. What happens is that we are calling
handleModifiedUnit() with $unitsToStart=\%unitsToRestart. We do this to
make sure that the unit is stopped before it's started again which is
not possible by regular means because the stop phase is already done
when calling the activation script.

recordUnit() still gets $startListFile, however which is the wrong file.
The bug would be triggered if an activation script requests a service
restart for a service that has `stopIfChanged = true` and
switch-to-configuration is killed before the restart phase was run. If
the script is run again, but the activation script is not requesting
more restarts, the unit would be started instead of restarted.
2022-03-11 13:30:03 +01:00
Janne Heß
c96180c53f
nixos/switch-to-configuration: Ignore some unit keys
Some unit keys don't need to restart the service to make them effective.
Reduce the amount of service restarts by ignoring these keys
2022-03-11 13:30:03 +01:00
Peter Hoeg
d44916d12f nixos/activation: allowed quoted values in /etc/os-release 2022-03-09 17:07:48 +08:00
Janne Heß
1def557525
nixos/switch-to-configuration: Document and test socket-activated services 2022-03-03 20:49:20 +01:00
Janne Heß
3617ecb67f
nixos/switch-to-configuration: Fix backslashes in unit names
systemd needs this so special characters (like the ones in wireguard
units that appear because they are part of base64) can be escaped using
the \x syntax.

Root of the issue is that `glob()` handles the backslash internally
which is obviously not what we want here.

Also add a test case and fix some perlcritic issues in the subroutine.
2022-02-17 12:49:45 +01:00
Stig Palmquist
3d713efe41 nixos/switch-to-configuration: avoid Array::Compare dependency
Replace Array::Compare with a simple function, since Array::Compare
pulls down Module::Build which breaks cross compilation.
2022-02-15 15:37:37 +01:00
Janne Heß
08cd8ab8b6
nixos/switch-to-configuration: Don't stop swaps in dry-activate 2022-02-09 15:14:38 +01:00
Janne Heß
b9bb1de341
nixos/switch-to-configuration: Implement reload support
This is accomplished by comparing the hashes that the unit files
contain. By filtering for a special key `X-Reload-Triggers` in the
`[Unit]` section, we can differentiate between reloads and restarts.

Since activation scripts can request reloads of units as well, more
checking of this behaviour is implemented. If a unit is to be restarted,
it's never reloaded as well which would make no sense.

Also removes a useless subroutine and perl dependencies that are
nowadays handled by the propagated build inputs feature of
`perl.withPackages`.
2022-02-09 14:31:45 +01:00
Janne Heß
78db7b6529
nixos/switch-to-configuration: Allow passing parsed unit contents 2022-02-09 14:31:44 +01:00
Janne Heß
d729cc8a53
nixos/switch-to-configuration: Skip [Install] section 2022-02-09 14:31:44 +01:00
Bernardo Meurer
5f9b470ff0
Merge pull request #154809 from helsinki-systems/feat/stc-proper-unit-file-parser
nixos/switch-to-configuration: Proper unit file parser and clean/fix lower part of the script
2022-01-27 09:35:34 -08:00
polykernel
4a9d9928dc nixos/nix-daemon: use structural settings
The `nix.*` options, apart from options for setting up the
daemon itself, currently provide a lot of setting mappings
for the Nix daemon configuration. The scope of the mapping yields
convience, but the line where an option is considered essential
is blurry. For instance, the `extra-sandbox-paths` mapping is
provided without its primary consumer, and the corresponding
`sandbox-paths` option is also not mapped.

The current system increases the maintenance burden as maintainers have to
closely follow upstream changes. In this case, there are two state versions
of Nix which have to be maintained collectively, with different options
avaliable.

This commit aims to following the standard outlined in RFC 42[1] to
implement a structural setting pattern. The Nix configuration is encoded
at its core as key-value pairs which maps nicely to attribute sets, making
it feasible to express in the Nix language itself. Some existing options are
kept such as `buildMachines` and `registry` which present a simplified interface
to managing the respective settings. The interface is exposed as `nix.settings`.

Legacy configurations are mapped to their corresponding options under `nix.settings`
for backwards compatibility.

Various options settings in other nixos modules and relevant tests have been
updated to use structural setting for consistency.

The generation and validation of the configration file has been modified to
use `writeTextFile` instead of `runCommand` for clarity. Note that validation
is now mandatory as strict checking of options has been pushed down to the
derivation level due to freeformType consuming unmatched options. Furthermore,
validation can not occur when cross-compiling due to current limitations.

A new option `publicHostKey` was added to the `buildMachines`
submodule corresponding to the base64 encoded public host key settings
exposed in the builder syntax. The build machine generation was subsequently
rewritten to use `concatStringsSep` for better performance by grouping
concatenations.

[1] - https://github.com/NixOS/rfcs/blob/master/rfcs/0042-config-option.md
2022-01-26 21:04:50 -05:00
Martin Weinelt
48f17360d9 Merge remote-tracking branch 'origin/master' into staging-next 2022-01-25 15:53:19 +01:00
Robert Hensing
4800f30841 nixos: Explain system.build.installBootLoader's odd default
I don't really approve of this solution, but documenting its
purpose was the least I could do for now.
2022-01-24 16:17:20 +01:00
Robert Hensing
511e89f5a6 nixos: Make system.build.installBootLoader a proper option
This improves the error message when the configuration contains
more than one boot loader.
2022-01-24 16:17:19 +01:00
Robert Hensing
2aa7c25808 nixos: Document system.build.toplevel 2022-01-24 16:16:46 +01:00
Robert Hensing
3ac955acf4 nixos/system/build: Extract
Modules that do not depend on e.g. toplevel should not have to include it just to set
things in `system.build`. As a general rule, this keeps tests simple, usage flexible
and evaluation fast. While one module is insignificant, consistency and good practices
are.
2022-01-24 00:47:57 +01:00
github-actions[bot]
fc3ddb8979
Merge master into staging-next 2022-01-21 00:01:42 +00:00
Naïm Favier
1147d72481
nixos: use uniq in the type of system.build
`unspecified` will happily concatenate strings together from two
unrelated modules, causing spurious errors (see #155925).
2022-01-20 23:28:09 +01:00
Alyssa Ross
b1724b2f81 nixos/activation-script: ensure gcroots dir exists
If the Nix daemon has never been enabled (nix.enable has always been
set to false), the gcroots directory won't exist.  If the Nix daemon
is later enabled, the GC roots for booted-system and current-system
will be missing, and they might end up being garbage collected.  Since
it's cheap to add GC roots even if the daemon will never be enabled,
let's just always add them so we're okay in the case where the daemon
is enabled later.
2022-01-20 20:24:32 +00:00
Janne Heß
b52372675d
nixos/switch-to-configuration: Clean up lower part of the script
- Fully get rid of `parseKeyValues` and use systemctl features for that
- Add some regex modifiers recommended by perlcritic
- Get rid of a postfix if
- Sort units when showing their status
- Clean the logic for showing what failed from `elif` to `next`
- Switch from `state` to `substate` for `auto-restart` because that's
  actually where the value is stored
- Show status of units with one single systemctl call and get rid of
  COLUMNS in favor of --full
- Add a test for failing units
2022-01-20 17:10:02 +01:00
Janne Heß
96d36b0c2e
nixos/switch-to-configuration: Proper unit file parser
This replaces the naive K=V unit parser with a proper INI parser from a
library and adds proper support for override files. Also adds a bunch of
comments about parsing, I hope this makes it easier to understand and
maintain in the future.

There are multiple reasons to do so, the first one is just general
correctness with is nice imo. But to get to more serious reasons (I
didn't put in all that effort for nothing) is that this is the first
step torwards more clever restart/reload handling. By using a library
like Data::Compare a future PR could replace the current way of
fingerprinting units (which is to compare store paths) by comparing the
hashes. This is more precise because units won't get restarted because
the order of the options change, comments are added, some dependency of
writeText changes, .... Also this allows us to add a feature like
`X-Reload-Triggers` so the unit can either be reloaded when these change
or restarted when everything else changes, giving module authors the
ability to have their services reloaded without having to fear that
updates are not applied because the service doesn't get restarted.
Another reason why this feature is nice is that now that the unit files
are parsed correctly (and values are just extracted from one section),
potential future rewrites can just rely on some INI library without
having to implement their own weird parser that is compatible with this
script.

This also comes with a new subroutine to handle systemd booleans because
I thought the current way of handling it was just ugly. This also allows
overriding values this script reads in an override file.

Apart from making this script more compatible with the world around it,
this also fixes two issues I saw bugging exactly 0 (zero) people. First
is that this script now supports multiple override files, also ones that
are not called override.conf and the second one is that `1` and `on` are
treated as bools by systemd but were previously not parsed as such by
switch-to-configuration.
2022-01-20 15:10:23 +01:00
github-actions[bot]
b456d67c98
Merge master into staging-next 2022-01-18 00:01:41 +00:00
Janne Heß
2cf157c781
nixos/switch-to-configuration: Rework activation script restarts
This removes `/run/nixos/activation-reload-list` (which we will need in
the future when reworking the reload logic) and makes
`/run/nixos/activation-restart-list` honor `restartIfChanged` and
`reloadIfChanged`. This way activation scripts don't have to bother with
choosing between reloading and restarting.
2022-01-17 17:57:23 +01:00
github-actions[bot]
d5e672b839
Merge master into staging-next 2022-01-14 18:01:18 +00:00
Robert Hensing
2bf5958169
Merge pull request #151082 from hercules-ci/nixos-cleanup-vmWithBootLoader
nixos: turn vmWithBootLoader into option (`nixos-rebuild build-vm`)
2022-01-14 18:49:27 +01:00