nixos/doc: Add chapter “necessary system state”

Suggested in the discussion of #268995.

nixos/doc/manual/administration/nixos-state.section.md

@@ -0,0 +1,13 @@
+# NixOS {#sec-nixos-state}
+
+NixOS itself obviously needs `/nix` to be persistent, so the system environment
+can be linked from the nix store during activation.
+
+Moreover, `/boot` should also be persistent, as it contains the kernel, initrd,
+and the command-line determining the system environment's path in the nix store.
+
+Lastly, `users.mutableUsers` should be false, *or* the following files under
+`/etc` should all persist:
+- {manpage}`passwd(5)` and {manpage}`group(5)`,
+- {manpage}`shadow(5)` and {manpage}`gshadow(5)`,
+- {manpage}`subuid(5)` and {manpage}`subgid(5)`.

nixos/doc/manual/administration/running.md

@@ -8,6 +8,7 @@ rebooting.chapter.md
 user-sessions.chapter.md
 control-groups.chapter.md
 logging.chapter.md
+system-state.chapter.md
 cleaning-store.chapter.md
 containers.chapter.md
 troubleshooting.chapter.md

nixos/doc/manual/administration/system-state.chapter.md

@@ -0,0 +1,17 @@
+# Necessary system state {#ch-system-state}
+
+Normally — on systems with a persistent `rootfs` — system services can persist state to
+the filesystem without administrator intervention.
+
+However, it is possible and not-uncommon to create [impermanent systems], whose
+`rootfs` is either a `tmpfs` or reset during boot. While NixOS itself supports
+this kind of configuration, special care needs to be taken.
+
+[impermanent systems]: https://nixos.wiki/wiki/Impermanence
+
+
+```{=include=} sections
+nixos-state.section.md
+systemd-state.section.md
+zfs-state.section.md
+```

nixos/doc/manual/administration/systemd-state.section.md

@@ -0,0 +1,52 @@
+# systemd {#sec-systemd-state}
+
+## `machine-id(5)` {#sec-machine-id}
+
+`systemd` uses per-machine identifier — {manpage}`machine-id(5)` — which must be
+unique and persistent; otherwise, the system journal may fail to list earlier
+boots, etc.
+
+`systemd` generates a random `machine-id(5)` during boot if it does not already exist,
+and persists it in `/etc/machine-id`.  As such, it suffices to make that file persistent.
+
+Alternatively, it is possible to generate a random `machine-id(5)`; while the
+specification allows for *any* hex-encoded 128b value, systemd itself uses
+[UUIDv4], *i.e.* random UUIDs, and it is thus preferable to do so as well, in
+case some software assumes `machine-id(5)` to be a UUIDv4. Those can be
+generated with `uuidgen -r | tr -d -` (`tr` being used to remove the dashes).
+
+Such a `machine-id(5)` can be set by writing it to `/etc/machine-id` or through
+the kernel's command-line, though NixOS' systemd maintainers [discourage] the
+latter approach.
+
+[UUIDv4]: https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random)
+[discourage]: https://github.com/NixOS/nixpkgs/pull/268995
+
+
+## `/var/lib/systemd` {#sec-var-systemd}
+
+Moreover, `systemd` expects its state directory — `/var/lib/systemd` — to persist, for:
+- {manpage}`systemd-random-seed(8)`, which loads a 256b “seed” into the kernel's RNG
+  at boot time, and saves a fresh one during shutdown;
+- {manpage}`systemd.timer(5)` with `Persistent=yes`, which are then run after boot if
+  the timer would have triggered during the time the system was shut down;
+- {manpage}`systemd-coredump(8)` to store core dumps there by default;
+  (see {manpage}`coredump.conf(5)`)
+- {manpage}`systemd-timesyncd(8)`;
+- {manpage}`systemd-backlight(8)` and {manpage}`systemd-rfkill(8)` persist hardware-related
+  state;
+- possibly other things, this list is not meant to be exhaustive.
+
+In any case, making `/var/lib/systemd` persistent is recommended.
+
+
+## `/var/log/journal/{machine-id}` {#sec-var-journal}
+
+Lastly, {manpage}`systemd-journald(8)` writes the system's journal in binary
+form to `/var/log/journal/{machine-id}`; if (locally) persisting the entire log
+is desired, it is recommended to make all of `/var/log/journal` persistent.
+
+If not, one can set `Storage=volatile` in {manpage}`journald.conf(5)`
+([`services.journald.storage = "volatile";`](#opt-services.journald.storage)),
+which disables journal persistence and causes it to be written to
+`/run/log/journal`.

nixos/doc/manual/administration/zfs-state.section.md

@@ -0,0 +1,16 @@
+# ZFS {#sec-zfs-state}
+
+When using ZFS, `/etc/zfs/zpool.cache` should be persistent (or a symlink to a persistent
+location) as it is the default value for the `cachefile` [property](man:zpoolprops(7)).
+
+This cachefile is used on system startup to discover ZFS pools, so ZFS pools
+holding the `rootfs` and/or early-boot datasets such as `/nix` can be set to
+`cachefile=none`.
+
+In principle, if there are no other pools attached to the system, `zpool.cache`
+does not need to be persisted; it is however *strongly recommended* to persist
+it, in case additional pools are added later on, temporarily or permanently:
+
+While mishandling the cachefile does not lead to data loss by itself, it may
+cause zpools not to be imported during boot, and services may then write to a
+location where a dataset was expected to be mounted.