nixpkgs/nixos/modules/virtualisation
nikstur 289dd22132 nixos/qemu-vm: simplify building nix store image
Summary of this change:

- Simplify code.
- Stop a disk image from being cached in the binary cache.
- Make erofs Nix Store image build in an acceptable time outside of
  testing environments (like `darwin.builder`).
- Do not regress on performance for tests that use many store paths in
  their Nix store image.
- Slightly longer startup time for tests where not many store paths are
  included in the image (these probably shouldn't use `useNixStoreImage`
  anyways).
- Slightly longer startup time when inputs of VM do not change because
  the Nix store image is not cached anymore.

Remove the `storeImage` built with make-disk-image.nix. This produced a
separate derivation which is then cached in the binary cache. These
types of images should be avoided because they gunk up the cache as they
change frequently. Now all Nix store images, whether read-only or
writable are based on the erofs image previously only used for read-only
images.

Additionally, simplify the way the erofs image is built by copying the
paths to include to a separate directory and build the erofs image from
there.

Before this change, the list of Nix store paths to include in the Nix
store image was converted to a complex regex that *excludes* all other
paths from a potentially large Nix store.

This previous approach suffers from two issues:

1. The regex is complex and, as admitted in the source code of the
   includes-to-excludes.py script, most likely contains at least one
   error. This means that it's unlikely that anyone will touch this
   piece of software again.

2. When the Nix store image is built from a large Nix store (like when
   you build the VM script to run outside of any testing context) this
   regex becomes painfully slow. There is at least one prominent
   use-case where this matters: `darwin.builder`.

Benchmarking impressions:

- Building Nix store via make-disk-image.nix takes ~25s
- Building Nix store as an erofs image takes ~4s
- Running nixosTests.qemu-vm-writable-store-image takes ~10s when
  building the erofs image with the regex vs ~14s when building by
  copying to a temporary directory.
- nixosTests.gitlab which had the biggest gains from the initial erofs
  change takes the same time as before.
- On a host with ~140k paths in /nix/store, building the erofs image
  with the regex takes 410s as opposed to 6s when copying to a temporary
  directory.
2024-07-18 20:24:58 +02:00
..
podman nixos/podman: Create docker socket symlink via socket unit (#305803) 2024-05-02 14:04:37 +00:00
amazon-ec2-amis.nix nixos/amazon-ec2-amis: Add NixOS 23.11 2024-03-02 22:40:50 +01:00
amazon-image.nix nixos/amazon-image: avoid top-level with statements 2024-07-15 22:02:39 -07:00
amazon-init.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
amazon-options.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
anbox.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
appvm.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
azure-agent.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
azure-bootstrap-blobs.nix
azure-common.nix nixos/*: remove boot.grub.version 2023-05-10 21:51:26 +02:00
azure-config-user.nix
azure-config.nix
azure-image.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
azure-images.nix
brightbox-config.nix
brightbox-image.nix
build-vm.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
cloudstack-config.nix
container-config.nix
containerd.nix nixos/containerd: remove LimitNOFILE from service (#313507) 2024-06-02 17:51:10 +02:00
containers.nix nixos/containers: set storage.settings via mkDefault 2024-05-06 16:34:00 +02:00
cri-o.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
digital-ocean-config.nix digital-ocean: revert "make it easier to use disko" 2024-04-21 11:36:17 +02:00
digital-ocean-image.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
digital-ocean-init.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
docker-image.nix
docker-rootless.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
docker.nix treewide: big opengl cleanup 2024-06-16 14:11:33 +03:00
ec2-amis.nix
ec2-data.nix ec2-data: prefer 'install/umask' over 'chmod' 2024-07-03 23:18:43 +02:00
ec2-metadata-fetcher.sh
ecs-agent.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
gce-images.nix
google-compute-config.nix nixos/google-compute-config: Add sudo-rs rules 2023-11-05 20:33:19 +00:00
google-compute-image.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
grow-partition.nix
hyperv-guest.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
hyperv-image.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
incus.nix nixos/incus: INCUS_OVMF_PATH -> INCUS_EDK2_PATH 2024-07-15 12:10:50 -04:00
kubevirt.nix
kvmgt.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
libvirtd.nix nixos/virtualisation: increase priority for libvirt NSS modules 2024-06-25 20:22:14 +02:00
linode-config.nix treewide: remove deprecated grub.version references 2024-02-25 08:29:58 -08:00
linode-image.nix maintainers: Update houstdav000 -> cyntheticfox 2023-03-30 18:50:29 -04:00
lxc-container.nix nixos/lxc-container: link to prepare-root when boot.initrd.systemd.enable is on 2024-02-20 23:12:41 +01:00
lxc-image-metadata.nix treewide: Remove usage of pkgs.{system,hostPlatform} aliases 2024-05-16 11:48:32 +02:00
lxc-instance-common.nix nixos/lxd: add virtual-machine support, image and module 2023-09-03 20:06:44 -04:00
lxc.nix nixos/lxc: add package option and use for incus/lxd 2024-04-22 08:31:05 -04:00
lxcfs.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
lxd-agent.nix treewide: fix all obviously wrong mkEnableOptions 2024-06-03 13:20:32 +02:00
lxd-virtual-machine.nix nixos/lxd-virtual-machine: enable CPU hotplug for x86 VMs 2024-04-18 10:36:55 -04:00
lxd.nix nixos/lxc: add package option and use for incus/lxd 2024-04-22 08:31:05 -04:00
multipass.nix treewide: fix mkEnableOption usage 2024-06-14 02:41:42 -04:00
nixos-containers.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
oci-common.nix nixos/virtualisation: use mkDefault in networking.useNetworkd in oci-common 2023-09-28 22:05:34 +01:00
oci-config-user.nix oci-image: init scripts to build and upload image 2023-09-21 22:15:22 +01:00
oci-containers.nix nixos/oci-containers: add preRunExtraOptions (#315356) 2024-06-05 18:18:37 -04:00
oci-image.nix nixos/oci-image: expose diskSize as a configurable option 2024-06-21 08:54:08 +02:00
oci-options.nix nixos/oci-image: expose diskSize as a configurable option 2024-06-21 08:54:08 +02:00
openstack-config.nix
openstack-metadata-fetcher.nix
openstack-options.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
openvswitch.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
parallels-guest.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
proxmox-image.nix nixos/proxmox-image: change scsi controller model to upstream default 2024-04-29 11:01:45 +05:30
proxmox-lxc.nix nixos/proxmox-lxc: allow importing module without activation, for used in mixed machine clusters 2024-06-27 13:05:05 +01:00
qemu-guest-agent.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
qemu-vm.nix nixos/qemu-vm: simplify building nix store image 2024-07-18 20:24:58 +02:00
rosetta.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
spice-usb-redirection.nix maintainers: drop lheckemann 2024-06-21 15:54:54 +02:00
vagrant-guest.nix nixos/vagrant-guest: prefer 'install' over 'chmod' 2024-04-09 21:41:00 +02:00
vagrant-virtualbox-image.nix nixos/alsa: kill sound.enable and friends with fire 2024-07-13 13:56:18 +03:00
virtualbox-guest.nix virtualbox guest module: draganddrop became dragAndDrop (#317756) 2024-06-06 16:48:47 +02:00
virtualbox-host.nix nixos/virtualbox-host: remove obsolete warnings 2024-06-19 09:42:54 +02:00
virtualbox-image.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
vmware-guest.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
vmware-host.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
vmware-image.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
waydroid.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
xe-guest-utilities.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
xen-dom0.nix nixos: remove all uses of lib.mdDoc 2024-04-13 10:07:35 -07:00
xen-domU.nix nixos/*: remove boot.grub.version 2023-05-10 21:51:26 +02:00