Commit Graph

452 Commits

Author SHA1 Message Date
Wolfgang Walther
d2593f01e1
runInLinuxVM: pass .attrs.sh explicitly instead of whole /build directory
The approach taken in [1] breaks down as soon as vmRunCommand is
manually called with an overriden TMPDIR, like disko does it. /build
will just not be available.

By moving the .attrs.sh file into the xchg folder explicitly, we can all
the "exchange infrastructure" the same as before, thus avoid more
breakage.

This reverts some parts of [1].

[1]: 97ed6b4565
2024-11-30 13:46:45 +01:00
Wolfgang Walther
58570e75d9
runInLinuxVM: refactor vmRunCommand
This makes it simpler to copy more files to xchg for the structuredAttrs
case in the next commit.
2024-11-30 13:40:27 +01:00
Wolfgang Walther
437e6dbbb0
runInLinuxVM: load stdenv/setup with fixed environment in stage2Init
In [1] we started sourcing stdenv/setup in stage2Init to allow for
structuredAttrs. We failed to take the changed NIX_BUILD_TOP etc.
variables into account. We need to load stdenv/setup after changing
them, because the structuredAttrs startup code makes use of it.

[1]: 97ed6b4565
2024-11-30 13:40:26 +01:00
Wolfgang Walther
9f6b99e1ef
runInLinuxVM: minimize saved-env
The export to saved-env was very intentionally done at the very
beginning of vmRunCommand, even jumping through extra hoops just to
avoid the PATH variable from polluting the saved variable.

In 26eba25577 we loaded stdenv in the
wrong place, we should do it after saving the previous environment. This
is also more consistent with the order of how we load those values back
in stage2Init.
2024-11-30 13:40:26 +01:00
Wolfgang Walther
3952f870fc
runInLinuxVM: clean up
Those were left-over after 97ed6b4565.

This also cleans up some confusion around TMPDIR. We had the following
lines:

  mkdir xchg
  ...
  cd $TMPDIR
  ...
  path=$TMPDIR/xchg

Those only worked because the **current directory** is the same as
$TMPDIR. Both are /build by default. To refer to the same directory in
two different ways is very confusing at best.
2024-11-30 13:39:58 +01:00
Wolfgang Walther
de7867c226
runInLinuxVM: add simple structuredAttrs test 2024-11-30 12:26:26 +01:00
Wolfgang Walther
d232880c20
various: remove left-over rtc_cmos rootModule
This is not needed anymore after
65a50ea453.
2024-11-26 22:02:46 +01:00
Maximilian Bosch
6f3d8a72ea
runInLinuxVM: remove hwclock -s invocation
Commit 97ed6b4565 broke the systemd-boot
tests (among others) because of the `hwclock -s` invocation. This was
broken for a while, but not noticed because we didn't have a `set -e`
before.

The error

    hwclock: select() to /dev/rtc0 to wait for clock tick timed out

MAY be related to an open QEMU bug[1]: I can't reproduce the error on
aarch64-linux and x86_64-linux with `partitionTableType = "legacy";`.
Also, the issue disappears on x86_64-linux when adding `--directisa`.

However, the invocation was added in f73ff0529e
10 years ago which didn't give any reasoning or pointer to what KVM bug
this may be. Given that this must have happened on an ancient version,
we agreed on removing it altogether[2].

[1] https://gitlab.com/qemu-project/qemu/-/issues/1762
[2] https://github.com/NixOS/nixpkgs/pull/354535#issuecomment-2500791222
2024-11-26 15:51:11 +01:00
Maximilian Bosch
26eba25577
runInLinuxVM: re-add sourcing of stdenv & .attrs.sh
This is required for `preVM` which can be arbitrary code: e.g.
`vmTools.createEmptyImage` assumes the presence of `$out` in its default
configuration.

Also, before this patch, `preVM` wasn't even set in the
`__structuredAttrs` case.
2024-11-24 14:24:37 +01:00
Maximilian Bosch
97ed6b4565
runInLinuxVM: fix for structured attrs
Closes #334705
Addresses #205690

The main issue was that the output variable (i.e. `$out` and friends)
didn't exist. I figured the easiest way to add those is to source
`stdenv` here. Given that we build another derivation in this builder,
it's pretty likely that `stdenv` gets pulled already, so I don't expect
a real overhead here.

Also, this mounts `/build` into the VM: this is required to make sure
`.attrs.json` & `.attrs.sh` are available. Dropped the mount of `xchg`
into `/tmp` now since it's also part of `/build`.
2024-11-22 20:02:22 +01:00
matthewcroughan
65c851cd75 vmTools: allow qemu to be overridden via customQemu argument 2024-07-08 19:47:08 +01:00
Maximilian Bosch
378177d269
linux kernel: prefer zstd where possible
Closes #302291
Closes #301536

The following things have changed:

* For 5.7+: ZSWAP compressor uses zstd now.
* For 5.11+: ZRAM compressor uses zstd now.
* For 5.13+: kernel modules are compressed with zstd instead of xz.
* For 5.19+: support zstd-compressed firmware.

The modules-closure functionality needed explicit support for copying
over `.zst` files. Also, the VM image builder used busybox's `insmod`
before which doesn't support zstd. Switched to `kmod` and added xz/zstd
as dependencies for it, similar to how it's done for the actual stage1
in d33e52b253. The use of `kmod` here
doesn't seem to be such a big deal since it's only a build-time
dependency.
2024-04-19 16:45:35 +02:00
Linus Heckemann
9d50a5b0d8
Merge pull request #264147 from lheckemann/foreign-distros
vmTools: misc debian tool improvements
2024-04-10 13:31:39 +02:00
Philip Taron
102a33d30e Avoid top-level with ...; in pkgs/build-support/vm/test.nix 2024-03-19 22:31:19 +01:00
Nikolay Korotkiy
7ec994050c
vmTools: update debian versions 2023-11-24 20:59:42 +04:00
Linus Heckemann
5a4f40797c vmTools.debClosureGenerator: remove apparently obsolete bugfix 2023-10-29 11:26:11 +01:00
Linus Heckemann
e6a4a3feea vmTools/deb tools: pass through args
This is useful for adding preVM/postVM steps.
2023-10-29 11:24:54 +01:00
Linus Heckemann
01d078d970 vmTools.runInLinuxImage: allow setting root filesystem 2023-10-28 12:48:26 +02:00
Brian McGee
9aa91ec7b5 build-support/vm: fix makeImageTestScript
`qemu-img create` now requires an additional `-F <second_image_format>` flag.

https://github.com/qemu/qemu/blob/master/docs/about/removed-features.rst#qemu-img-backing-file-without-format-removed-in-61
2023-08-24 09:09:43 +01:00
Felix Bühler
0a2745684e
Merge pull request #239624 from Stunkymonkey/use-optionalString-then
treewide: use optionalString instead of 'then ""'
2023-07-22 13:02:47 +02:00
Felix Buehler
f3719756b5 treewide: use optionalString instead of 'then ""' 2023-06-24 20:19:19 +02:00
Alyssa Ross
64bfa05b36
vmTools: download debs from snapshot URLs
The deb files we want are no longer available in the Debian mirrors,
so we need to download them from the snapshots we download the package
lists from.

This makes it possible to build the os-prober NixOS test again.
2023-06-11 19:13:31 +00:00
ajs124
3f8ad50f9a build-support/vm/deb/deb-closure: quote generated urls
otherwise, eval fails when the experimental no-url-literals feature is activated

unquoted urls are discouraged after https://github.com/NixOS/rfcs/pull/45
2023-02-20 01:38:31 +01:00
superherointj
0eae5c0532
Merge pull request #205156 from yu-re-ka/musl-runinlinuxvm
runInLinuxVM: fix on musl
2023-02-16 12:24:34 -03:00
Nikolay Korotkiy
8cf41cf4ef
Merge pull request #215713 from sikmir/vmtools
vmTools: update debian versions
2023-02-14 23:33:34 +04:00
Felix Buehler
cdb39a86e0 treewide: use optionalString 2023-02-13 21:52:34 +01:00
Nikolay Korotkiy
4ed059d041
vmTools: update debian versions 2023-02-10 21:39:15 +03:00
Artturin
fe1c7a1945 treewide: remove usages of header and stopNest
they're obsolete
2023-01-16 00:08:12 +02:00
Alexander Bich
aaa4244387 vmTools: add ubuntu 22.04 2022-12-26 10:58:52 +00:00
Sandro
9c624f5879
Merge pull request #203117 from sikmir/vmtools
vmTools: update current maintained debian versions
2022-12-20 15:51:50 +01:00
figsoda
ec8cb34358 treewide: fix typos 2022-12-17 19:39:44 -05:00
Yureka
03b2a82ca9 runInLinuxVM: fix on musl 2022-12-08 18:36:59 +01:00
Nikolay Korotkiy
dd1c4ff5fc
vmTools: update current maintained debian versions 2022-11-26 23:53:24 +03:00
Artturin
c1fffdfffb treewide: change some glibc to stdenv.cc.libc 2022-05-27 05:57:43 +03:00
Artturin
0c4d65b21e treewide: stdenv.glibc -> glibc 2022-05-25 15:51:20 +03:00
Jörg Thalheim
d6fee2a44a vmTools/test: fix pcmanfm instead of pan
This goes much quicker. Also disable parallel build for machines with
many cores to avoid running out of memory.
2021-12-31 16:47:22 +01:00
Alyssa Ross
37c5db5a29
vmTools/test: add missing backing format 2021-12-28 13:10:16 +00:00
Alyssa Ross
80b5b4917c
vmTools/test: give pan build more memory 2021-12-28 13:10:15 +00:00
Alyssa Ross
55b883448e
vmTools.buildRPM: properly derive source name
This fixes e.g. building the rpm of patchelf as in
pkgs/build-support/vm/test.nix.  The glob won't work if there are
multiple spec files but I don't think rpmbuild would work in that case
either.
2021-12-28 13:10:13 +00:00
Alyssa Ross
e0455e1e29
vmTools: fix phases
These gave missing attribute errors.  They should have been written as
strings in ae9337179b.

Fixes: ae9337179b ("vm: deprecate phases")
Fixes: 52b10ee872 ("vmTools refactor: don't use huge `with pkgs;`")
2021-12-28 13:10:11 +00:00
Alyssa Ross
812fe6a2b3
vmTools/test: update for new releases 2021-12-28 12:40:57 +00:00
Alyssa Ross
0acf53bdb1
runInLinuxVM: set 9p msize to 128K
This fixes the following warning from QEMU, by using the value that
will be the default in newer kernels[1].

> 9p: degraded performance: a reasonable high msize should be chosen
> on client/guest side (chosen msize is <= 8192). See
> https://wiki.qemu.org/Documentation/9psetup#msize for details.

For a runInLinuxVM derivation I have that does a large amount of
IO[2], this change results in a fairly small but repeatable speed
increase.

[1]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=9c4d94dc9a64426d2fa0255097a3a84f6ff2eebe
[2]: https://spectrum-os.org/git/spectrum/tree/installer/default.nix?id=06dd70500a9e475d23e3296dddf409d84abf9b9c#n52
2021-12-17 07:52:11 +00:00
Alyssa Ross
9c4c0f89a5
vmTools.runInLinuxVM: create fd symlinks in /dev
These are usually set up by systemd, but there's no systemd in the
VM.  /dev/fd is required for <(...) syntax in bash, and I'm sure lots
of things depend on the stdin/stdout/stderr links as well.
2021-12-08 00:32:29 +00:00
Artturi
c4851c0d71
Revert msize related commits (#147180) 2021-11-24 02:33:53 +02:00
Ryan Burns
2175b157ac treewide: refactor isi686 && isx86_64 -> isx86 2021-11-20 17:50:41 -08:00
Artturin
d1c3a49fdd vmTools: set msize to 16KiB temporarily
hopefully this will reduce the ram usage enough
2021-11-21 01:13:54 +02:00
Artturin
e2ab6321ed vmTools: set msize to 128KiB
500KiB was causing issues https://github.com/NixOS/nixpkgs/pull/142577
2021-11-20 20:49:49 +02:00
Janne Heß
ecf388e90b
vmTools: Make msize larger to silence warning
See https://issues.guix.gnu.org/47225
2021-11-18 21:21:49 +01:00
Christian Kögler
9bf244964d runInLinuxImage: Add missing backing format 2021-10-28 00:05:27 +02:00
Christian Kögler
0625121793 buildInDebian: fix test 2021-10-28 00:03:27 +02:00