In Linux 4.19 there has been a major rework of the overlayfs
implementation and it now opens files in lowerdir with O_NOATIME, which
in turn caused issues in our VM tests because the process owner of QEMU
doesn't match the file owner of the lowerdir.
The crux here is that 9p propagates the O_NOATIME flag to the host and
the guest kernel has no way of verifying whether that flag will lead to
any problems beforehand.
There is ongoing work to possibly fix this in the kernel, but it will
take a while until there is a working patch and consensus.
So in order to bring our default kernel back to 4.19 and of course make
it possible to run newer kernels in VM tests, I'm merging a small QEMU
patch as an interim solution, which we can drop once we have a working
fix in the next round of stable kernels.
Now we already had Linux 4.19 set as the default kernel, but that was
subsequently reverted in 048c36ccaa
because the patch we have used was the revert of the commit I bisected a
while ago.
This patch broke overlayfs in other ways, so I'm also merging in a VM
test by @bachp, which only tests whether overlayfs is working, just to
be on the safe side that something like this won't happen in the future.
Even though this change could be considered a moderate mass-rebuild at
least for GNU/Linux, I'm merging this to master, mainly to give us some
time to get it into the current 19.03 release branch (and subsequent
testing window) once we got no new breaking builds from Hydra.
Cc: @samueldr, @lheckemann
Fixes: https://github.com/NixOS/nixpkgs/issues/54509
Fixes: https://github.com/NixOS/nixpkgs/issues/48828
Merges: https://github.com/NixOS/nixpkgs/pull/57641
Merges: https://github.com/NixOS/nixpkgs/pull/54508
This reverts commit 048c36ccaa.
With the patch applied for fixing the overlayfs bug in QEMU, there
really shouldn't stand anything in our way to use 4.19 as the default
kernel.
Signed-off-by: aszlig <aszlig@nix.build>
Our VM tests and everything related to our virtualisation infrastructure
is currently broken if used with kernel 4.19 or later.
The reason for this is that since 4.19, overlayfs uses the O_NOATIME
flag when opening files in lowerdir and this doesn't play nice with the
way we pass the Nix store to our QEMU guests.
On a NixOS system, paths in the Nix store are typically owned by root
but the QEMU process is usually run by an ordinary user. Using O_NOATIME
on a file where you're not the owner (or superuser) will return with
EPERM (Operation not permitted).
This is exactly what happens in our VM tests, because we're using
overlayfs in the guests to allow writes to the store.
Another implication of this is that the default kernel version for NixOS
19.03 has been reverted to Linux 4.14.
Work on getting this upstream is still ongoing and the patch I posted
previously was incomplete, needs rework and also some more review from
upstream maintainers - in summary: This will take a while.
So instead of rushing in a kernel patch to nixpkgs, which will affect
all users of overlayfs, not just NixOS VM tests, I opted to patch QEMU
for now to ignore the O_NOATIME flag in 9p.
I think this is also the least impacting change, because even if you
care about whether access times are written or not, you get the same
behaviour as with Linux 4.19 in conjunction with QEMU.
Signed-off-by: aszlig <aszlig@nix.build>
Fixes: https://github.com/NixOS/nixpkgs/issues/54509
