This change removes the bespoke logic around identifying block devices.
Instead of trying to find the right device by iterating over
`qemu.drives` and guessing the right partition number (e.g.
/dev/vda{1,2}), devices are now identified by persistent names provided
by udev in /dev/disk/by-*.
Before this change, the root device was formatted on demand in the
initrd. However, this makes it impossible to use filesystem identifiers
to identify devices. Now, the formatting step is performed before the VM
is started. Because some tests, however, rely on this behaviour, a
utility function to replace this behaviour in added in
/nixos/tests/common/auto-format-root-device.nix.
Devices that contain neither a partition table nor a filesystem are
identified by their hardware serial number which is injecetd via QEMU
(and is thus persistent and predictable). PCI paths are not a reliably
way to identify devices because their availability and numbering depends
on the QEMU machine type.
This change makes the module more robust against changes in QEMU and the
kernel (non-persistent device naming) and by decoupling abstractions
(i.e. rootDevice, bootPartition, and bootLoaderDevice) enables further
improvement down the line.
This change fixes two problems with the qemu testing code:
1. Previously, the qemu-img command was missing a disk image format
argument.
2. Previously, if a test assertion failed, the test hung because the VM
was not torn down.
Lego has a built-in mechanism for sleeping for a random amount
of time before renewing a certificate. In our environment this
is not only unnecessary (as our systemd timer takes care of it)
but also unwanted since it slows down the execution of the
systemd service encompassing it, thus also slowing down the
start up of any services its depending on.
Also added FixedRandomDelay to the timer for more predictability.
Fixes#190493
Check if an actual key file exists. This does not
completely cover the work accountHash does to ensure
that a new account is registered when account
related options are changed.
conversions were done using https://github.com/pennae/nix-doc-munge
using (probably) rev f34e145 running
nix-doc-munge nixos/**/*.nix
nix-doc-munge --import nixos/**/*.nix
the tool ensures that only changes that could affect the generated
manual *but don't* are committed, other changes require manual review
and are discarded.
- Make tests/lxd.nix use NixOS's lxdMeta & lxdImage to avoid relying on
3rd party containers such as Alpine Linux for testing purposes.
- Merge tests/lxd-image.nix into tests/lxd.nix, since now both have a
similar structure.
- Extract duplicated inline LXD configuration into a separate file,
- Add passthru.lxd-nftables & passthru.lxd-image-server.
Allows configuring many default settings for certificates,
all of which can still be overridden on a per-cert basis.
Some options have been moved into .defaults from security.acme,
namely email, server, validMinDays and renewInterval. These
changes will not break existing configurations thanks to
mkChangedOptionModule.
With this, it is also now possible to configure DNS-01 with
web servers whose virtualHosts utilise enableACME. The only
requirement is you set `acmeRoot = null` for each vhost.
The test suite has been revamped to cover these additions
and also to generally make it easier to maintain. Test config
for apache and nginx has been fully standardised, and it
is now much easier to add a new web server if it follows
the same configuration patterns as those two. I have also
optimised the use of switch-to-configuration which should
speed up testing.
the default hasn't been changed since 2009
this can improve our test performances
nixos/tests: remove explicit memorySize <1024
1024MiB is now the default
- Use an acme user and group, allow group override only
- Use hashes to determine when certs actually need to regenerate
- Avoid running lego more than necessary
- Harden permissions
- Support "systemctl clean" for cert regeneration
- Support reuse of keys between some configuration changes
- Permissions fix services solves for previously root owned certs
- Add a note about multiple account creation and emails
- Migrate extraDomains to a list
- Deprecate user option
- Use minica for self-signed certs
- Rewrite all tests
I thought of a few more cases where things may go wrong,
and added tests to cover them. In particular, the web server
reload services were depending on the target - which stays alive,
meaning that the renewal timer wouldn't be triggering a reload
and old certs would stay on the web servers.
I encountered some problems ensuring that the reload took place
without accidently triggering it as part of the test. The sync
commands I added ended up being essential and I'm not sure why,
it seems like either node.succeed ends too early or there's an
oddity of the vm's filesystem I'm not aware of.
- Fix duplicate systemd rules on reload services
Since useACMEHost is not unique to every vhost, if one cert
was reused many times it would create duplicate entries in
${server}-config-reload.service for wants, before and
ConditionPathExists
Shimming out the Let's Encrypt domain name to reuse client configuration
doesn't work properly (Pebble uses different endpoint URL formats), is
recommended against by upstream,[1] and is unnecessary now that the ACME
module supports specifying an ACME server. This commit changes the tests
to use the domain name acme.test instead, and renames the letsencrypt
node to acme to reflect that it has nothing to do with the ACME server
that Let's Encrypt runs. The imports are renamed for clarity:
* nixos/tests/common/{letsencrypt => acme}/{common.nix => client}
* nixos/tests/common/{letsencrypt => acme}/{default.nix => server}
The test's other domain names are also adjusted to use *.test for
consistency (and to avoid misuse of non-reserved domain names such
as standalone.com).
[1] https://github.com/letsencrypt/pebble/issues/283#issuecomment-545123242
Co-authored-by: Yegor Timoshenko <yegortimoshenko@riseup.net>
This module allows root autoLogin, so we would break that for users, but
they shouldn't be using it anyways. This gives the impression like auto
is some special display manager, when it's just lightdm and special pam
rules to allow root autoLogin. It was created for NixOS's testing
so I believe this is where it belongs.
Updates required:
- Use vpc image format (new default, supported by Amazon)
- Pass full image filename to makeEc2Test
- Increase memory allocation for nixos-rebuild
- Set a networking.hostName for services.httpd
- Add appropriate escaping in literal userdata
While I'm here, try to make it fail fast.
The upstream session files display managers use have no concept of sessions being composed from
desktop manager and window manager. To be able to set upstream session files as default
session, we need a single option. Having two different ways to set default session would be confusing,
though, so we decided to deprecate the old method.
We also created separate script for each session, just like we already had a separate desktop
file for each one, and started using displayManager.sessionPackages mechanism to make the
session handling more uniform.
The recent custom endpoint addition allows us to directly point
certbot to the custom Pebble directory endpoint.
Thanks to that, we can ditch the Pebble patch we were using so far;
making this test maintenance easier.
Let's encrypt bumped ACME to V2. We need to update our nixos test to
be compatible with this new protocol version.
We decided to drop the Boulder ACME server in favor of the more
integration test friendly Pebble.
- overriding cacert not necessary
- this avoids rebuilding lots of packages needlessly
- nixos/tests/acme: use pebble's ca for client tests
- pebble always generates its own ca which has to be fetched
TODO: write proper commit msg :)
Since the switch to check the nginx config with gixy in
59fac1a6d7, the ACME test doesn't build
anymore, because gixy reports the following false-positive (reindented):
>> Problem: [alias_traversal] Path traversal via misconfigured alias.
Severity: MEDIUM
Description: Using alias in a prefixed location that doesn't ends with
directory separator could lead to path traversal
vulnerability.
Additional info: https://github.com/yandex/gixy/blob/master/docs/en/plugins/aliastraversal.md
Pseudo config:
server {
server_name letsencrypt.org;
location /documents/2017.11.15-LE-SA-v1.2.pdf {
alias /nix/store/y4h5ryvnvxkajkmqxyxsk7qpv7bl3vq7-2017.11.15-LE-SA-v1.2.pdf;
}
}
The reason this is a false-positive is because the destination is not a
directory, so something like "/foo.pdf../other.txt" won't work here,
because the resulting path would be ".../destfile.pdf../other.txt".
Nevertheless it's a good idea to use the exact match operator (=), to
not only shut up gixy but also gain a bit of performance in lookup (not
that it would matter in our test).
Signed-off-by: aszlig <aszlig@nix.build>
The Openstack metadata service exposes the EC2 API. We use the
existing `ec2.nix` module to configure the hostname and ssh keys of an
Openstack Instance.
A test checks the ssh server is well configured.
This is mainly to reduce the size of the image (700MB). Also,
declarative features provided by cloud-init are not really useful
since we would prefer to use our `configuration.nix` file instead.
In 0c7c1660f7 I have set allowSubstitutes
to false, which avoided the substitution of the certificates.
Unfortunately substitution may still happen later when the certificate
is merged with the CA bundle. So the merged CA bundle might be
substituted from a binary cache but the certificate itself is built
locally, which could result in a different certificate in the bundle.
So instead of adding just yet another workaround, I've now hardcoded all
the certificates and keys in a separate file. This also moves
letsencrypt.nix into its own directory so we don't mess up
nixos/tests/common too much.
This was long overdue and should finally make the dependency graph for
the ACME test more deterministic.
Signed-off-by: aszlig <aszlig@nix.build>
Since e95f17e272, Go packages no longer
contain the source tree, however Boulder seems to need that as it
generates a few files during build.
Ideally we would only pick the files that are needed and put it into a
separate output, but I currently don't have time for this so I'm marking
this with XXX to get back to it later.
Signed-off-by: aszlig <aszlig@nix.build>
