Commit Graph

283 Commits

Author SHA1 Message Date
Silvan Mosberger
4f0dadbf38 treewide: format all inactive Nix files
After final improvements to the official formatter implementation,
this commit now performs the first treewide reformat of Nix files using it.
This is part of the implementation of RFC 166.

Only "inactive" files are reformatted, meaning only files that
aren't being touched by any PR with activity in the past 2 months.
This is to avoid conflicts for PRs that might soon be merged.
Later we can do a full treewide reformat to get the rest,
which should not cause as many conflicts.

A CI check has already been running for some time to ensure that new and
already-formatted files are formatted, so the files being reformatted here
should also stay formatted.

This commit was automatically created and can be verified using

    nix-build a08b3a4d19.tar.gz \
      --argstr baseRev b32a094368
    result/bin/apply-formatting $NIXPKGS_PATH
2024-12-10 20:26:33 +01:00
Emily
8221c09ff5
nixos/lib/test-driver: fix linting after compatibility clean‐up
The previous commit removed the handling of `dict` arguments, but
didn’t adjust the type, leading to the following type-checking error:

    test_driver/driver.py:216: error: Argument 1 to "NixStartScript" has incompatible type "str | dict[Any, Any]"; expected "str"  [arg-type]

It also left an unused import that Ruff is unhappy about:

    build/lib/test_driver/driver.py:11:22: F401 [*] `colorama.Fore` imported but unused
    …
    build/lib/test_driver/driver.py:11:28: F401 [*] `colorama.Style` imported but unused

Fixes: 71306e6b36
(cherry picked from commit d490680530)
(cherry picked from commit ff31b814b6)
2024-11-30 15:11:39 +01:00
Wolfgang Walther
a92ea1ff26
nixos/lib/test-driver: remove legacy args handling
Scheduled for removal in 24.11, so let's follow through.

Added in #291544.

(cherry picked from commit 71306e6b36)
(cherry picked from commit 8427b6f640)
2024-11-30 15:11:38 +01:00
Nick Cao
172a35f8ce
nixos/test-driver: target python 3.12 2024-11-22 10:49:32 -05:00
Nick Cao
e23f1733c6
nixos/test-driver: use ruff format in place of black 2024-11-22 10:49:31 -05:00
Nick Cao
ef2d3c542a
nixos/test-driver: modernize 2024-11-22 10:49:31 -05:00
Nick Cao
42d4046e94
nixos/test-driver: format with nixfmt 2024-11-22 10:49:30 -05:00
Nick Cao
b25360a7e5
nixos/test-driver: apply ruff check suggestions 2024-11-22 10:49:30 -05:00
Jörg Thalheim
ef9502a009 nixos/test-driver: fix resource cleanup of vlan/qmp objects
Using __del__ is somewhat unsound resource cleanup in our clase the
logger already closed its logfile and therefor fails with exception
before the rest of the resources can be cleaned up.
2024-10-16 19:46:38 +03:00
Philip Taron
2b67819d55 nixos-test-driver: avoid top-level with in shell.nix 2024-07-29 19:46:18 +02:00
Silvan Mosberger
b3ad661e9f nixos/lib/test-driver: Prevent unnecessary rebuilds
E.g. when only Nix files change
2024-06-14 20:42:16 +02:00
Martin Weinelt
ab897a8c62
nixos/test-driver: fix return value of subtest function
Mypy since version 1.10.0 complains about this:

> test_driver/driver.py:109: error: No return value expected  [return-value]
2024-06-06 01:07:39 +02:00
Gaetan Lepage
622ccf42c2 nixosTests.nixos-test-driver: fix ruff check 2024-05-23 14:30:23 +02:00
Dawid Dziurla
f8a88fd22c
nixos/test-driver: adjust to ruff changes 2024-05-22 11:29:02 +02:00
Stefan Hertrampf
d07866cddc nixos/test-driver: rm global logger
We remove the global rootlog in favor of instantiating the logger as
required in the __init__.py and pass it down as a parameter (of our
AbstractLogger type).
2024-05-07 15:17:17 +02:00
Stefan Hertrampf
303618c7e1 nixos/test-driver: enable XML log selectively
Previously, the XML logging was always present and only created an
output file if a special environment variable was present. We now only
create the XML logger if the environment variable is present, saving us
from logging to XML internally if it is not required.
2024-05-07 15:17:17 +02:00
Stefan Hertrampf
9e8d6bbe24 nixos/test-driver: add junit-xml logger
We add a new logger that allows generating a junit-xml compatible report
listing the subtests used in the nixos integration test. Junit-xml is a
widely used standard for test reports. The report can be used for quick
evaluation of which subtest failed.
2024-05-07 15:17:16 +02:00
Stefan Hertrampf
9d90df51a9 nixos/test-driver: Separate XML and Terminal log
We use the newly AbstractLogger class and separate the XML and Terminal
logging that is currently mixed into one class. We restore the old
behavior by introducing a CompositeLogger that takes care of logging
both to terminal and XML.
2024-05-07 15:17:16 +02:00
Stefan Hertrampf
b505db6f6d nixos/test-driver: add AbstractLogger interface 2024-05-07 15:17:15 +02:00
Stefan Hertrampf
a6160e5763 nixos/test-driver: use function instead of var
Use a proper function to enable serial log printing instead of setting
class member variables directly.
2024-04-22 13:35:25 +02:00
Stefan Hertrampf
c2c525f5bc nixos/test-driver: add junit-xml python package 2024-04-22 13:35:24 +02:00
Stefan Hertrampf
0d015895de nixos/test-driver: separate the subtest log call
We do not use the generic "nested" function but introduce a separate
subtest log call. This will later allow us to track subtests and account
logs to specific subtests.
2024-04-22 13:35:24 +02:00
Alyssa Ross
4491b9ec0d
nixos/test-driver: don't add QEMU_OPTS twice
As the TODO says, this is already included by the script.

If adding a device, including this again here would result in either
two devices being added, or, if they were explicitly named, an error
due to reuse of the name.
2024-04-05 12:30:36 +02:00
Will Fancher
7935db1456 nixos/tests: Remove unnecesary wait from initrdBackdoor 2024-03-26 23:40:05 -04:00
K900
7764fc5335 nixos/lib/test-driver: fix mypy errors after staging-next merge 2024-03-08 15:33:48 +03:00
K900
e758ca61fe nixos/lib/test-driver: make the warning message more noticeable 2024-03-01 13:20:09 +03:00
K900
bdacdc46e4 nixos/lib/test-driver: provide legacy path for create_machine({"startCommand": "..."}) 2024-02-28 09:32:11 +03:00
K900
423098c284 nixos/lib/test-driver: drop LegacyStartCommand, clean up create_machine API
We can finally do this now that it's no longer used.
2024-02-27 23:15:59 +03:00
K900
d9a37adc04 lib/test-driver: don't swallow qemu stderr 2024-02-27 23:15:58 +03:00
Michael Schneider
fe4edb712c nixos/testDriver: remove pinned mypy python version 2024-02-23 02:36:58 +00:00
K900
07e7fcc46f nixos/test-driver: black 2024-01-26 16:27:53 +03:00
Jacek Galowicz
d6318f4a86
Merge pull request #262772 from RaitoBezarius/qemu-vm/wait-for-event
nixos/lib/test-driver: add `wait_for_qmp_event`
2024-01-26 08:27:06 +01:00
Raito Bezarius
1196ae6e6b nixos/lib/test-driver: add wait_for_qmp_event
Adds a function to wait for a new QMP event with a model filter
so that you can expect specific type of events with specific payloads.

e.g. a guest-reset-induced shutdown event.
2024-01-25 20:50:20 +01:00
K900
9490738be7 nixos/lib/test-driver: add setuptools build dep 2024-01-12 13:58:07 +03:00
Someone Serge
392fbc90a8
nixos/lib/test-driver: make wait_for_unit ask for ActiveState only
This is a hotfix for https://github.com/NixOS/nixpkgs/issues/266690,
where `systemctl show $unit` sometimes randomly leaves ActiveState out
2023-11-16 18:25:42 +00:00
Will Fancher
9a0f523372 systemd-stage-1: Enable backdoor in nixos tests 2023-11-09 17:53:48 -05:00
Raito Bezarius
05dd78cf4b nixos/lib/test-driver: add driver-timeout as a passthru test
From now on, we will aim to ensure that the test driver
gets tested by OfBorg using all our available tests.

This commit adds the driver timeout test to the driver.
2023-10-29 12:45:00 +01:00
Raito Bezarius
d4d7550108 nixos/test-driver: provide a global timeout
Since the debut of the test-driver, we didn't obtain
a race timer with the test execution to ensure that tests doesn't run beyond
a certain amount of time.

This is particularly important when you are running into hanging tests
which cannot be detected by current facilities (requires more pvpanic wiring up, QMP
API stuff, etc.).

Two easy examples:

- Some QEMU tests may get stuck in some situation and run for more than 24 hours → we default to 1 hour max.
- Some QEMU tests may panic in the wrong place, e.g. UEFI firmware or worse → end users can set a "reasonable" amount of time

And then, we should let the retry logic retest them until they succeed and adjust
their global timeouts.

Of course, this does not help with the fact that the timeout may need to be
a function of the actual busyness of the machine running the tests.
This is only one step towards increased reliability.
2023-10-29 12:45:00 +01:00
Jacek Galowicz
dda77fcc55
Merge pull request #257535 from RaitoBezarius/vmstate
nixos/lib/test-driver: use QMP API to watch for VM state
2023-10-22 14:22:34 +01:00
Raito Bezarius
f94876a65c nixos/lib/test-driver: wire up QMP client
Now that we have a QMP client, we can wire it up in the test driver.

For now, it is almost completely useless because of the need of a constant "event loop", especially
for event listening.

In the next commits, we will slowly enable more and more usecases.
2023-10-21 13:03:15 +02:00
Carl Dong
e5c2c71280 nixos/nginx: Allow empty port for listen directive
When listening on unix sockets, it doesn't make sense to specify a port
for nginx's listen directive.

Since nginx defaults to port 80 when the port isn't specified (but the
address is), we can change the default for the option to null as well
without changing any behaviour.
2023-10-09 21:16:03 -04:00
Jörg Thalheim
f1450e660f nixos/test-driver: whitelist variable names that don't follow pep8
https://docs.astral.sh/ruff/rules/invalid-argument-name/
2023-09-30 10:31:22 +02:00
Jörg Thalheim
1810265b57 nixos/test-driver: name exception according to pep8
see https://docs.astral.sh/ruff/rules/error-suffix-on-exception-name/
2023-09-30 10:31:22 +02:00
Jörg Thalheim
a1f01abe53 nixos/test-driver: apply ruff fixes & suggestions 2023-09-30 10:31:22 +02:00
Jörg Thalheim
93b1fa09d5 nixos/test-driver: disable typecheck for dependencies where don't have typing 2023-09-30 10:18:07 +02:00
Jörg Thalheim
d746557260 nixos/test-driver: add shell.nix
this useful for local development
2023-09-30 10:18:07 +02:00
Jörg Thalheim
9ac9e8407f nixos/test-driver: fix type errors in extract-docstrings 2023-09-30 10:18:06 +02:00
Jörg Thalheim
a1666863fd nixos/test-driver: convert to pyproject from setup.py
This also makes configuration available if you just run those tools locally.
Also use ruff instead of pylint because it's faster and more
comprehensive.
2023-09-30 10:17:47 +02:00
Raito Bezarius
81becd3c44 nixos/lib/test-driver: reduce spam at boot hangs
Since 008f9f0cd4
 ("nixos/test-driver: actually use the backdoor message to wait for backdoor"),

when boot is still computering, we can get a tons of empty strings in response to the shell.

This is not really useful to print and waste the disk space for any CI system that logs them.

We stop logging chunks whenever they are empty.
2023-09-29 03:45:31 +02:00
Maciej Krüger
03762aa42a
test-driver: revert stderr nullpipe
Removes 2>/dev/null which re-adds stderr output breaking execute
2023-09-22 15:14:13 +02:00