Commit Graph

17 Commits

Author SHA1 Message Date
Robert Hensing
ae5cb919f5 nixos/testing/nodes: Do allow aliases
Aliases exist for a reason. Sure it is nice to make sure that
some aliases aren't used within Nixpkgs, but this creates two problems
which are far worse than your failing to meet your neatness compulsions.

- Users encounter missing attributes, https://github.com/NixOS/nixpkgs/issues/264577
  wasting their time, stalling their progress, and even occupying others
  time that would be better spent on fixing *real* issues.

- Hydra doesn't treat evaluation errors seriously enough, with the
  effect that actual relevant test failures are masked by evaluation
  failures such as those caused by this no aliases business.

- We don't even have the infrastructure to get rid of aliases, because
  all warnings in package attributes are disallowed by Nixpkgs CI
  tooling, last I checked.

Before re-disabling this, make sure that

- An actually helpful deprecation process is in place.

- Aliases are still allowed when `nixos-lib.runTests` and
  `pkgs.testers.runNixOSTest` are invoked by external projects.
  For instance, `all-tests.nix` could provide such an
  override (e.g. with `newScope`).
2023-11-06 12:30:28 +01:00
K900
80cc25eb51 nixos/testing: fix eval for tests created without make-python-test.nix
We have too many test entrypoints and this is a problem.
2023-10-06 23:52:04 +03:00
K900
8a7282f90b nixosTest: fix tests defined with pkgs.nixosTest 2023-10-06 16:33:15 +03:00
Arthur Gautier
18c734d7f2 nixosTest: adds support for lib.extend
When lib overrides were used, before this commit, they would not be made
available in the configuration evaluation of nixosTest's nodes.

Sample code:
``` nix
let
  pkgs = import ./. {
    overlays = [
      (new: old: {
        lib = old.lib.extend (self: super: {
          sorry_dave = builtins.trace "There are no pod bay doors" "sorry dave";
        });
      })
    ];
  };
in
pkgs.testers.nixosTest {
  name = "demo lib overlay";

  nodes = {
    machine = { lib, ... }: {
      environment.etc."got-lib-overlay".text = lib.sorry_dave;
    };
  };

  testScript = { nodes }:
    ''
      start_all()
      machine.succeed('grep dave /etc/got-lib-overlay')
    '';
}
```
2023-06-29 09:13:44 -07:00
Robert Hensing
b0e17891f2 nixos/testing/nodes.nix: Do not rely on disabledModules
It's just not necessary.
2023-05-11 16:24:01 +02:00
Robert Hensing
0f83261f0e nixos/testing: Add node.pkgsReadOnly escape hatch
By adding this option indirection, a test can declare all by itself
that it needs a custom nixpkgs. This is a more convenient way of
going about this when the caller of the test framework receives a
`node.pkgs` unconditionally.
2023-05-11 16:24:01 +02:00
Robert Hensing
f659db7ba2 nixos/testing: Add node.pkgs option
By factoring out this logic, it's easier for other projects to make
use of it this optimization too (and do it correctly).
2023-05-11 16:24:00 +02:00
Robert Hensing
b213791e7e nixos/all-tests.nix: Add readOnlyPkgs module 2023-05-11 16:24:00 +02:00
Robert Hensing
cd358fe24e nixos/all-tests.nix: Set nixpkgs.system 2023-05-10 15:55:09 +02:00
Naïm Favier
f677cbabe9
nixos/tests: remove minimal-kernel module
It's not used, doesn't build, and seems like the only reason to have
`manualConfig` take `stdenv` as an argument.
2022-12-22 01:42:15 +01:00
Robert Hensing
1d9b913088 nixos/lib/testing: Delay nodes.machine.~config~ migration
Provide a window during which both solutions are valid without
warnings, in order to fight warning fatigue, and not to push 3rd
party repo maintainers to add unnecessary compat code.
2022-10-16 16:30:21 +02:00
Robert Hensing
216c5dc10d nixos/doc: Disambiguate test option ids
Changing this later on would break external links into the manual.
2022-09-29 12:41:59 +02:00
Robert Hensing
1c0b9c4a48 nixos/testing/network.nix: Add network config to specialisations 2022-09-24 17:42:52 +01:00
Robert Hensing
666e969da0 nixos/testing: Add nodes.config backcompat to nodes module argument 2022-09-24 17:38:10 +01:00
Robert Hensing
6205d37747 nixos/testing: Improve option docs 2022-09-24 17:38:10 +01:00
Robert Hensing
24d1d74e4e nixos/testing: Extract nixos-test-base.nix NixOS module 2022-09-24 17:38:09 +01:00
Robert Hensing
b3de22483c nixos/testing-python.nix: Add evalTest
This is a decomposition of the testing-python.nix and build-vms.nix
files into modules.

By refactoring the glue, we accomplish the following:

 - NixOS tests can now use `imports` and other module system features.
    - Network-wide test setup can now be reusable; example:
       - A setup with all VMs configured to use a DNS server
       - Split long, slow tests into multiple tests that import a
         common module that has most of the setup.
    - Type checking for the test arguments
    - (TBD) "generated" options reference docs
 - Aspects that had to be wired through all the glue are now in their
   own files.
    - Chief example: interactive.nix.
    - Also: network.nix

In rewriting this, I've generally stuck as close as possible to the
existing code; copying pieces of logic and rewiring them, without
changing the logic itself.

I've made two exceptions to this rule

 - Introduction of `extraDriverArgs` instead of hardcoded
   interactivity logic.

 - Incorporation of https://github.com/NixOS/nixpkgs/pull/144110
   in testScript.nix.

I might revert the latter and split it into a new commit.
2022-09-21 10:55:11 +01:00