Render un`_type`d defaults and examples as `literalExpression`s using
`lib.generators.toPretty` so that consumers don't have to reinvent Nix
pretty-printing. `renderOptionValue` is kept internal for now intentionally.
Make `toPretty` print floats as valid Nix values (without a tilde).
Get rid of the now-obsolete `substSpecial` function.
Move towards disallowing evaluation of packages in the manual by
raising a warning on `pkgs.foo.{outPath,drvPath}`; later, this should
throw an error. Instead, module authors should use `literalExpression`
and `mkPackageOption`.
Unlike the XML doc renderer, the AsciiDoc and CommonMark renderers don't
pretty-print certain complex types, like literal expressions, DocBook
literals, and derivations. These types are dumped into the documentation
as JSON.
This commit parses and unwraps these types when loading the
JSON-formatted NixOS options. The AsciiDoc and CommonMark renders have
also been combined into a single script to allow code reuse.
Before this patch, if we:
1. add a new user in the config,
2. mount /home seperately,
3. not set neededForBoot for /home,
4. and run `nixos-rebuild boot`,
the newly added user's home will not be created after a reboot. This
is because when nixos/modules/config/update-users-groups.pl is running
in stage 2 to setup users, /home is not mounted.
This patch fixes this issue.
\<foo\> will often be displayed like \<foo>, for example by mkdocs.
I've tested a number of markdown renderers and they render html escape
sequences fine.
Until this commit, the documentation suggested `multi-user.target`
as `wantedBy` for all services.
Since `multi-user.target` is not available for user services,
propose a different default for those in the documentation.
Co-authored-by: Naïm Favier <n@monade.li>
Co-authored-by: Florian Warzecha <liketechnik@disroot.org>
Provide a window during which both solutions are valid without
warnings, in order to fight warning fatigue, and not to push 3rd
party repo maintainers to add unnecessary compat code.
Fixes the problem introduced by 12b3066aae
which caused nixos/release.nix to return the wrong attributes, while
intending to only affect nixos/lib's runTest.
This also removes callTest from the test options, because callTest is
only ever invoked by all-tests.nix.
It has been replaced by the modular test framework in nixos/lib/testing.
If you are looking for a way to produce a VM-test-like configuration
outside of the test framework, use the nixos/lib/testing/nixos-test-base.nix
NixOS module, possibly in combination with { _module.args.nodes = .....; }.
Before this, it relied on being able to `imports` from the `name`
module argument, which wasn't really necessary and required a
potentially quite breaking change. We can revert that now.
This is a decomposition of the testing-python.nix and build-vms.nix
files into modules.
By refactoring the glue, we accomplish the following:
- NixOS tests can now use `imports` and other module system features.
- Network-wide test setup can now be reusable; example:
- A setup with all VMs configured to use a DNS server
- Split long, slow tests into multiple tests that import a
common module that has most of the setup.
- Type checking for the test arguments
- (TBD) "generated" options reference docs
- Aspects that had to be wired through all the glue are now in their
own files.
- Chief example: interactive.nix.
- Also: network.nix
In rewriting this, I've generally stuck as close as possible to the
existing code; copying pieces of logic and rewiring them, without
changing the logic itself.
I've made two exceptions to this rule
- Introduction of `extraDriverArgs` instead of hardcoded
interactivity logic.
- Incorporation of https://github.com/NixOS/nixpkgs/pull/144110
in testScript.nix.
I might revert the latter and split it into a new commit.
deprecate literalDocBook by adding a warning (that will not fire yet) to
its uses and other docbook literal strings by adding optional warning
message to mergeJSON.
For example, the wait_for_unit() call in the Moodle test times out for
myself and others[1], so it would be good to be able to increase it to
something less likely to be hit by a test that would otherwise pass.
[1]: https://github.com/NixOS/nixpkgs/pull/177052#issue-1266336706
conversions were done using https://github.com/pennae/nix-doc-munge
using (probably) rev f34e145 running
nix-doc-munge nixos/**/*.nix
nix-doc-munge --import nixos/**/*.nix
the tool ensures that only changes that could affect the generated
manual *but don't* are committed, other changes require manual review
and are discarded.
leaving some newlines around after an admonition was closed causes the
newline rule to match, which in turn inserts literallayout newlines into
te xml output. that's not what we want.
Pass `-t` to pixz to prevent it from appending an index to the end of
the uncompressed stream, confusing tools such as `machinectl import-tar`.
Fixes: #187816
with ever more options being markdown rather than docbook the conversion
time is starting to become a significant factor of doc build time.
luckily we can pre-convert all nixos option docs to MD and cache the
result of this conversion, then merge the already-converted json file
with user option docs. we leave options.json unconverted to keep it as
close to the actual nix code as possible.
during docs conversion it can be very useful to know exactly *where* the
error the script complained about is. the name of the option should be
sufficient since option merging is rather rare, and won't merge doc
attributes anyway.
Within a dual VM test-setup a strange behaviour was observed.
The two VMs are connected via one vde_switch instance
(instancevirtualisation.vlans = [ 1 ]; IMO a bad attribute name for
switch instances, has nothing to do with VLANs in sense of 802.1Q).
A ping on the base interface (eth1) works, but not on VLAN
subinterfaces (vlan1@eth1). A tcpdump of eth1 includes the ARP requests
tagged with the subinterfaces VLAN ID, but responses seems not to pass
the vde_switch. This works fine if performed on the base interface.
Putting the vde_switch in hub mode results in flooding
traffic to all vde_switch ports. This results in a expected behaviour
and a ping on a VLAN subinterface works as expected.
Signed-off-by: Philippe Schaaf <philippe.schaaf@secunet.com>
Previously, the location logic was hardcoded, supporting only
Nixpkgs and NixOps properly, leaving other uses of the module
system without good location support.
- initialSystem was keeping track of the evaluating system
- it had been used by `nesting.children`
- since, 20.09, `nesting.children` has been replaced with named
specializations
It appears that this option was left over and not cleand up properly.
Without this fix, setting the shellopts in `machine.execute` is
inconsitent. When no timeout is used, shellopts `set -euo pipefail` are
applied to the command as expected. When a timeout is specified, the
shellopts are not applied to the command itself (which is called inside
a `sh -c` that doesn't inherit the shellopts) but rather to the
`timeout` command, leading to the following full command:
```bash
(set -euo pipefail; timeout 900 sh -c 'cmd') | (base64 --wrap 0; echo)\n
```
With this fix, this is the command we get:
```bash
timeout 900 sh -c 'set -euo pipefail; false | true') | (base64 --wrap 0; echo)\n
```
Documents the _module.args option, motivated by many usages in Flakes,
especially with the deprecation of extraArgs
(78ada83361)
The documentation rendering for this option had to be handled a bit
specially, since it's not declared in nixos/modules like all the other
NixOS options.
Co-Authored-By: pennae <github@quasiparticle.net>
Co-Authored-By: Robert Hensing <robert@roberthensing.nl>
This patch allows creation of files like
/etc/systemd/system/user-.slice.d/limits.conf with
systemd.units."user-.slice.d/limits.conf" = {
text = ''
[Slice]
CPUAccounting=yes
CPUQuota=50%
'';
};
which previously threw an error
Also renames the systemd-unit-path test to sytsemd-misc, and extends it to
test that `systemd.units` can handle directories. In this case we make
sure that resource limits specified in user slices apply.
Naively deduplicate VLANs in the python driver for NixOS tests. The
current implementation accidentally works, since the VLan class mutates
the environment. On construction it sets QEMU_VDE_SOCKET_${id} and this
environment variable gets overwritten once a second VLAN with the same
id is constructed. Because the NIC flags passed to qemu just use the
QEMU_VDE_SOCKET_${id} environment variable, this implicitly chooses a
single vde_switch process for each VLAN.
However, this leads to unusable vde_switch processes being spawned in
each test run and as a side effect makes it impossible to access the
correct VLan objects in the interactive test driver. It also makes it
remarkably hard to understand why the current implementation ever
worked.
it's really easy to accidentally write the wrong systemd Exec* directive, ones
that works most of the time but fails when users include systemd metacharacters
in arguments that are interpolated into an Exec* directive. add a few functions
analogous to escapeShellArg{,s} and some documentation on how and when to use them.
Makes all options rendered in the manual throw an error if they don't
have a type specified.
This is a follow-up to #76184
Co-Authored-By: Silvan Mosberger <contact@infinisil.com>
With the previous change that enabled error propagation through
`inherit_errexit`, the script would fail if `errexit` was set, but
`inherit_errexit` was not. This is due to `shopt -p` exiting with an
error if the option is disabled. To work around this, use the exit
code instead of the text value returned by `shopt -p`.
Fixes#160869.
The aarch64-linux versions of the boot.uefiUsb and boot.uefiCdrom tests
were broken by b0fc9da879.
That commit was a refactor which omitted the qemuBinary option, which was
previously available in the legacy start command. This restores that
option and fixes the tests previously mentioned.
If an error occurs while trying to read a secret file, we want that
error to propagate to the main shell context. That means we have to
set the `inherit_errexit` option, which allows errors from subshells
to propagate to the outer shell. Also, the subshell cannot run as part
of another command, such as `export`, since that will simply ignore
the subshell exit status and only respect `export`s exit status; first
assigning the value to a variable and then exporting it solves issue.
this partially solves the problem of "missing description" warnings of the
options doc build being lost by nix build, at the cost of failing builds that
previously ran. an option to disable this behaviour is provided.
link to search.nixos.org instead of pulling package metadata out of pkgs. this
lets us cache docs of a few more modules and provides easier access to package
info from the HTML manual, but makes the manpage slightly less useful since
package description are no longer rendered.
most modules can be evaluated for their documentation in a very
restricted environment that doesn't include all of nixpkgs. this
evaluation can then be cached and reused for subsequent builds, merging
only documentation that has changed into the cached set. since nixos
ships with a large number of modules of which only a few are used in any
given config this can save evaluation a huge percentage of nixos
options available in any given config.
in tests of this caching, despite having to copy most of nixos/, saves
about 80% of the time needed to build the system manual, or about two
second on the machine used for testing. build time for a full system
config shrank from 9.4s to 7.4s, while turning documentation off
entirely shortened the build to 7.1s.
by default all cores are used
hoping this will fix the hydra i686 squashfs build issues as all the
failures were using 64 cores
Parallel mksquashfs: Using 64 processors
Creating 4.0 filesystem on ..., block size 1048576.
FATAL ERROR: mangle2:: xz compress failed with error code 5
to do this we must replace derivations with attrsets in make-options-doc, since
xml can represent derivations differently from attrset but json cannot. this
also given asciidoc and mddoc the ability to handle derivation differently,
which they previously didn't have.
use the json file derivation we already have to also generate the asciidoc and
md options docs instead of formatting the options in nix. docbook docs are
already produced in derivations.
the new script produce the exact same output as the old in-nix generation.
When displaying the amount of time some step took, with no other
context, it becomes nigh impossible (especially in longer tests) to see
when specific steps finished.
The flag -cpu max leaves QEMU 6.1.0 stuck on some systems,
for example when /dev/kvm is not read-writable.
This does not happen with -cpu qemu64.
Getting stuck like that is a regression in 6.1.0 not yet present in 6.0.0
and should be fixed with 6.2.0 according to early testing with rc1.
We should consider reverting this change when we merge QEMU 6.2.0.
See #146526.
fixes#141596
