most modules can be evaluated for their documentation in a very
restricted environment that doesn't include all of nixpkgs. this
evaluation can then be cached and reused for subsequent builds, merging
only documentation that has changed into the cached set. since nixos
ships with a large number of modules of which only a few are used in any
given config this can save evaluation a huge percentage of nixos
options available in any given config.
in tests of this caching, despite having to copy most of nixos/, saves
about 80% of the time needed to build the system manual, or about two
second on the machine used for testing. build time for a full system
config shrank from 9.4s to 7.4s, while turning documentation off
entirely shortened the build to 7.1s.
As suggested in #131205.
Now it's possible to pretty-print a value with `lib.generators` like
this:
with lib.generators;
toPretty { }
(withRecursion { depthLimit = 10; } /* arbitrarily complex value */)
Also, this can be used for any other pretty-printer now if needed.
When having a bogus declaration such as
{ lib, ... }:
{
foo.bar = mkOption {
type = types.str;
};
}
the evaluation will terminate with a not-so helpful
error: stack overflow (possible infinite recursion)
To make sure a useful error is still provided, I added a `depthLimit` of
`10` which should be perfectly sufficient to `toPretty` when it's used
in an error-case for `showDefs`.
For an option definition that uses `lib.options.mergeEqualOption`
underneath, like `types.anything`, an error is thrown when multiple
functions are assigned, indicating that functions can't be compared for
equivalence:
error: The option `test' has conflicting definition values:
- In `xxx': <function>
- In `xxx': <function>
(use '--show-trace' to show detailed location information)
However, the error message didn't use the correct files. While above
error indicates that both definitions are in the xxx file, that's in
fact not true. Above error was generated with
options.test = lib.mkOption {
type = lib.types.anything;
};
imports = [
{
_file = "yyy";
test = y: y;
}
{
_file = "xxx";
test = x: x;
}
];
With this change, the error uses the correct file locations:
error: The option `test' has conflicting definition values:
- In `xxx': <function>
- In `yyy': <function>
(use '--show-trace' to show detailed location information)
Initially https://github.com/NixOS/nixpkgs/pull/82897 prevented
non-visible options from being rendered in the manual, but
visible-but-internal options were still being recursed into. This fixes
this, aligning the recurse condition here with the one in
make-options-doc/default.nix
Nix can perform static scope checking, but whenever code is inside
a `with` expression, the analysis breaks down, because it can't
know statically what's in the attribute set whose attributes were
brought into scope. In those cases, Nix has to assume that
everything works out.
Except it doesnt. Removing `with` from lib/ revealed an undefined
variable in an error message.
If that doesn't convince you that we're better off without `with`,
I can tell you that this PR results in a 3% evaluation performance
improvement because Nix can look up local variables by index.
This adds up with applications like the module system.
Furthermore, removing `with` makes the binding site of each
variable obvious, which helps with comprehension.
This is used in in the manual generation for option identifiers that can
be linked. This, unike what the example describes, doesn't preserve
quotes which is needed for these identifiers to be valid.
This reverts commit 124cccbe3b.
124cccbe3b
broke the build of NixOS manual.
It does not make sense to be as strict as with attributes since we
are not limited by the CLI's inability to handle numbers.
Placeholders should not be quoted either as they are not part of Nix
syntax but a meta-level construct.
The error can be reproduced like:
```
$ nix-instantiate ./nixos -A system --arg configuration '
{ fileSystems."/".device = "nodev";
boot.loader.grub.devices = [ "nodev" ];
containers.t.config.imports = [ <nixpkgs/nixos/modules/virtualisation/amazon-image.nix> ];
}'
```
Previously error was:
```
error: The unique option `containers.t.networking.hostName' is defined multiple times, in `/nix/var/nix/profiles/per-user/root/channels/nixpkgs/nixos/modules/virtualisation/amazon-image.nix' and `module at /home/danbst/dev/nixpkgs/nixos/modules/virtualisation/containers.nix:470'.
(use '--show-trace' to show detailed location information)
```
Now it is:
```
error: The unique option `containers.t.networking.hostName' is defined multiple times, in:
- /nix/var/nix/profiles/per-user/root/channels/nixpkgs/nixos/modules/virtualisation/amazon-image.nix
- module at /home/danbst/dev/nixpkgs/nixos/modules/virtualisation/containers.nix:470.
(use '--show-trace' to show detailed location information)
```
Related: https://github.com/NixOS/nixpkgs/issues/15747
The explicit remove helped to uncover some hidden uses of `optionSet`
in NixOps. However it makes life harder for end-users of NixOps - it will
be impossible to deploy 19.03 systems with old NixOps, but there is no
new release of NixOps with `optionSet` fixes.
Also, "deprecation" process isn't well defined. Even that `optionSet` was
declared "deprecated" for many years, it was never announced. Hence, I
leave "deprecation" announce. Then, 3 releases after announce,
we can announce removal of this feature.
This type has to be removed, not `throw`-ed in runtime, because it makes
some perfectly fine code to fail. For example:
```
$ nix-instantiate --eval -E '(import <nixpkgs/lib>).types' --strict
trace: `types.list` is deprecated; use `types.listOf` instead
error: types.optionSet is deprecated; use types.submodule instead
(use '--show-trace' to show detailed location information)
```
Documents functions in `lib.options` for docs generation with nixdoc.
The formatting change in the `mkOption` arguments is due to the way
`nixdoc` parses documentation comments on pattern arguments. It's not
ideal, but it works.
Handle the case where options have funny symbols inside of them.
Example:
If I reference the following attribute without it being defined:
security.acme.certs."example.com".webroot
I now get the error:
The option `security.acme.certs."example.com".webroot' is used but
not defined.
where before I got:
The option `security.acme.certs.example.com.webroot' is used but
not defined.
which is not true.
This allows one to specify "related packages" in NixOS that get rendered into
the configuration.nix(5) man page. The interface philosophy is pretty much
stolen from TeX bibliography.
See the next several commits for examples.
This allows one to specify "related packages" in NixOS that get rendered into
the configuration.nix(5) man page. The interface philosophy is pretty much
stolen from TeX bibliography.
This does break the API of being able to import any lib file and get
its libs, however I'm not sure people did this.
I made this while exploring being able to swap out docFn with a stub
in #2305, to avoid functor performance problems. I don't know if that
is going to move forward (or if it is a problem or not,) but after
doing all this work figured I'd put it up anyway :)
Two notable advantages to this approach:
1. when a lib inherits another lib's functions, it doesn't
automatically get put in to the scope of lib
2. when a lib implements a new obscure functions, it doesn't
automatically get put in to the scope of lib
Using the test script (later in this commit) I got the following diff
on the API:
+ diff master fixed-lib
11764a11765,11766
> .types.defaultFunctor
> .types.defaultTypeMerge
11774a11777,11778
> .types.isOptionType
> .types.isType
11781a11786
> .types.mkOptionType
11788a11794
> .types.setType
11795a11802
> .types.types
This means that this commit _adds_ to the API, however I can't find a
way to fix these last remaining discrepancies. At least none are
_removed_.
Test script (run with nix-repl in the PATH):
#!/bin/sh
set -eux
repl() {
suff=${1:-}
echo "(import ./lib)$suff" \
| nix-repl 2>&1
}
attrs_to_check() {
repl "${1:-}" \
| tr ';' $'\n' \
| grep "\.\.\." \
| cut -d' ' -f2 \
| sed -e "s/^/${1:-}./" \
| sort
}
summ() {
repl "${1:-}" \
| tr ' ' $'\n' \
| sort \
| uniq
}
deep_summ() {
suff="${1:-}"
depth="${2:-4}"
depth=$((depth - 1))
summ "$suff"
for attr in $(attrs_to_check "$suff" | grep -v "types.types"); do
if [ $depth -eq 0 ]; then
summ "$attr" | sed -e "s/^/$attr./"
else
deep_summ "$attr" "$depth" | sed -e "s/^/$attr./"
fi
done
}
(
cd nixpkgs
#git add .
#git commit -m "Auto-commit, sorry" || true
git checkout fixed-lib
deep_summ > ../fixed-lib
git checkout master
deep_summ > ../master
)
if diff master fixed-lib; then
echo "SHALLOW MATCH!"
fi
(
cd nixpkgs
git checkout fixed-lib
repl .types
)
Previously, conflicting definitions would merge to "true". Now they
give an error, e.g.
error: The option `hardware.enableAllFirmware' has conflicting definitions, in `/etc/nixos/configurations/misc/eelco/stuff.nix' and `/etc/nixos/configurations/misc/eelco/mandark.nix'.
E.g.
The unique option `fileSystems./.device' is defined multiple times, in `/etc/nixos/configuration.nix' and `/etc/nixos/foo.nix'.
This requires passing file/value tuples to the merge functions.
For instance, if time.timeZone is defined multiple times, you now get
the error message:
error: user-thrown exception: The unique option `time.timeZone' is defined multiple times, in `/etc/nixos/configurations/misc/eelco/x11vnc.nix' and `/etc/nixos/configuration.nix'.
while previously you got:
error: user-thrown exception: Multiple definitions of string. Only one is allowed for this option.
and only an inspection of the stack trace gave a clue as to what
option caused the problem.
