Deeply-curried functions are pretty error-prone in untyped languages
like Nix. This is a particularly bad case because
`top-level/splice.nix` *also* declares a makeScopeWithSplicing, but
it takes *two fewer arguments*.
Let's switch to attrset-passing form, to provide some minimal level
of sanity-checking.
ofborg relies on the behavior that existed prior to
1c00bf3948, where evaluation would
immediately abort due to a missing argument (whether it be an aliased
package when `allowAliases = false;` or a typo'd or otherwise
nonexistent package).
If `callPackageWith` `throw`s instead of `abort`s, the following
`nix-env` invocation does not fail fast but instead silently skips the
attribute (assuming there is a package that has an aliased package in
its `autoArgs`):
$ nix-env -qa --json --file . --arg config '{ allowAliases = false; }' &>/dev/null
$ echo $?
0
This does change the error output when there is a missing package (for
any of the reasons mentioned above), though. Before this change, the
errors looked like this:
$ nix-build -A hello --arg config '{ allowAliases = false; }'
error:
… while calling the 'throw' builtin
at /home/vin/workspace/vcs/nixpkgs/master/lib/customisation.nix:179:65:
178|
179| in if missingArgs == [] then makeOverridable f allArgs else throw error;
| ^
180|
error: Function called without required argument "bash_5" at /home/vin/workspace/vcs/nixpkgs/master/pkgs/applications/misc/hello/default.nix:8, did you mean "bash" or "bashdb"?
And the errors now look like this:
$ nix-build -A hello --arg config '{ allowAliases = false; }'
error:
… while calling the 'abort' builtin
at /home/vin/workspace/vcs/nixpkgs/master/lib/customisation.nix:179:65:
178|
179| in if missingArgs == [] then makeOverridable f allArgs else abort error;
| ^
180|
error: evaluation aborted with the following error message: 'Function called without required argument "bash_5" at /home/vin/workspace/vcs/nixpkgs/master/pkgs/applications/misc/hello/default.nix:8, did you mean "bash" or "bashdb"?'
By allowing null, we allow code to avoid filterAttrs, improving
laziness in real world use cases.
Specifically, this strategy prevents infinite recursion errors,
performance issues and possibly other errors that are unrelated to
the user's code.
The new derivation should evaluate only if the old derivation does.
Sadly this means that the old derivation cannot depend on the new one
any more, which was used by xorgserver on Darwin. But this is not a
problem as `overrideAttrs` can (and should) usually be used instead.
This change allowed catching an invalid `meta.platforms` in the linux_rpi
kernels, which use `overrideDerivation`.
This uses the levenshtein distance to look through all possible
arguments to find ones that are close to what was requested:
error: Function in /home/infinisil/src/nixpkgs/pkgs/tools/text/ripgrep/default.nix
called without required argument "fetchFromGithub",
did you mean "fetchFromGitHub" or "fetchFromGitLab"?
With https://github.com/NixOS/nix/pull/3468 (in current nixUnstable) the error
message becomes even better, adding line location info
the fix to extendDerivation in #140051 unwittingly worsened eval performance by
quite a bit. set elements alone needed over 1GB extra after the change, which
seems disproportionate to how small it was. if we flip the logic used to
determine which outputs to install around and keep a "this one exactly" flag in
the specific outputs instead of a "all of them" in the root we can avoid most
of that cost.
if extendDerivation is called on something that already had extendDerivation
called on it (eg a mkDerivation result) the second call will set
outputUnspecified=true on every output by way of propagating attributes of the
full derivation to the individual outputs. this in turn causes buildEnv--and
thus nix-shell and environment.systemPackages--to install every output of such a
derivation even when only a specific output was requested, which renders the
point of multiple outputs moot. this happens in python modules (see #139756),
though it seems that tcl and possibly others should also be affected.
I am taking the non-invasive parts of #110914 to hopefully help out with #111988.
In particular:
- Use `lib.makeScopeWithSplicing` to make the `darwin` package set have
a proper `callPackage`.
- Adjust Darwin `stdenv`'s overlays keeping things from the previous
stage to not stick around too much.
- Expose `binutilsNoLibc` / `darwin.binutilsNoLibc` to hopefully get us
closer to a unified LLVM and GCC bootstrap.
This allows querying function arguments of things like fetchFromGitHub:
nix-repl> lib.functionArgs pkgs.fetchFromGitHub
{ fetchSubmodules = true; githubBase = true; ... }
The `overrideScope` bound by `makeScope` (via special `callPackage`)
took an override in the form `super: self { … }`. But this is
dangerously close to the `self: super { … }` form used by *everything*
else, even other definitions of `overrideScope`! Since that
implementation did not even share any code either until I changed it
recently in 3cf43547f4, this inconsistency
is almost certainly an oversight and not intentional.
Unfortunately, just as the inconstency is hard to debug if one just
assumes the conventional order, any sudden fix would break existing
overrides in the same hard-to-debug way. So instead of changing the
definition a new `overrideScope'` with the conventional order is added,
and old `overrideScope` deprecated with a warning saying to use
`overrideScope'` instead. That will hopefully get people to stop using
`overrideScope`, freeing our hand to change or remove it in the future.
Another attempt after my sloppy 48ccdf322d.
@Infinisil, thanks again, reverted in 4794aa5de2 and explained my mistakes in 48ccdf322d (commitcomment-29678643). I start with their work and provide this proof of this commit's correctness:
```nix
(lib.fixedPoints.extends (lib.flip g) f) # now
((f: rattrs: self: let super = rattrs self; in super // f self super) (lib.flip g) f) # inline extends
(self: let super = f self; in super // (lib.flip g) self super) # beta reduce
(self: let super = f self; in super // g super self) # beta reduce
(self_: let super = f self_; in super // g super self_) # alpha rename
(self_: let super = f self_; in super // g super self_) # original, same
```
Eventually we might harmonize `overrideScope`'s `g` parameter with the general pattern, but I leave that breaking change as a separate step. Best not to refactor and break at once, and at least the abstractions make the oddity clearer.
Among other things, this will allow *2nix tools to output plain data
while still being composable with the traditional
callPackage/.override interfaces.
addPassthru became unused in #33057, but its signature was changed at the same
time. This commit restores the original signature and updates the warning and
the changelog.
This does break the API of being able to import any lib file and get
its libs, however I'm not sure people did this.
I made this while exploring being able to swap out docFn with a stub
in #2305, to avoid functor performance problems. I don't know if that
is going to move forward (or if it is a problem or not,) but after
doing all this work figured I'd put it up anyway :)
Two notable advantages to this approach:
1. when a lib inherits another lib's functions, it doesn't
automatically get put in to the scope of lib
2. when a lib implements a new obscure functions, it doesn't
automatically get put in to the scope of lib
Using the test script (later in this commit) I got the following diff
on the API:
+ diff master fixed-lib
11764a11765,11766
> .types.defaultFunctor
> .types.defaultTypeMerge
11774a11777,11778
> .types.isOptionType
> .types.isType
11781a11786
> .types.mkOptionType
11788a11794
> .types.setType
11795a11802
> .types.types
This means that this commit _adds_ to the API, however I can't find a
way to fix these last remaining discrepancies. At least none are
_removed_.
Test script (run with nix-repl in the PATH):
#!/bin/sh
set -eux
repl() {
suff=${1:-}
echo "(import ./lib)$suff" \
| nix-repl 2>&1
}
attrs_to_check() {
repl "${1:-}" \
| tr ';' $'\n' \
| grep "\.\.\." \
| cut -d' ' -f2 \
| sed -e "s/^/${1:-}./" \
| sort
}
summ() {
repl "${1:-}" \
| tr ' ' $'\n' \
| sort \
| uniq
}
deep_summ() {
suff="${1:-}"
depth="${2:-4}"
depth=$((depth - 1))
summ "$suff"
for attr in $(attrs_to_check "$suff" | grep -v "types.types"); do
if [ $depth -eq 0 ]; then
summ "$attr" | sed -e "s/^/$attr./"
else
deep_summ "$attr" "$depth" | sed -e "s/^/$attr./"
fi
done
}
(
cd nixpkgs
#git add .
#git commit -m "Auto-commit, sorry" || true
git checkout fixed-lib
deep_summ > ../fixed-lib
git checkout master
deep_summ > ../master
)
if diff master fixed-lib; then
echo "SHALLOW MATCH!"
fi
(
cd nixpkgs
git checkout fixed-lib
repl .types
)
