This reverts commit PR #167947.
Flakes aren't standardised and the `lib` namespace shouldn't be
polluted with utilities that serve only experimental uses.
This commit creates flakes.nix, which is a library containing functions
which relate to interacting with flakes. It also moves related functions
from trivial.nix into it.
This is essentially a copy of the function of the same name, from
flake-compat. callLocklessFlake is useful when trying to utilise a
flake.nix without a lock file, often for when you want to create a
subflake from within a parent flake.
Co-authored-by: Tom Bereknyei <tomberek@gmail.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Move function spdxLicense, internally used in yarn2nix
to lib/meta.nix, and
rename to getLicenseFromSpdxId
A similar function is implemented in poetry2nix,
but the one originally in yarn2nix seems beter.
since it falls back to an license-like attrset
for mismatched case
instead of a plain string
mkDerivedConfig : Option a -> (a -> Definition b) -> Definition b
Create config definitions with the same priority as the definition of another option.
This should be used for option definitions where one option sets the value of another as a convenience.
For instance a config file could be set with a `text` or `source` option, where text translates to a `source`
value using `mkDerivedConfig options.text (pkgs.writeText "filename.conf")`.
It takes care of setting the right priority using `mkOverride`.
It's a common pattern in Nixpkgs to want to emit a warning in certain
cases, but not actually change behaviours.
This is often expressed as either
if cond then lib.warn "Don't do that thing" x else x
Or
(if cond then lib.warn "Don't do that thing" else lib.id) x
Neither of which really expresses the intent here, because it looks
like 'x' is being chosen conditionally.
To make this clearer, I introduce a "warnIf" function, which makes it
clear that the only thing being affected by the condition is whether
the warning is generated, not the value being returned.
Immensely helpful when you want to see the changes a function makes to
its value as it passes through.
Example:
```
$ nix-instantiate --strict --eval -E '(with import ./lib; traceFnSeqN 2 "id" (x: x) { a.b.c = 3; })'
trace: {
fn = "id";
from = {
a = {
b = {…};
};
};
to = {
a = {
b = {…};
};
};
}
{ a = { b = { c = 3; }; }; }
```
Nix can perform static scope checking, but whenever code is inside
a `with` expression, the analysis breaks down, because it can't
know statically what's in the attribute set whose attributes were
brought into scope. In those cases, Nix has to assume that
everything works out.
Except it doesnt. Removing `with` from lib/ revealed an undefined
variable in an error message.
If that doesn't convince you that we're better off without `with`,
I can tell you that this PR results in a 3% evaluation performance
improvement because Nix can look up local variables by index.
This adds up with applications like the module system.
Furthermore, removing `with` makes the binding site of each
variable obvious, which helps with comprehension.
`toHex` converts the given positive integer to a string of the hexadecimal
representation of that integer. For example:
```
toHex 0 => "0"
toHex 16 => "10"
toHex 250 => "FA"
```
`toBase base i` converts the positive integer `i` to a list of it
digits in the given `base`. For example:
```
toBase 10 123 => [ 1 2 3 ]
toBase 2 6 => [ 1 1 0 ]
toBase 16 250 => [ 15 10 ]
```
The previous hash was too short and caused evaluation-time errors like:
invalid SRI hash 'sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA='
Additionally, since the fact that this is broken implies that nobody
could have been using it, "SRI" is a bit of a vague and obscure term,
`fakeSriHash` would be somewhat of a mouthful, and the relevant fetcher
parameters are just called `hash`, rename it to `fakeHash`.
lib/cli is very similar to generators, so it should follow largely the
same interface. Similar to how generators isn’t exported, we should
also namespace cli by default (plus “cli” is only three characters to
type).
Before c9214c394b and
9d396d2e42 if .git is symlink the version
would gracefully default to no git revision. With those changes an
exception is thrown instead.
This introduces a new function `pathIsGitRepo` that checks if
`commitIdFromGitRepo` fails without error so we don't have to
reimplement this logic again and can fail gracefully.
lib.commitIdFromGitRepo now resolves the refs from the
parent repository in case the supplied path is a file
containing the path to said repository. this adds support
for git-worktree and things alike. see gitrepository-layout(5).
this also:
- adds a new boolean function lib.pathIsRegularFile to
check whether a path is a regular file
- patches lib.revisionWithDefault and
the revision and versionSuffix attributes in
config.system.nixos in order to support git-worktrees
With this change, disabledModules applies recursively, meaning if you
have a module "foo.nix" with
imports = [ ./bar.nix ];
then setting
disabledModules = [ "foo.nix" ];
will disable both "foo.nix" and "bar.nix", whereas previously only
"foo.nix" would be disabled.
This change along with https://github.com/NixOS/nixpkgs/pull/61570 allows
modules to be fully disabled even when they have some `mkRenamedOption`
imports.
This adds a new utility to intelligently convert Nix records to
command line options to reduce boilerplate for simple use cases and to
also reduce the likelihood of malformed command lines
`pipe` is a useful operator for creating pipelines of functions.
It works around the usual problem of e.g. string operations becoming
deeply nested functions.
In principle, there are four different ways this function could be
written:
pipe val [ f1 .. fn ]
pipe val [ fn .. f1 ]
compose [ f1 .. fn ] val
compose [ fn .. f1 ] val
The third and fourth form mirror composition of functions, they would
be the same as e.g. `(f1 << f2 << f3 .. << fn) val`.
However, it is not clear which direction the list should have (as one
can see in the second form, which is the most absurd.
In order not to confuse users, we decide for the most “intuitive”
form, which mirrors the way unix pipes work (thus the name `pipe`).
The flow of data goes from left to right.
Co-Authored-By: Silvan Mosberger <infinisil@icloud.com>
Remove the "version" parameter in order to make it more widely
available.
Starts making some kernel configuration helpers available.
The intent is to be able to better build and check the linux kernel
configuration.
This makes the function available without having to evaluate the
Nixpkgs fix-point, making it available in a more natural way for
code that deals with multiple Nixpkgs invocations.
Its definition is coupled to Nix rather than Nixpkgs, so it will
feel right at home in lib.
The main purpose is to bring attention to `flip map`, which improves
code readablity. It is useful when ad-hoc anonymous function
grows two or more lines in `map` application:
```
map (lcfg:
let port = lcfg.port;
portStr = if port != defaultPort then ":${toString port}" else "";
scheme = if cfg.enableSSL then "https" else "http";
in "${scheme}://cfg.hostName${portStr}"
) (getListen cfg);
```
Compare this to `foreach`-style:
```
foreach (getListen cfg) (lcfg:
let port = lcfg.port;
portStr = if port != defaultPort then ":${toString port}" else "";
scheme = if cfg.enableSSL then "https" else "http";
in "${scheme}://cfg.hostName${portStr}"
);
```
This is similar to Haskell's `for` (http://hackage.haskell.org/package/base-4.12.0.0/docs/Data-Traversable.html#v:for)
This commit changes the `mkAliasOptionModule` function to make sure that
the priority for the aliased option is propagated to the non-aliased
option.
This also affects the `mkRenamedOptionModule` function in a similar
fashion.
This also removes the `mkAliasOptionModuleWithPriority` function, since
its functionality is now subsumed by `mkAliasOptionModule`.
This change was recommended by @nbp:
https://github.com/NixOS/nixpkgs/pull/53397#discussion_r245487432
Fake hashes can be used as placeholders for all the places, where
Nix expression requires a hash, but we don't yet have one.
This should be more convenient than following:
- echo|sha256sum, copy into clipboard, go to editor, paste into previously
edited place
- search nixpkgs for a random package, copy it's hash to cliboard, go to
editor, paste into previously edited place
Nix can add support for these fake hashes. In that case printed error should contain
only 1 hash, so no more problem "which of two hashes from error should I use?"
Idea by irc:Synthetica
This commit adds a function `mkAliasOptionModuleWithPriority`. This
function will make an alias to an existing option and copy over the
priority.
This functionality is needed for PRs like #53041. In that case
`nixos-generate-config` added an option to `hardware-configuration.nix`
with `mkDefault`. That option was then changed and an alias created for
the old name.
The end user should be able to set the non-alias option in their
`configuration.nix` and have everything work correctly. Without this
function, the priority for the option won't be copied over correctly
and the end-user will get a message saying they have the same option
set to two different values.
Suppose I have a Gemfile like this:
source "https://rubygems.org"
gem "actioncable"
gem "websocket-driver", group: :test
The gemset.nix generated by Bundix 2.4.1 will set ActionCable's groups
to [ "default" ], and websocket-driver's to [ "test" ]. This means that
the generated bundlerEnv wouldn't include websocket-driver unless the
test group was included, even though it's required by the default group.
This is arguably a bug in Bundix (websocket-driver's groups should
probably be [ "default" "test" ] or just [ "default" ]), but there's no
reason bundlerEnv should omit dependencies even given such an input --
it won't necessarily come from Bundix, and it would be good for
bundlerEnv to do the right thing.
To fix this, filterGemset is now a recursive function, that adds
dependencies of gems in the group to the filtered gemset until it
stabilises on the gems that match the required groups, and all of their
recursive dependencies.
- moved function into strings.nix
- renamed function from makePerl5Lib
- removed duplicates entries in the resulting value
- rewrote the function from scratch after learning a few things (much cleaner now)
As suggested in https://github.com/NixOS/nixpkgs/pull/39416#discussion_r183845745
the versioning attributes in `lib` should be consistent to
`nixos/version` which implicates the following changes:
* `lib.trivial.version` -> `lib.trivial.release`
* `lib.trivial.suffix` -> `lib.trivial.versionSuffix`
* `lib.nixpkgsVersion` -> `lib.version`
As `lib.nixpkgsVersion` is referenced several times in `NixOS/nixpkgs`,
`NixOS/nix` and probably several user's setups. As the rename will cause
a notable impact it's better to keep `lib.nixpkgsVersion` as alias with
a warning yielded by `builtins.trace`.
This allows the lib fixed point to be extended with
myLib = lib.extend (self: super: {
foo = "foo";
})
With this it's possible to have the new modified lib attrset available to all
modules when using evalModules
myLib.evalModules {
modules = [ ({ lib, ... }: {
options.bar = lib.mkOption {
default = lib.foo;
};
}) ];
}
=> { config = { bar = "foo"; ... }; options = ...; }
First, we need check against the host platform, not the build platform.
That's simple enough.
Second, we move away from exahustive finite case analysis (i.e.
exhaustively listing all platforms the package builds on). That only
work in a closed-world setting, where we know all platforms we might
build one. But with cross compilation, we may be building for arbitrary
platforms, So we need fancier filters. This is the closed world to open
world change.
The solution is instead of having a list of systems (strings in the form
"foo-bar"), we have a list of of systems or "patterns", i.e. attributes
that partially match the output of the parsers in `lib.systems.parse`.
The "check meta" logic treats the systems strings as an exact whitelist
just as before, but treats the patterns as a fuzzy whitelist,
intersecting the actual `hostPlatform` with the pattern and then
checking for equality. (This is done using `matchAttrs`).
The default convenience lists for `meta.platforms` are now changed to be
lists of patterns (usually a single pattern) in
`lib/systems/for-meta.nix` for maximum flexibility under this new
system.
Fixes#30902
Based on a request by @oxij:
“Can we also rename this file to `maintainers/maintainers-list.nix` while we at
this? Motivation: much saner `git log ./lib`.”
Based on https://github.com/NixOS/nixpkgs/pull/34842, the
nix-instantiate output was pretty-printed and the validity of the github handles
manually verified, by automatically checking whether the user handles exist on
github (https://github.com/userhandle, status 200 or 404).
Each handle under 5 characters was manually checked (because the collision
probability with non-maintainer accounts is high), each missing entry was
manually researched.
The script used is kept in `maintainers/scripts` as an example of how to work
with the mainainers list through nix’ JSON interface.
Among other things, this will allow *2nix tools to output plain data
while still being composable with the traditional
callPackage/.override interfaces.
This does break the API of being able to import any lib file and get
its libs, however I'm not sure people did this.
I made this while exploring being able to swap out docFn with a stub
in #2305, to avoid functor performance problems. I don't know if that
is going to move forward (or if it is a problem or not,) but after
doing all this work figured I'd put it up anyway :)
Two notable advantages to this approach:
1. when a lib inherits another lib's functions, it doesn't
automatically get put in to the scope of lib
2. when a lib implements a new obscure functions, it doesn't
automatically get put in to the scope of lib
Using the test script (later in this commit) I got the following diff
on the API:
+ diff master fixed-lib
11764a11765,11766
> .types.defaultFunctor
> .types.defaultTypeMerge
11774a11777,11778
> .types.isOptionType
> .types.isType
11781a11786
> .types.mkOptionType
11788a11794
> .types.setType
11795a11802
> .types.types
This means that this commit _adds_ to the API, however I can't find a
way to fix these last remaining discrepancies. At least none are
_removed_.
Test script (run with nix-repl in the PATH):
#!/bin/sh
set -eux
repl() {
suff=${1:-}
echo "(import ./lib)$suff" \
| nix-repl 2>&1
}
attrs_to_check() {
repl "${1:-}" \
| tr ';' $'\n' \
| grep "\.\.\." \
| cut -d' ' -f2 \
| sed -e "s/^/${1:-}./" \
| sort
}
summ() {
repl "${1:-}" \
| tr ' ' $'\n' \
| sort \
| uniq
}
deep_summ() {
suff="${1:-}"
depth="${2:-4}"
depth=$((depth - 1))
summ "$suff"
for attr in $(attrs_to_check "$suff" | grep -v "types.types"); do
if [ $depth -eq 0 ]; then
summ "$attr" | sed -e "s/^/$attr./"
else
deep_summ "$attr" "$depth" | sed -e "s/^/$attr./"
fi
done
}
(
cd nixpkgs
#git add .
#git commit -m "Auto-commit, sorry" || true
git checkout fixed-lib
deep_summ > ../fixed-lib
git checkout master
deep_summ > ../master
)
if diff master fixed-lib; then
echo "SHALLOW MATCH!"
fi
(
cd nixpkgs
git checkout fixed-lib
repl .types
)
Many configurations are INI-style files. Attribute sets can be mapped
rather painlessly to the INI format.
This adds a function toINI inside a new generators library section.
Also, unit tests for the default values are provided.
The major changes are:
* The evaluation is now driven by the declared options. In
particular, this fixes the long-standing problem with lack of
laziness of disabled option definitions. Thus, a configuration like
config = mkIf false {
environment.systemPackages = throw "bla";
};
will now evaluate without throwing an error. This also improves
performance since we're not evaluating unused option definitions.
* The implementation of properties is greatly simplified.
* There is a new type constructor "submodule" that replaces
"optionSet". Unlike "optionSet", "submodule" gets its option
declarations as an argument, making it more like "listOf" and other
type constructors. A typical use is:
foo = mkOption {
type = type.attrsOf (type.submodule (
{ config, ... }:
{ bar = mkOption { ... };
xyzzy = mkOption { ... };
}));
};
Existing uses of "optionSet" are automatically mapped to
"submodule".
* Modules are now checked for unsupported attributes: you get an error
if a module contains an attribute other than "config", "options" or
"imports".
* The new implementation is faster and uses much less memory.
