makeOverridable is very careful to ensure the arguments to the
overridden function are the same as the input function. As a result,
the arguments of hello.override are exactly the same as the original
arguments of the hello function that produced the derivation.
However, callPackagesWith calls makeOverridable with a lambda that
does not propagate the arguments. The override function for a package
instantiated with callPackagesWith will not have the original
arguments.
For example:
nix-repl> lib.functionArgs hello.override
{ callPackage = false; fetchurl = false; hello = false; lib = false; nixos = false; stdenv = false; testers = false; }
nix-repl> lib.functionArgs openssl.override
{ }
By copying the arguments onto the inner lambda before passing it to
makeOverridable, we can make callPackage and callPackages behave the
same.
nix-repl> lib.functionArgs openssl.override
{ buildPackages = false; coreutils = false; cryptodev = false; enableSSL2 = true; enableSSL3 = true; fetchurl = false; lib = false; perl = false; removeReferencesTo = false; static = true; stdenv = false; withCryptodev = true; withPerl = true; }
This was found when trying to run the fileset tests on Darwin
(https://github.com/NixOS/nix/pull/9546#issuecomment-1967409445), which mysteriously fail on Darwin:
test case at lib/fileset/tests.sh:342 failed: toSource { root = "/nix/store/foobar"; fileset = ./.; } should have errored with this regex pattern:
lib.fileset.toSource: `root` \(/nix/store/foobar\) is a string-like value, but it should be a path instead.
\s*Paths in strings are not supported by `lib.fileset`, use `lib.sources` or derivations instead.
but this was the actual error:
error: lib.fileset.toSource: `root` (/nix/store/foobar) is a string-like value, but it should be a path instead.
Paths in strings are not supported by `lib.fileset`, use `lib.sources` or derivations instead.
After dissecting this, I find out that apparently \s works on Linux, but not on Darwin for some reason!
From the bash source code, it looks like <regex.h> with `REG_EXTENDED` is used for all platforms the same,
so there's nothing odd there.
It's almost impossible to know where <regex.h> comes from,
but it looks to be a POSIX thing.
So after digging through the almost impossible to find POSIX specifications
(https://pubs.opengroup.org/onlinepubs/007908799/xbd/re.html#tag_007_003_005),
I can indeed confirm that there's no mention of \s or the like!
_However_, there is a mention of `[[:blank:]]`, so we'll use that instead.
The previous code was optimized for the old uniq behavior, which did not
call merge. That's changed, so the legacy path is not a hot path anymore,
and is not worth any tech debt.
The deprecation warnings in lib were wildly inconsistent. Different
formulations were used in different places for the same meaning. Some warnings
used builtins.trace instead of lib.warn, which prevents silencing; one even
only had a comment instead. Make everything more uniform.
This allows for adding new, conditionally set, derivation attributes
to an existing derivation without changing any output paths in the
case where the condition is not met.
The only reason shallow clones are not the default in
`builtins.fetchGit` is that `revCount` can't be provided when cloning a
shallow repository. However, `revCount` isn't used or exposed by
`lib.fileset`. Hence, allowing cloning shallow repositories makes
`gitTracked` more general without any drawbacks.
Co-authored-by: Silvan Mosberger <github@infinisil.com>
It is useful that all (or almost all) .nix files in nixpkgs at least
parse since it allows for checking syntax in the repository
programmatically without evaluating anything.
This doesn't change uniq. Why not?
- In NixOS it seems that uniq is only used with
simple types that are fully checked by t.check.
- It exists for much longer and is used more widely.
- I believe we should deprecate it, because unique was
already better.
- unique can be a proving ground.
According to the WebAssembly design doc, wasm32 is an ILP32 ABI like
x32, mips64n32, and aarch64_ilp32 (Apple Watch). This commits adds
it to the predicate.
1319968ca5/CAndC%2B%2B.md (L16)
Most of the time when we do a patchelf conditional on
hostPlatform.isLinux, what we really mean is hostPlatform.isElf.
Now that we are starting to support BSDs, this is becoming more important.
Type: either ints.positive (enum ["auto"])
Before: positive integer, meaning >0 or value "auto" (singular enum)
After: positive integer, meaning >0, or value "auto" (singular enum)
Currently, the `lib/tests/modules.sh` test checks the output of
`nix-instantiate --eval` without `--json`, which outputs an unspecified
human-readable format.
This patch modifies `modules.sh` to use the `--json` output instead, to
be robust against future changes to `nix-instantiate` output.
This commit temporarily adds pkgs/test/release to the
lib/tests/release.nix test suite, because ofborg already knows about
that entry point.
We should move the list of test entry points out of ofborg and into
a central place in nixpkgs:
https://github.com/NixOS/nixpkgs/issues/272591
Once we do that we won't need to have this ugly kludge in an
inappropriate place.
* Improves the comments of `lib/flake-version-info.nix` and drops the
`__`-prefix from the filename.
* `lib'` -> `lib0` in `nixpkgs/lib`.
* Drop the declaration of `trivial.version` in the overlay because this
declaration already uses the final expressions of `versionSuffix` and
`release` now.
* No need to fall back to `self.lastModified` anymore, this was a
workaround for pre2.4 Nix.
Co-authored-by: Robert Hensing <robert@roberthensing.nl>
Co-authored-by: Silvan Mosberger <contact@infinisil.com>
That way each expression uses the final version of other lib.trivial
declarations.
For instance, when replacing `versionSuffix` with the string `"fnord"`
in a lib overlay, `trivial.version` uses `"fnord"` as suffix now rather
than `pre-git`.
A more efficient sort in some cases, and often convenient.
This exposes `lib.lists.sortOn` immediately on `lib`, because it is
a sibling of `sort`, which is already present there.
Omitting it would lead to more confusion, and worse outcomes.
There's no confusion about the types `sort` or `sortOn` operate on.
Haskell agrees about the type for `sortOn`, and it is in its `base`.
This type is necessary to have correct merging behavior for
`allowUnfreePredicate` and `allowInsecurePredicate`
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
For the time being, we're moving towards https://nix.dev/ containing
all tutorials and guides. The Nixpkgs manual is reinforced to be a
_reference_ manual. While it's not just reference for now, that's what
the docs team is working towards.
This commits rewrites the Nixpkgs manual introduction to reflect that
and point to some more useful links. The contribution docs are updated
similarly so it's not missed.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
An important idea around the rust stuff in lib.systems is that it's
elaborated — this means that it should idempotently add to the values
passed in, if any. But we missed that the names used for the
parameter and the elaborated value for "rustcTarget"/"config" didn't
line up. The intention was to use "rustcTarget" everywhere in the new
interface, as a more descriptive name than "config".
This fixes setting the system in NixOS configuration, which results in
an already elaborated system being elaborated again. Before, this
wouldn't produce the correct result:
% nix-instantiate --eval -A stdenv.hostPlatform.rust.rustcTarget --system armv7l-linux
"armv7-unknown-linux-gnueabihf"
% NIX_PATH= nix-instantiate --eval -E '(import nixos/lib/eval-config.nix { system = "armv7l-linux"; modules = []; }).pkgs.stdenv.hostPlatform.rust.rustcTarget'
"arm-unknown-linux-gnueabihf"
Fixes: e3e57b8f18 ("lib.systems: elaborate Rust metadata")
Fixes: https://github.com/NixOS/nixpkgs/issues/271000
Usually, attributes passed explicitly to elaborate take precedence
over the elaborated ones, but since we also elaborate the nested
"rust" attrset, we need to push that one level down, so the rest of
"rust" is still filled in if you just pass
{ rust = { config = ... } }.
I've had to drop the assertion that checked that at most one of "rust"
and "rustc" was part of the un-elaborated system, because doing this
broke passing an elaborated system in, which should be idempotent.
For the same reason, I've also had to make it possible for
rust.rustcTargetSpec to be passed in. Otherwise, on the second call,
since platform was filled in by the first, the custom target file
would be constructed. The only other way to avoid this would be to
compare the platform attrs to all built in Rust targets to check it
wasn't one of those, and that isn't feasible.
Fixes: e3e57b8f18 ("lib.systems: elaborate Rust metadata")
- Make fromSource's missing file error message more consistent with others,
and add a test for it
- Indent some function arguments
- Fix an internal type
The aws-sdk-cpp tests are flaky.
Since pull requests to staging cause nix to be rebuilt, this means
that staging PRs end up getting false CI failures due to whatever is
flaky in the AWS SDK tests. Since none of our CI needs to (or
should be able to) contact AWS S3, let's just omit it all. Bonus:
the tests build way faster.
While this change is backwards-incompatible, I think it's okay because:
- The `fileFilter` function is not yet in a stable NixOS release, it was only merged about [a month ago](https://github.com/NixOS/nixpkgs/pull/257356).
- All public uses of the function on GitHub only pass a path
- Any `fileFilter pred fileset` can also be expressed as `intersection fileset (fileFilter pred path)` without loss of functionality.
- This is furthermore pointed out in the new error message when a file set is passed
Along the lines of `assertOneOf`, but expects a list of values to be
compared. This gives a good error message and is useful for lists of
values, like `supportedGhcVersions` in the arguments of
`haskell-language-server`.
We need this stuff to be available in lib so make-derivation.nix can
access it to construct the Meson cross file.
This has a couple of other advantages:
- It makes Rust less special. Now figuring out what Rust calls a
platform is the same as figuring out what Linux or QEMU call it.
- We can unify the schema used to define Rust targets, and the schema
used to access those values later. Just like you can set "config"
or "system" in a platform definition, and then access those same
keys on the elaborated platform, you can now set "rustcTarget" in
your crossSystem, and then access "stdenv.hostPlatform.rustcTarget"
in your code.
"rustcTarget", "rustcTargetSpec", "cargoShortTarget", and
"cargoEnvVarTarget" have the "rustc" and "cargo" prefixes because
these are not exposed to code by the compiler, and are not
standardized. The arch/os/etc. variables are all named to match the
forms in the Rust target spec JSON.
The new rust.target-family only takes a list, since we don't need to
worry about backwards compatibility when that name is used.
The old APIs are all still functional with no warning for now, so that
it's possible for external code to use a single API on both 23.05 and
23.11. We can introduce the warnings once 23.05 is EOL, and make them
hard errors when 23.11 is EOL.
Just minor changes like:
- Always using "X is a Y, but it should be Z"
- "X is a path that does not exist" rather than "X does not exist"
- Always using multi-line strings for errors
- Always quoting string-like values and not quoting path-like values
- But do quote filesystem roots. Even though they're paths, they might
be very small, good to have quotes to know the start/end
- Capitalise the first word
- Distinguish root vs filesystem root more
We have several cross-compilation bugs that show up if
hostPlatform!=buildPlatform yet
hostPlatform.config==buildPlatform.config.
These bugs have appeared and disappeared as we've fiddled with the
definition of equality for platform objects. This commit adds a
clear-cut case where they are *not* equal and never will be, so we
can test it.
This does decrease performance unfortunately
Benchmarking expression toSource { root = ./.; fileset = ./.; }
Mean CPU time 0.103747 (σ = 0.012415) for 10 runs is 97.32181384964636% (σ = 16.34179537413021%) of the old value 0.106602 (σ = 0.0125571)
Statistic .envs.elements (205920) is 105.5842% (+10891) of the old value 195029
Statistic .gc.totalBytes (20247696) is 101.7495% (+348160) of the old value 19899536
Statistic .nrThunks (134824) is 108.7878% (+10891) of the old value 123933
Statistic .symbols.number (996) is 100.1005% (+1) of the old value 995
Statistic .values.number (275238) is 104.1199% (+10891) of the old value 264347
Currently just throws the Nix error because unknown file types are not
supported by the Nix store, but nothing catches this error earlier (yet,
see next commit)
gnu-config will ignore the portion of a triple matching the regex
`e?abi.*$` when determining the validity of a triple. In other
words, `i386-linuxabichickenlips` is a valid triple.
This commit updates our parsing routines to match gnu-config.
I was recently surprised to discover that it is in fact possible to
shoehorn ABI flavors into nix doubles in a way which preserves their
property of being a (non-canonical) subset of the valid gnu-config
triples. This commit is required in order to exploit that discovery
to add automatic detection of ILP32 platforms (64-bit void*, 32-bit
int, like the Apple Watch and MIPS n32) to Nix.
Add the "Type:" blocks.
Move the examples below the descriptions whenever possibles
Add "Example:" tags before the examples moved below the descriptions.
For transforming back between lists and attrsets, it makes sense to have
a quasi-inverse of `builtins.listToAttrs` available as a library
function.
Co-authored-by: Silvan Mosberger <github@infinisil.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
