* doc: add details on `mapAttrsRecursive[Cond]`
from first reading it wasn't clear that `f` also takes the current
attribute path. also the value f receives is tricky due to how the
condition is evaluated.
Co-authored-by: Daniel Sidhion <DanielSidhion@users.noreply.github.com>
makeOverridable is very careful to ensure the arguments to the
overridden function are the same as the input function. As a result,
the arguments of hello.override are exactly the same as the original
arguments of the hello function that produced the derivation.
However, callPackagesWith calls makeOverridable with a lambda that
does not propagate the arguments. The override function for a package
instantiated with callPackagesWith will not have the original
arguments.
For example:
nix-repl> lib.functionArgs hello.override
{ callPackage = false; fetchurl = false; hello = false; lib = false; nixos = false; stdenv = false; testers = false; }
nix-repl> lib.functionArgs openssl.override
{ }
By copying the arguments onto the inner lambda before passing it to
makeOverridable, we can make callPackage and callPackages behave the
same.
nix-repl> lib.functionArgs openssl.override
{ buildPackages = false; coreutils = false; cryptodev = false; enableSSL2 = true; enableSSL3 = true; fetchurl = false; lib = false; perl = false; removeReferencesTo = false; static = true; stdenv = false; withCryptodev = true; withPerl = true; }
This was found when trying to run the fileset tests on Darwin
(https://github.com/NixOS/nix/pull/9546#issuecomment-1967409445), which mysteriously fail on Darwin:
test case at lib/fileset/tests.sh:342 failed: toSource { root = "/nix/store/foobar"; fileset = ./.; } should have errored with this regex pattern:
lib.fileset.toSource: `root` \(/nix/store/foobar\) is a string-like value, but it should be a path instead.
\s*Paths in strings are not supported by `lib.fileset`, use `lib.sources` or derivations instead.
but this was the actual error:
error: lib.fileset.toSource: `root` (/nix/store/foobar) is a string-like value, but it should be a path instead.
Paths in strings are not supported by `lib.fileset`, use `lib.sources` or derivations instead.
After dissecting this, I find out that apparently \s works on Linux, but not on Darwin for some reason!
From the bash source code, it looks like <regex.h> with `REG_EXTENDED` is used for all platforms the same,
so there's nothing odd there.
It's almost impossible to know where <regex.h> comes from,
but it looks to be a POSIX thing.
So after digging through the almost impossible to find POSIX specifications
(https://pubs.opengroup.org/onlinepubs/007908799/xbd/re.html#tag_007_003_005),
I can indeed confirm that there's no mention of \s or the like!
_However_, there is a mention of `[[:blank:]]`, so we'll use that instead.
The previous code was optimized for the old uniq behavior, which did not
call merge. That's changed, so the legacy path is not a hot path anymore,
and is not worth any tech debt.
The deprecation warnings in lib were wildly inconsistent. Different
formulations were used in different places for the same meaning. Some warnings
used builtins.trace instead of lib.warn, which prevents silencing; one even
only had a comment instead. Make everything more uniform.
This allows for adding new, conditionally set, derivation attributes
to an existing derivation without changing any output paths in the
case where the condition is not met.
The only reason shallow clones are not the default in
`builtins.fetchGit` is that `revCount` can't be provided when cloning a
shallow repository. However, `revCount` isn't used or exposed by
`lib.fileset`. Hence, allowing cloning shallow repositories makes
`gitTracked` more general without any drawbacks.
Co-authored-by: Silvan Mosberger <github@infinisil.com>
It is useful that all (or almost all) .nix files in nixpkgs at least
parse since it allows for checking syntax in the repository
programmatically without evaluating anything.
This doesn't change uniq. Why not?
- In NixOS it seems that uniq is only used with
simple types that are fully checked by t.check.
- It exists for much longer and is used more widely.
- I believe we should deprecate it, because unique was
already better.
- unique can be a proving ground.
According to the WebAssembly design doc, wasm32 is an ILP32 ABI like
x32, mips64n32, and aarch64_ilp32 (Apple Watch). This commits adds
it to the predicate.
1319968ca5/CAndC%2B%2B.md (L16)
Most of the time when we do a patchelf conditional on
hostPlatform.isLinux, what we really mean is hostPlatform.isElf.
Now that we are starting to support BSDs, this is becoming more important.
Type: either ints.positive (enum ["auto"])
Before: positive integer, meaning >0 or value "auto" (singular enum)
After: positive integer, meaning >0, or value "auto" (singular enum)
Currently, the `lib/tests/modules.sh` test checks the output of
`nix-instantiate --eval` without `--json`, which outputs an unspecified
human-readable format.
This patch modifies `modules.sh` to use the `--json` output instead, to
be robust against future changes to `nix-instantiate` output.
This commit temporarily adds pkgs/test/release to the
lib/tests/release.nix test suite, because ofborg already knows about
that entry point.
We should move the list of test entry points out of ofborg and into
a central place in nixpkgs:
https://github.com/NixOS/nixpkgs/issues/272591
Once we do that we won't need to have this ugly kludge in an
inappropriate place.
* Improves the comments of `lib/flake-version-info.nix` and drops the
`__`-prefix from the filename.
* `lib'` -> `lib0` in `nixpkgs/lib`.
* Drop the declaration of `trivial.version` in the overlay because this
declaration already uses the final expressions of `versionSuffix` and
`release` now.
* No need to fall back to `self.lastModified` anymore, this was a
workaround for pre2.4 Nix.
Co-authored-by: Robert Hensing <robert@roberthensing.nl>
Co-authored-by: Silvan Mosberger <contact@infinisil.com>
That way each expression uses the final version of other lib.trivial
declarations.
For instance, when replacing `versionSuffix` with the string `"fnord"`
in a lib overlay, `trivial.version` uses `"fnord"` as suffix now rather
than `pre-git`.
A more efficient sort in some cases, and often convenient.
This exposes `lib.lists.sortOn` immediately on `lib`, because it is
a sibling of `sort`, which is already present there.
Omitting it would lead to more confusion, and worse outcomes.
There's no confusion about the types `sort` or `sortOn` operate on.
Haskell agrees about the type for `sortOn`, and it is in its `base`.
This type is necessary to have correct merging behavior for
`allowUnfreePredicate` and `allowInsecurePredicate`
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
For the time being, we're moving towards https://nix.dev/ containing
all tutorials and guides. The Nixpkgs manual is reinforced to be a
_reference_ manual. While it's not just reference for now, that's what
the docs team is working towards.
This commits rewrites the Nixpkgs manual introduction to reflect that
and point to some more useful links. The contribution docs are updated
similarly so it's not missed.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
An important idea around the rust stuff in lib.systems is that it's
elaborated — this means that it should idempotently add to the values
passed in, if any. But we missed that the names used for the
parameter and the elaborated value for "rustcTarget"/"config" didn't
line up. The intention was to use "rustcTarget" everywhere in the new
interface, as a more descriptive name than "config".
This fixes setting the system in NixOS configuration, which results in
an already elaborated system being elaborated again. Before, this
wouldn't produce the correct result:
% nix-instantiate --eval -A stdenv.hostPlatform.rust.rustcTarget --system armv7l-linux
"armv7-unknown-linux-gnueabihf"
% NIX_PATH= nix-instantiate --eval -E '(import nixos/lib/eval-config.nix { system = "armv7l-linux"; modules = []; }).pkgs.stdenv.hostPlatform.rust.rustcTarget'
"arm-unknown-linux-gnueabihf"
Fixes: e3e57b8f18 ("lib.systems: elaborate Rust metadata")
Fixes: https://github.com/NixOS/nixpkgs/issues/271000
Usually, attributes passed explicitly to elaborate take precedence
over the elaborated ones, but since we also elaborate the nested
"rust" attrset, we need to push that one level down, so the rest of
"rust" is still filled in if you just pass
{ rust = { config = ... } }.
I've had to drop the assertion that checked that at most one of "rust"
and "rustc" was part of the un-elaborated system, because doing this
broke passing an elaborated system in, which should be idempotent.
For the same reason, I've also had to make it possible for
rust.rustcTargetSpec to be passed in. Otherwise, on the second call,
since platform was filled in by the first, the custom target file
would be constructed. The only other way to avoid this would be to
compare the platform attrs to all built in Rust targets to check it
wasn't one of those, and that isn't feasible.
Fixes: e3e57b8f18 ("lib.systems: elaborate Rust metadata")
- Make fromSource's missing file error message more consistent with others,
and add a test for it
- Indent some function arguments
- Fix an internal type
The aws-sdk-cpp tests are flaky.
Since pull requests to staging cause nix to be rebuilt, this means
that staging PRs end up getting false CI failures due to whatever is
flaky in the AWS SDK tests. Since none of our CI needs to (or
should be able to) contact AWS S3, let's just omit it all. Bonus:
the tests build way faster.
While this change is backwards-incompatible, I think it's okay because:
- The `fileFilter` function is not yet in a stable NixOS release, it was only merged about [a month ago](https://github.com/NixOS/nixpkgs/pull/257356).
- All public uses of the function on GitHub only pass a path
- Any `fileFilter pred fileset` can also be expressed as `intersection fileset (fileFilter pred path)` without loss of functionality.
- This is furthermore pointed out in the new error message when a file set is passed
Along the lines of `assertOneOf`, but expects a list of values to be
compared. This gives a good error message and is useful for lists of
values, like `supportedGhcVersions` in the arguments of
`haskell-language-server`.
We need this stuff to be available in lib so make-derivation.nix can
access it to construct the Meson cross file.
This has a couple of other advantages:
- It makes Rust less special. Now figuring out what Rust calls a
platform is the same as figuring out what Linux or QEMU call it.
- We can unify the schema used to define Rust targets, and the schema
used to access those values later. Just like you can set "config"
or "system" in a platform definition, and then access those same
keys on the elaborated platform, you can now set "rustcTarget" in
your crossSystem, and then access "stdenv.hostPlatform.rustcTarget"
in your code.
"rustcTarget", "rustcTargetSpec", "cargoShortTarget", and
"cargoEnvVarTarget" have the "rustc" and "cargo" prefixes because
these are not exposed to code by the compiler, and are not
standardized. The arch/os/etc. variables are all named to match the
forms in the Rust target spec JSON.
The new rust.target-family only takes a list, since we don't need to
worry about backwards compatibility when that name is used.
The old APIs are all still functional with no warning for now, so that
it's possible for external code to use a single API on both 23.05 and
23.11. We can introduce the warnings once 23.05 is EOL, and make them
hard errors when 23.11 is EOL.
Just minor changes like:
- Always using "X is a Y, but it should be Z"
- "X is a path that does not exist" rather than "X does not exist"
- Always using multi-line strings for errors
- Always quoting string-like values and not quoting path-like values
- But do quote filesystem roots. Even though they're paths, they might
be very small, good to have quotes to know the start/end
- Capitalise the first word
- Distinguish root vs filesystem root more
We have several cross-compilation bugs that show up if
hostPlatform!=buildPlatform yet
hostPlatform.config==buildPlatform.config.
These bugs have appeared and disappeared as we've fiddled with the
definition of equality for platform objects. This commit adds a
clear-cut case where they are *not* equal and never will be, so we
can test it.
This does decrease performance unfortunately
Benchmarking expression toSource { root = ./.; fileset = ./.; }
Mean CPU time 0.103747 (σ = 0.012415) for 10 runs is 97.32181384964636% (σ = 16.34179537413021%) of the old value 0.106602 (σ = 0.0125571)
Statistic .envs.elements (205920) is 105.5842% (+10891) of the old value 195029
Statistic .gc.totalBytes (20247696) is 101.7495% (+348160) of the old value 19899536
Statistic .nrThunks (134824) is 108.7878% (+10891) of the old value 123933
Statistic .symbols.number (996) is 100.1005% (+1) of the old value 995
Statistic .values.number (275238) is 104.1199% (+10891) of the old value 264347
Currently just throws the Nix error because unknown file types are not
supported by the Nix store, but nothing catches this error earlier (yet,
see next commit)
gnu-config will ignore the portion of a triple matching the regex
`e?abi.*$` when determining the validity of a triple. In other
words, `i386-linuxabichickenlips` is a valid triple.
This commit updates our parsing routines to match gnu-config.
I was recently surprised to discover that it is in fact possible to
shoehorn ABI flavors into nix doubles in a way which preserves their
property of being a (non-canonical) subset of the valid gnu-config
triples. This commit is required in order to exploit that discovery
to add automatic detection of ILP32 platforms (64-bit void*, 32-bit
int, like the Apple Watch and MIPS n32) to Nix.
Add the "Type:" blocks.
Move the examples below the descriptions whenever possibles
Add "Example:" tags before the examples moved below the descriptions.
For transforming back between lists and attrsets, it makes sense to have
a quasi-inverse of `builtins.listToAttrs` available as a library
function.
Co-authored-by: Silvan Mosberger <github@infinisil.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
To maintain backwards compatibility, this can't be changed in the Nix language.
We can however ensure that the version Nixpkgs has the more intuitive behavior.
This license is used by the Fraunhofer FDK codec, which is currently
mislicensed to Apache 2.0.
Of course, this can't be corrected without having the proper license
available in lib.licenses can't it??
Thanks @trofi for noticing this has an SPDX ID.
The v2 dashboard (included since version 1.41.0) is licensed under
Netdata Cloud UI License v1.0, which is a non-free license.
Patch the source code so that the v2 is not included in the source
and the result derivation. Users that want the v2 dashboard can still
request it by overriding this derivation and pass `withCloudUi = true`.
Co-authored-by: Raito Bezarius <masterancpp@gmail.com>
Fixes: https://github.com/NixOS/nixpkgs/issues/256964
$ ./benchmark.sh HEAD
[...]
Mean CPU time 0.04006 (σ = 0.0040146) for 10 runs is 8.193619775953792% (σ = 0.9584251052704821%) of the old value 0.488917 (σ = 0.0294955)
[...]
We can now test returned paths being equal, no need to work around it
anymore by making sure paths aren't returned (which would import them
with the previous --json)
The Minimalist Gnu for Windows distribution comes with support for
the traditional msvcrt libc, as well as ucrt64 libc. The latter
being the newer universal compiler runtime. We follow the msys2
environment naming convention[1]:
| name | toolchain | arch | libc | libc++ |
|------------|-----------|---------|--------|-----------|
| mingw32 | gcc | i686 | msvcrt | libstdc++ |
| mingw64 | gcc | x86_64 | msvcrt | libstdc++ |
| ucrt64 | gcc | x86_64 | ucrt | libstdc++ |
| clang32 | llvm | i686 | ucrt | libc++ |
| clang64 | llvm | x86_64 | ucrt | libc++ |
| clangarm64 | llvm | aarch64 | ucrt | libc++ |
For now nixpkgs only supports the first three with this commit.
--
[1]: https://www.msys2.org/docs/environments/
What it does: line and column level *declaration* position information:
$ nix repl .
nix-repl> :p nixosConfigurations.micro.options.environment.systemPackages.declarationPositions
[ { column = 7; file = "/nix/store/24aj3k7fgqv3ly7qkbf98qvphasrw9nb-source/nixos/modules/config/system-path.nix"; line = 63; } ]
Use cases:
- ctags over NixOS options, as will be presented at NixCon 2023 ;)
- improving the documentation pages to go to the exact line of the
declarations.
Related work:
- https://github.com/NixOS/nixpkgs/pull/65024
This one does it for all *definitions* rather than declarations, and
it was not followed through with due to performance worries.
- https://github.com/NixOS/nixpkgs/pull/208173
The basis for this change. This change is just a rebase of that one.
I split it out to add the capability before adding users of it, in
order to simplify review. However, the ctags script in there is a
sample user of this feature.
Benchmarks: conducted by evaluating my own reasonably complex NixOS
configuration with the command:
`hyperfine -S none -w 1 -- "nix eval .#nixosConfigurations.snowflake.config.system.build.toplevel.outPath"`
```
Benchmark 1: nix eval .#nixosConfigurations.snowflake.config.system.build.toplevel.outPath
Time (mean ± σ): 8.971 s ± 0.254 s [User: 5.872 s, System: 1.388 s]
Range (min … max): 8.574 s … 9.327 s 10 runs
Benchmark 1: nix eval .#nixosConfigurations.snowflake.config.system.build.toplevel.outPath
Time (mean ± σ): 8.766 s ± 0.160 s [User: 5.873 s, System: 1.346 s]
Range (min … max): 8.496 s … 9.033 s 10 runs
```
Summary of results: it seems to be in the noise, this does not cause any
visible regression in times.
This license was introduced in the Elastic search repository by this
commit: a92a647b9f
It appears that all the the packages in nixpkgs using the Elastic License are using the v2.0.
Added basic generators.toGitINI test.
Mostly taken from 958c06303f/tests/modules/programs/git/git.nix.
The ${"\t} escape is used so that the lines aren't recognized as "Wrong
indent style".
Right now converting `makeScope` to `makeScopeWithSplicing` is not
transparent to users and requires adding a warning for `overrideScope'`
in the set itself.
Warning and `overrideScope'` were added in 2018 b9dce11712 and there should be no users left after 5 years.
Deeply-curried functions are pretty error-prone in untyped languages
like Nix. This is a particularly bad case because
`top-level/splice.nix` *also* declares a makeScopeWithSplicing, but
it takes *two fewer arguments*.
Let's add a version that uses attrset-passing form, to provide some
minimal level of sanity-checking.
This also provides defaults for keep and extra (these are often
unneeded by the user).
These statements are taken from the `lib/test/release.nix` tests,
which previously also worked on darwin.
Unblocks https://github.com/NixOS/nix/pull/8569 when backported
This reverts commit c59c6b1c57.
This was a bit too ambitious, because no warnings were previously
triggered when `string` was nested e.g. `attrsOf string`, `nullOr
string`, etc.
Support for nested type deprecation warnings was introduced in
4b54aedee5, but had to be reverted in
a36e6760e9 because it caused infinite
recursion for some users, and I couldn't remember that it was reverted.
The type has given a warning on use since [4 years ago](03392cd336b128a1639c648baf0f6c1a1271e0d2), I think it's safe to move the deprecation to the next stage: An error instead of a warning.
Before this commit, getExe assumes that if `meta.mainProgram` is unset,
it has a main program that's named after the package name.
While this is probable, it leads to a bad error when the assumption does
not hold. If the user called `getExe` themselves, they might narrow down
the location of the assumption quite easily, but if that's not the case,
they'll have to go down the rabbit hole to figure out what went wrong.
For example, a NixOS module may use `lib.getExe` on a package-typed option
which is then used in the system configuration. This then typically leads
to a failure *after* deployment, which is bad, and it's quite likely that
the user will debug the package output contents before digging through the
NixOS module, which is bad.
Furthermore the `getExe` call is rather inconspicuous as it does not
contain something like "/bin/foo", which is bad.
Also modules can be hard to read for a newbie, which is bad.
All of this can be avoided by requiring `meta.mainProgram`.
Many packages already have the attribute, and I would expect 80% of
`getExe` usages to be covered by 20% of packages, because plenty of
packages aren't used with `getExe` anyway.
Finally we could make an effort to set `mainProgram` semi-automatically
using `nix-index`.
Deeply-curried functions are pretty error-prone in untyped languages
like Nix. This is a particularly bad case because
`top-level/splice.nix` *also* declares a makeScopeWithSplicing, but
it takes *two fewer arguments*.
Let's switch to attrset-passing form, to provide some minimal level
of sanity-checking.
mkOption does not require a `type` argument and does not set the
resulting attribute if it is not given. Consequently, we need to be
prepared to merge options that have no type information.
Add support for Nvidia's Bluefield 2 plattform as a compilation
target. There exists a version with and without crypto support,
while the crypto supported version is the most common one.
Support for the non-crypto version can be easily added in the future,
if needed.
For a datasheet of the hardware, see:
https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/documents/datasheet-nvidia-bluefield-2-dpu.pdf
Signed-off-by: Markus Theil <theil.markus@gmail.com>
This is a non-trivial refactor that slightly changes the semantics
of the internal definition lists.
Whereas previously only individual list items would trigger the exception,
now the error is promoted to the whole list.
This is mostly ok, because we compute the value, it is wrong to ignore a definition.
However, we don't always compute the value. For instance `readOnly`
only needs to count definitions. That won't be possible anymore when
the error is raised for one of the items. As a consequence, an error
will be raised for the errant definition instead of the number of
definitions.
