Much like the previous commit that adds dependencies &
optional-dependencies this aligns PEP-517 build systems with how they
are defined in PEP-518/PEP-621.
The naming `build-system` (singular) is aligned with upstream Python standards.
Since https://github.com/NixOS/nixpkgs/pull/161835 we've had the
concept of `passthru.optional-dependencies` for Python optional deps.
Having to explicitly put optional-dependencies in the passthru attrset
is a bit strange API-wise, even though it semantically makes sense.
This change unifies the handling of non-optional & optional Python
dependencies using the names established from PEP-621 (standardized pyproject.toml project metadata).
This commit adds support for swapping out the compression algorithm
used in all major docker-tools commands that generate images. The
default algorithm remains unchanged (gzip).
There is an arbitrary mapping being done right now between
nixpkgs lua infrastructre and luarocks config schema.
This is confusing if you use lua so let's make it possible to use the
lua names in the nixpkgs, thanks to the lib.generators.toLua convertor.
The only nixpkgs thing to remember should be to put the config into `luarocksConfig`
`buildLuarocksPackage.extraVariables` should become `buildLuarocksPackage.luarocksConfig.variables`
I believe it would be helpful to better explain how to use
`nuget-to-nix` for those who aren't familar with the .NET ecosystem as I
was personally stumped on how to use it.
This allows for adding new, conditionally set, derivation attributes
to an existing derivation without changing any output paths in the
case where the condition is not met.
`lib.recursiveUpdate` indiscriminately recurses into all attribute sets,
also into derivations. This means that it is possible that evaluating a
derivation in the final haskell package set can cause something in
`prev.haskell` to be forced by `recursiveUpdate`, potentially causing an
evaluation error that should not happen.
It can be fixed using a well-crafted predicate for
`lib.recursiveUpdateUntil`, but most robust is just explicitly writing
out the desired merging manually.
Rename channel `nixpkgs` to `nixpkgs-unstable`. Based on the
[repo branches](https://github.com/NixOS/nixpkgs/branches)
and [channel status](https://status.nixos.org) I don't believe there is
a `nixpkgs` channel. This confused me as a Nixpkgs beginner.
Rename to `nixpkgs-unstable`, which does exist.
Refer to "nixpkgs repository" consitently. Make the capitalization and
"code quoting" consistent when referring to the repository itself.
doc: add figure definition to bespoke syntax reference
doc: add example definition to bespoke syntax reference
doc: add footnote definition to beskpoke syntax reference
The usage of footnotes in the manuals is not the one documented
in markdown-it-py: https://python-markdown.github.io/extensions/footnotes/
doc: add inline comment definition to beskpoke syntax reference
doc: add typographic replacements to beskpoke syntax reference
doc: Fix rendering of bespoke syntax reference
doc: remove references to DocBook in the NixOS manual
doc: add entry on lack of HTML support
doc: Minor improvement
doc: update typographic replacements entry in beskpoke syntax reference
doc: add link reference definitions to beskpoke syntax reference
doc: fix footnote definition in beskpoke syntax reference
doc: Minor improvements from code review
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
* Note on how to use shadowSetup with buildLayeredImage
* Update doc/build-helpers/images/dockertools.section.md
---------
Co-authored-by: Silvan Mosberger <github@infinisil.com>
* doc: add types to template
* mention types explicitly
* use separator that allows for more items in the type declaration
Co-authored-by: Daniel Sidhion <DanielSidhion@users.noreply.github.com>
---------
Co-authored-by: Daniel Sidhion <DanielSidhion@users.noreply.github.com>
* doc: improve documentation for trivial text writing functions
Co-authored-by: Brian Merchant <bzm3r@proton.me>
Co-authored-by: Robert Hensing <robert@roberthensing.nl>
Co-authored-by: Alexander Groleau <alex@proof.construction>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Per RFC 9110, [section 8.8.1][1], different representations of the same
resource should have different Etags:
> A strong validator is unique across all versions of all
> representations associated with a particular resource over time.
> However, there is no implication of uniqueness across representations
> of different resources (i.e., the same strong validator might be in
> use for representations of multiple resources at the same time and
> does not imply that those representations are equivalent)
When serving statically compressed files (ie, when there is an existing
corresponding .gz/.br/etc. file on disk), Nginx sends the Etag marked
as strong. These tags should be different for each compressed format
(as shown in an explicit example in section [8.8.3.3][2] of the RFC).
Upstream Etags are composed of the file modification timestamp and
content length, and the latter generally changes between these
representations.
Previous implementation of Nix-specific Etags for things served from
store used the store hash. This is fine to share between different
files, but it becomes a problem for statically compressed versions of
the same file, as it means Nginx was serving different representations
of the same resource with the same Etag, marked as strong.
This patch addresses this by imitating the upstream Nginx behavior, and
appending the value of content length to the store hash.
[1]: https://www.rfc-editor.org/rfc/rfc9110.html#name-validator-fields
[2]:
https://www.rfc-editor.org/rfc/rfc9110.html#name-example-entity-tags-varying
lualatex assumes a writeable font cache relative to `$HOME`, for nix this has two implications.
First, the cache might diverge from the nix store if users use LuaLaTeX.
Second, `$HOME` needs to be set to a writable path in derivations.
All other functions are in the form of `*{c,C}heckpointBuild*`, so we
deprecate the `mkCheckpointedBuild` function in favor of `mkCheckpointBuild`.
Also address some inconsistencies in the docs: some `buildOutput` should
actually be `incrementalBuildArtifacts`.
https://www.pcre.org/
The `pcre` library is "now at end of life, and is no longer being
maintained," according to the upstream maintainers. Accordingly, we
should replace uses of `pcre` with its actively maintained successor,
`pcre2`, wherever possible .
It is fine to use `with` on the inputs, since that increases the
overall readability of the package.
Removes `wheel` from `nativeBuildInputs`, since it is a result of
cargo culting from an earlier setuptools example, that was wrong, and
it is not required, since it is provided by setuptools itself.
The python-updates branch was formerly called python-unstable, but the
new branch name was never mentioned in the docs. This commit changes the
branch name in the docs to python-updates.
We get a dependency list with pub2nix now. We can no longer easily distinguish between development dependency dependencies and regular dependency dependencies, but we weren't doing this anyway.
The mkBinaryCache section now follows the convention of using one
line per sentence, as well as using admonitions for notes and
examples.
The example syntax was updated to define a fully-working nix package.
The following changes are made:
- Document how `wrapType2` and `wrapType1` are the same thing.
- Expand on how `wrapType2` works and additional arguments it uses.
- Document `extract` and show how it's used in combination with
`wrapType2`.
- Provide full working examples using the new admonition syntax.
Fixes some mistakes regarding the references to cargoHash in the codes referenced.
Fixes a typo for cargoSha256.
States that cargoHash should be preferred.
Without this, it's impossible to override the lockFile as the default
overrideAttrs is applied after the composition in buildNimPackage has
read the lock file and generated the nim flags from it.
For the time being, we're moving towards https://nix.dev/ containing
all tutorials and guides. The Nixpkgs manual is reinforced to be a
_reference_ manual. While it's not just reference for now, that's what
the docs team is working towards.
This commits rewrites the Nixpkgs manual introduction to reflect that
and point to some more useful links. The contribution docs are updated
similarly so it's not missed.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
GitHub supported special markdown syntax for emphasising blocks for some
time. This was however a beta feature, and still is, so it's subject to
changes.
Recently such a change happened: The syntax is different now.
See https://github.com/orgs/community/discussions/16925 for more
information
We keep running into situations where we can't get the right
combination of rustc flags through build systems into rustc.
RUSTFLAGS is the only variable supported across build systems, but if
RUSTFLAGS is set, Cargo will ignore all other ways of specifying rustc
flags, including the target-specific ones, which we need to make
dynamic musl builds work. (This is why pkgsCross.musl64.crosvm is
currently broken — it works if you unset separateDebugInfo, which
causes RUSTFLAGS not to be set.)
So, we need to do the same thing we do for C and C++ compilers, and
add a compiler wrapper so we can inject the flags we need, regardless
of the build system.
Currently the wrapper only supports a single mechanism for injecting
flags — the NIX_RUSTFLAGS environment variable. As time goes on,
we'll probably want to add additional features, like target-specific
environment variables.
The Nixpkgs documentation on the linux kernel builders focused on
using and extending kernels that were already packaged, but never
mentioned that it's possible to also build a kernel almost "from
scratch".
The NixOS documentation went a bit deeper on manual linux kernel
configs, but that information wasn't particularly NixOS-specific.
This commit consolidates the information related to building the
kernel on Nixpkgs's documentation, while keeping any additional
NixOS-specific information on NixOS's documentation.
An additional README.md was created for contributor-facing
documentation.
Update instructions for manual build phases execution via `nix-shell` to
cover all phases. There is no easy way of getting those commands, so it
makes a sense to have them all properly documented.
While the word 'simply' is usually added to encourage readers, it often has the
opposite effect and may even appear condescending, especially when the reader
runs into trouble trying to apply the suggestions from the documentation. It is
almost always an improvement to simply drop the word from the sentence.
(there are more possible improvements like this, we can apply those in separate
PRs)
Replaced with python3Packages except for the following to avoid misuse of pythonPackages
1. which we need for the explanation
2. which are replaced by python3Packages as variables
This seems like a pretty easy-to-fix oversight, and it documents the
getExe function, which I never knew about until one day I saw a PR
using it. Let's include it in the manual.
* luarocks-packages-updater: init
Goal is to make it possible to maintain out-of-tree luarocks packages
without needing to clone nixpkgs.
maintainers/scripts/update-luarocks-packages gets renamed to
pkgs/development/lua-modules/updater/updater.py
Once merged you can run for instance
nix run nixpkgs#luarocks-packages-updater -- -i contrib/luarocks-packages.csv -o contrib/generated-packages.nix
I also set the parallelism (--proc) to 1 by default else luarocks fails
because of https://github.com/luarocks/luarocks/issues/1540
* Update maintainers/scripts/pluginupdate.py
Co-authored-by: Marc Jakobi <mrcjkb89@outlook.com>
---------
Co-authored-by: Marc Jakobi <mrcjkb89@outlook.com>
* doc/stdenv: rewrite manual build procedure to be closer to an auto-build
This is based on
<https://jade.fyi/blog/building-nix-derivations-manually/> plus some
more original research.
The previous version of this section did not work for your choice of
simple Haskell package, e.g. haskellPackages.hscolour, due to things
like `compileBuildDriverPhase` and other custom phases that it
does not address at all.
It seems more correct to use genericBuild in development to harmonize it
with what is actually done.
I feel a little bit like I am committing a sin by suggesting using the
experimental CLI in the manual (afaict I am the first to do this), but I
have given the old version of the command, and there are justifiable
reasons to do it:
* The noted limitations with env-vars are fixed. The one with the
non-empty temp directory was one I ran into myself and oh boy was that
not fun to debug.
* Additionally the outputs are set *before* sourcing `setup.sh`: there
is an issue with nix-shell where the original version of `$out` winds
up in `NIX_LDFLAGS` due to _addRpathPrefix, which means that resulting
executables may not run properly.
It is sad that `nix develop` propagates a wrong value of `SHELL` to
builders, though. It is equally sad that `nix-shell` is essentially
abandoned upstream, with undocumented and not insignificant differences
from `nix develop`.
For the exact script differences:
17e6b85d05/src/nix-build/nix-build.cc (L516-L551)db026103b1/src/nix/get-env.sh
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Projects building with meson are currently installTargets. Map these
to install tags, which are roughly equivalent. This allows projects to
selectively install components.
I chose not to do nixos.{config, options} because that would make it
look too much like a configuration object, which it is not.
A configuration object I would define as for example the result of
calling NixOS, an attrset with `_type = "configuration";`.
Recreating a configuration object without evalModules is quite
feasible but not guaranteed to be correct, and not maintainable.
Meson now comes with its own set of commands for building, testing,
installing etc., that by default wrap around Ninja. The reason to
prefer using the Meson commands is that they take additional
options (e.g. setting custom timeouts for tests — my motivation for
this change).
Here, I've modified the Meson setup hook so that Meson's test and
install commands will be used instead of Ninja's when Meson's
configurePhase is used. This restriction is to avoid starting to run
Meson directly when dealing with custom build systems that wrap around
Meson, like QEMU's. We don't use meson's compile command, as it just
runs ninja, and that's handled fine by the existing Ninja setup hook.
Naturally the Meson commands don't support entirely the same set of
options that the Ninja ones did, but I checked through Nixpkgs to find
any packages using Meson that used any options that wouldn't be picked
up by this new system. I only found one, and it was just setting
checkTarget = "test", which is the default value for Ninja and has no
Meson equivalent (because we directly tell Meson to run the tests
rather than going through a generic job system like Ninja).
Link: https://github.com/NixOS/nixpkgs/issues/113829
Co-authored-by: Jan Tojnar <jtojnar@gmail.com>
This commit adds minimal documentation of the supported platforms.
More exhaustive documentation would require producing a list of
platforms for each of the 7 tiers. This was attempted in #245368, but it
quickly became clear that that would be a long-term effort.
In the meantime, this commit adds the most important information to the
manual.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
The nixpkgs documentation mentions how to update out of tree plugins but
one problem is that it requires a nixpkgs clone.
This makes it more convenient.
I've had the need to generate vim plugins and lua overlays for other
projects unrelated to nix and this will make updates easier (aka just
run `nix run nixpkgs#vimPluginsUpdater -- --proc=1` or with the legacy commands:
`nix-shell -p vimPluginsUpdater --run vim-plugins-updater`.
I added an optional "nixpkgs" argument to command line parser, which is the path
towards a nixpkgs checkout. By default the current folder.
update-luarocks-packages: format with black
* nixpkgs manual: add an alternative example in stdenv-separateDebugInfo
This change gets rid of the indirect reference to `nix-env -i` usage
and shows how to achieve the same goal with a shell expression.
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
The text was originally added [0] following an apparently incomplete
research on how everything plays together. In fact, Nix propagates
`outputs` to the corresponding nested derivations, and there is some
messy behavior in Nixpkgs that only seems to propagate
`meta.outputsToInstall` in `buildEnv`[1].
This change moves the hints on how to use NixOS specifics to NixOS
module documentation (which is hopefully easier to find through
search.nixos.org), describes the default behavior in Nixpkgs (updating
a the link to the source), and removes the confusing mention of
`nix-env`.
the last of them should not be there to begin with. we don't want
beginners to use `nix-env`, as this is known to run them into trouble
eventually.
[0]: https://github.com/NixOS/nixpkgs/pull/76794
[1]: 1774d07242/pkgs/build-support/buildenv/default.nix (L66)
Allows splicing to work correctly
from issue 211340
>> 3: Discourage using package sets like python3.pkgs "directly"
> I think about 8 years ago I introduced this and it has been a clear mistake. Since then this pattern was copied throughout. We should indeed go to pythonPackages and also move the helpers such as buildEnv and withPackages into the package set, like haskell already had before.
- FRidh
* Updates meta.chapter.md with a reference link to the usage of the package description field instead of referring to nix-env
---------
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Clarify that the monochrome font is not included, per #221181.
The new name is also coherent with the name of the font,
according to `fontconfig`: Noto Color Emoji.
This change which involves creating multiple outputs for CUDA
redistributable packages.
We use a script to find out, ahead of time, the outputs each redist
package provides. From that, we are able to create multiple outputs for
supported redist packages, allowing users to specify exactly which
components they require.
Beyond the script which finds outputs ahead of time, there is some custom
code involved in making this happen. For example, the way Nixpkgs
typically handles multiple outputs involves making `dev` the default
output when available, and adding `out` to `dev`'s
`propagatedBuildInputs`.
Instead, we make each output independent of the others. If a user wants
only to include the headers found in a redist package, they can do so by
choosing the `dev` output. If they want to include dynamic libraries,
they can do so by specifying the `lib` output, or `static` for static
libraries.
To avoid breakages, we continue to provide the `out` output, which
becomes the union of all other outputs, effectively making the split
outputs opt-in.
1. Clarify what is the reason for importing and to where it saves
2. Clarify that packages.sqlite is a temporary file
3. Link to section about native dependencies from first mention of ql.nix
Add this hook to checkPhase to allow for running MPI application in
the sandbox. It detects the MPI implementations and sets the respective
environment variables.
