Allow customisation of the algorithm used to convert nix references
graph (created from docker image contents) to docker layers.
A collection of building blocks (python functions) is provided, which
use can assembled into a processing pipeline by specifying a list of
operations (and their initial arguments) via a nix list.
Nix references graph if first converted into a python igraph.Graph
object (with each vertex representing a nix path), which is then fed
into the user defined pipeline. Each stage in the pipeline represents a
function call, with initial arguments specified by the user in nix, and
the last argument being the result of the previous stage in the pipeline
(or the initial Graph object). Each step of the pipeline is expected to
produce a data structure consisting of arbitrarily nested lists/dicts
with Graph objects (representing docker layers) at it's leafs. The
result of the last stage in the pipeline is recursively flattened (with
each dict converted into list of values), until a flat list of Graphs
remains. This is then output as a json array of arrays (each Graph
converted into an array of paths).
This functionality is made available via new `layeringPipeline` argument
to the `streamLayeredImage`/`buildLayeredImage` functions. The default
value of the argument has been chosen to to preserve current layering
behaviour.
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
We are migrating packages that meet below requirements:
1. using `callPackage`
2. called path is a directory
3. overriding set is empty (`{ }`)
4. not containing path expressions other than relative path (to
makenixpkgs-vet happy)
5. not referenced by nix files outside of the directory, other
than`pkgs/top-level/all-packages.nix`
6. not referencing nix files outside of the directory
7. not referencing `default.nix` (since it's changed to `package.nix`)
8. `outPath` doesn't change after migration
The tool is here: https://github.com/Aleksanaa/by-name-migrate.
In preparation for the deprecation of `stdenv.isX`.
These shorthands are not conducive to cross-compilation because they
hide the platforms.
Darwin might get cross-compilation for which the continued usage of `stdenv.isDarwin` will get in the way
One example of why this is bad and especially affects compiler packages
https://www.github.com/NixOS/nixpkgs/pull/343059
There are too many files to go through manually but a treewide should
get users thinking when they see a `hostPlatform.isX` in a place where it
doesn't make sense.
```
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv.is" "stdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv'.is" "stdenv'.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "clangStdenv.is" "clangStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "gccStdenv.is" "gccStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenvNoCC.is" "stdenvNoCC.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "inherit (stdenv) is" "inherit (stdenv.hostPlatform) is"
fd --type f "\.nix" | xargs sd --fixed-strings "buildStdenv.is" "buildStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "effectiveStdenv.is" "effectiveStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "originalStdenv.is" "originalStdenv.hostPlatform.is"
```
