Using a simple algorithm, convert the references to a path in to a
sorted list of dependent paths based on how often they're referenced
and how deep in the tree they live. Equally-"popular" paths are then
sorted by name.
The existing writeReferencesToFile prints the paths in a simple
ascii-based sorting of the paths.
Sorting the paths by graph improves the chances that the difference
between two builds appear near the end of the list, instead of near
the beginning. This makes a difference for Nix builds which export a
closure for another program to consume, if that program implements its
own level of binary diffing.
For an example, Docker Images. If each store path is a separate layer
then Docker Images can be very efficiently transfered between systems,
and we get very good cache reuse between images built with the same
version of Nixpkgs. However, since Docker only reliably supports a
small number of layers (42) it is important to pick the individual
layers carefully. By storing very popular store paths in the first 40
layers, we improve the chances that the next Docker image will share
many of those layers.*
Given the dependency tree:
A - B - C - D -\
\ \ \ \
\ \ \ \
\ \ - E ---- F
\- G
Nodes which have multiple references are duplicated:
A - B - C - D - F
\ \ \
\ \ \- E - F
\ \
\ \- E - F
\
\- G
Each leaf node is now replaced by a counter defaulted to 1:
A - B - C - D - (F:1)
\ \ \
\ \ \- E - (F:1)
\ \
\ \- E - (F:1)
\
\- (G:1)
Then each leaf counter is merged with its parent node, replacing the
parent node with a counter of 1, and each existing counter being
incremented by 1. That is to say `- D - (F:1)` becomes `- (D:1, F:2)`:
A - B - C - (D:1, F:2)
\ \ \
\ \ \- (E:1, F:2)
\ \
\ \- (E:1, F:2)
\
\- (G:1)
Then each leaf counter is merged with its parent node again, merging
any counters, then incrementing each:
A - B - (C:1, D:2, E:2, F:5)
\ \
\ \- (E:1, F:2)
\
\- (G:1)
And again:
A - (B:1, C:2, D:3, E:4, F:8)
\
\- (G:1)
And again:
(A:1, B:2, C:3, D:4, E:5, F:9, G:2)
and then paths have the following "popularity":
A 1
B 2
C 3
D 4
E 5
F 9
G 2
and the popularity contest would result in the paths being printed as:
F
E
D
C
B
G
A
* Note: People who have used a Dockerfile before assume Docker's
Layers are inherently ordered. However, this is not true -- Docker
layers are content-addressable and are not explicitly layered until
they are composed in to an Image.
/cc @Ericson2314
PR was https://github.com/NixOS/nixpkgs/pull/46857
This line broke MacOS cross compilation. paxctl cannot be built on
macOS. Maybe it can be fixed, but no reason to break things
unnecessarily.
Regardless, you definitely need to be more careful about backporting.
I think it’s fine to move fast and break things on master but
with release-18.09 we should be more careful. Something like more
automated testing for cross compilation would also be
helpful (hopefully even making it block).
(cherry picked from commit f9c4075873)
/cc @Ericson2314
This line broke MacOS cross compilation. paxctl cannot be built on
macOS. Maybe it can be fixed, but no reason to break things
unnecessarily.
Regardless, you definitely need to be more careful about backporting.
I think it’s fine to move fast and break things on master but
with release-18.09 we should be more careful. Something like more
automated testing for cross compilation would also be
helpful (hopefully even making it block).
Changes the evaluation order in that it evaluates assertions before
warnings, so that eg. the following would work:
{ config, lib, ... }:
{
options.foo = lib.mkOption {
type = lib.types.bool;
default = true;
description = "...";
};
options.bar = lib.mkOption {
type = lib.types.bool;
default = false;
description = "...";
};
config = lib.mkMerge [
(lib.mkIf config.bar {
system.build.bar = "foobar";
})
(lib.mkIf config.foo {
assertions = lib.singleton {
assertion = config.bar;
message = "Bar needs to be enabled";
};
systemd.services.foo = {
description = "Foo";
serviceConfig.ExecStart = config.system.build.bar;
};
})
];
}
This is because the systemd module includes definitions for warnings
that would trigger evaluation of the config.system.build.bar definition.
The original pull request references a breakage due to the following:
{
services.nixosManual.enable = false;
services.nixosManual.showManual = true;
}
However, changing the eval order between asserts and warnings clearly is
a corner case here and it only happens because of the aforementioned
usage of warnings in the systemd module and needs more discussion.
Nevertheless, this is still useful because it lowers the evaluation time
whenever an assertion is hit, which is a hard failure anyway.
Introduced by 0f3b89bbed.
If services.nixosManual.showManual is enabled and
documentation.nixos.enable is not, there is no
config.system.build.manual available, so evaluation fails. For example
this is the case for the installer tests.
There is however an assertion which should catch exactly this, but it
isn't thrown because the usage of config.system.build.manual is
evaluated earlier than the assertions.
So I split the assertion off into a separate mkIf to make sure it is
shown appropriately and also fixed the installation-device profile to
enable documentation.nixos.
Signed-off-by: aszlig <aszlig@nix.build>
Cc: @oxij
Thanks to @symphorien for this work, which apart from the update itself
includes a few more fixes and cleanups.
I've tested building and running the upgraded Paperwork and while I
haven't done extensive testing on every little feature it seems to work
so far.
The changes also include an addition to fetchFromGitLab, which allows to
specify a group.
Merges: #46487
