mirror of
https://github.com/NixOS/nix.git
synced 2024-11-22 06:42:28 +00:00
extract reference documentation on remote builds (#9526)
- move all reference documentation to the `builders` configuration setting - reword documentation on machine specification, add examples - disable showing the default value, as it rendered as `@/dummy/machines`, which is wrong - highlight the examples - link to the configuration docs for distributed builds - builder -> build machine Co-authored-by: Janik H <janik@aq0.de>
This commit is contained in:
Valentin Gagarin
GitHub
parent
e37d502895
commit
fd82ba0985
@ -18,7 +18,7 @@ const redirects = {
|
||||
"chap-tuning-cores-and-jobs": "advanced-topics/cores-vs-jobs.html",
|
||||
"chap-diff-hook": "advanced-topics/diff-hook.html",
|
||||
"check-dirs-are-unregistered": "advanced-topics/diff-hook.html#check-dirs-are-unregistered",
|
||||
"chap-distributed-builds": "advanced-topics/distributed-builds.html",
|
||||
"chap-distributed-builds": "command-ref/conf-file.html#conf-builders",
|
||||
"chap-post-build-hook": "advanced-topics/post-build-hook.html",
|
||||
"chap-post-build-hook-caveats": "advanced-topics/post-build-hook.html#implementation-caveats",
|
||||
"chap-writing-nix-expressions": "language/index.html",
|
||||
|
||||
@ -36,16 +36,8 @@ error: cannot connect to 'mac'
|
||||
then you need to ensure that the `PATH` of non-interactive login shells
|
||||
contains Nix.
|
||||
|
||||
> **Warning**
|
||||
>
|
||||
> If you are building via the Nix daemon, it is the Nix daemon user account (that is, `root`) that should have SSH access to a user (not necessarily `root`) on the remote machine.
|
||||
>
|
||||
> If you can’t or don’t want to configure `root` to be able to access the remote machine, you can use a private Nix store instead by passing e.g. `--store ~/my-nix` when running a Nix command from the local machine.
|
||||
|
||||
The list of remote machines can be specified on the command line or in
|
||||
the Nix configuration file. The former is convenient for testing. For
|
||||
example, the following command allows you to build a derivation for
|
||||
`x86_64-darwin` on a Linux machine:
|
||||
The [list of remote build machines](@docroot@/command-ref/conf-file.md#conf-builders) can be specified on the command line or in the Nix configuration file.
|
||||
For example, the following command allows you to build a derivation for `x86_64-darwin` on a Linux machine:
|
||||
|
||||
```console
|
||||
$ uname
|
||||
@ -60,97 +52,20 @@ $ cat ./result
|
||||
Darwin
|
||||
```
|
||||
|
||||
It is possible to specify multiple builders separated by a semicolon or
|
||||
a newline, e.g.
|
||||
It is possible to specify multiple build machines separated by a semicolon or a newline, e.g.
|
||||
|
||||
```console
|
||||
--builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
|
||||
```
|
||||
|
||||
Each machine specification consists of the following elements, separated
|
||||
by spaces. Only the first element is required. To leave a field at its
|
||||
default, set it to `-`.
|
||||
|
||||
1. The URI of the remote store in the format
|
||||
`ssh://[username@]hostname`, e.g. `ssh://nix@mac` or `ssh://mac`.
|
||||
For backward compatibility, `ssh://` may be omitted. The hostname
|
||||
may be an alias defined in your `~/.ssh/config`.
|
||||
|
||||
2. A comma-separated list of Nix platform type identifiers, such as
|
||||
`x86_64-darwin`. It is possible for a machine to support multiple
|
||||
platform types, e.g., `i686-linux,x86_64-linux`. If omitted, this
|
||||
defaults to the local platform type.
|
||||
|
||||
3. The SSH identity file to be used to log in to the remote machine. If
|
||||
omitted, SSH will use its regular identities.
|
||||
|
||||
4. The maximum number of builds that Nix will execute in parallel on
|
||||
the machine. Typically this should be equal to the number of CPU
|
||||
cores. For instance, the machine `itchy` in the example will execute
|
||||
up to 8 builds in parallel.
|
||||
|
||||
5. The “speed factor”, indicating the relative speed of the machine. If
|
||||
there are multiple machines of the right type, Nix will prefer the
|
||||
fastest, taking load into account.
|
||||
|
||||
6. A comma-separated list of *supported features*. If a derivation has
|
||||
the `requiredSystemFeatures` attribute, then Nix will only perform
|
||||
the derivation on a machine that has the specified features. For
|
||||
instance, the attribute
|
||||
|
||||
```nix
|
||||
requiredSystemFeatures = [ "kvm" ];
|
||||
```
|
||||
|
||||
will cause the build to be performed on a machine that has the `kvm`
|
||||
feature.
|
||||
|
||||
7. A comma-separated list of *mandatory features*. A machine will only
|
||||
be used to build a derivation if all of the machine’s mandatory
|
||||
features appear in the derivation’s `requiredSystemFeatures`
|
||||
attribute.
|
||||
|
||||
8. The (base64-encoded) public host key of the remote machine. If omitted, SSH
|
||||
will use its regular known-hosts file. Specifically, the field is calculated
|
||||
via `base64 -w0 /etc/ssh/ssh_host_ed25519_key.pub`.
|
||||
|
||||
For example, the machine specification
|
||||
|
||||
nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 1 kvm
|
||||
nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 8 2
|
||||
nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy_auto 1 2 kvm benchmark
|
||||
|
||||
specifies several machines that can perform `i686-linux` builds.
|
||||
However, `poochie` will only do builds that have the attribute
|
||||
|
||||
```nix
|
||||
requiredSystemFeatures = [ "benchmark" ];
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```nix
|
||||
requiredSystemFeatures = [ "benchmark" "kvm" ];
|
||||
```
|
||||
|
||||
`itchy` cannot do builds that require `kvm`, but `scratchy` does support
|
||||
such builds. For regular builds, `itchy` will be preferred over
|
||||
`scratchy` because it has a higher speed factor.
|
||||
|
||||
Remote builders can also be configured in `nix.conf`, e.g.
|
||||
Remote build machines can also be configured in [`nix.conf`](@docroot@/command-ref/conf-file.md), e.g.
|
||||
|
||||
builders = ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd
|
||||
|
||||
Finally, remote builders can be configured in a separate configuration
|
||||
file included in `builders` via the syntax `@file`. For example,
|
||||
Finally, remote build machines can be configured in a separate configuration
|
||||
file included in `builders` via the syntax `@/path/to/file`. For example,
|
||||
|
||||
builders = @/etc/nix/machines
|
||||
|
||||
causes the list of machines in `/etc/nix/machines` to be included. (This
|
||||
is the default.)
|
||||
|
||||
If you want the builders to use caches, you likely want to set the
|
||||
option `builders-use-substitutes` in your local `nix.conf`.
|
||||
|
||||
To build only on remote builders and disable building on the local
|
||||
machine, you can use the option `--max-jobs 0`.
|
||||
causes the list of machines in `/etc/nix/machines` to be included.
|
||||
(This is the default.)
|
||||
|
||||
@ -147,10 +147,10 @@ Nix can be built for various platforms, as specified in [`flake.nix`]:
|
||||
|
||||
In order to build Nix for a different platform than the one you're currently
|
||||
on, you need a way for your current Nix installation to build code for that
|
||||
platform. Common solutions include [remote builders] and [binary format emulation]
|
||||
platform. Common solutions include [remote build machines] and [binary format emulation]
|
||||
(only supported on NixOS).
|
||||
|
||||
[remote builders]: ../advanced-topics/distributed-builds.md
|
||||
[remote builders]: @docroot@/language/derivations.md#attr-builder
|
||||
[binary format emulation]: https://nixos.org/manual/nixos/stable/options.html#opt-boot.binfmt.emulatedSystems
|
||||
|
||||
Given such a setup, executing the build only requires selecting the respective attribute.
|
||||
|
||||
@ -37,7 +37,7 @@
|
||||
This can be achieved by:
|
||||
- Fetching a pre-built [store object] from a [substituter]
|
||||
- Running the [`builder`](@docroot@/language/derivations.md#attr-builder) executable as specified in the corresponding [derivation]
|
||||
- Delegating to a [remote builder](@docroot@/advanced-topics/distributed-builds.html) and retrieving the outputs
|
||||
- Delegating to a [remote machine](@docroot@/command-ref/conf-file.md#conf-builders) and retrieving the outputs
|
||||
<!-- TODO: link [running] to build process page, #8888 -->
|
||||
|
||||
See [`nix-store --realise`](@docroot@/command-ref/nix-store/realise.md) for a detailed description of the algorithm.
|
||||
|
||||
@ -257,7 +257,7 @@ Derivations can declare some infrequently used optional attributes.
|
||||
of the environment (typically, a few hundred kilobyte).
|
||||
|
||||
- [`preferLocalBuild`]{#adv-attr-preferLocalBuild}\
|
||||
If this attribute is set to `true` and [distributed building is enabled](../advanced-topics/distributed-builds.md), then, if possible, the derivation will be built locally instead of being forwarded to a remote machine.
|
||||
If this attribute is set to `true` and [distributed building is enabled](@docroot@/command-ref/conf-file.md#conf-builders), then, if possible, the derivation will be built locally instead of being forwarded to a remote machine.
|
||||
This is useful for derivations that are cheapest to build locally.
|
||||
|
||||
- [`allowSubstitutes`]{#adv-attr-allowSubstitutes}\
|
||||
|
||||
@ -36,7 +36,7 @@ It outputs an attribute set, and produces a [store derivation] as a side effect
|
||||
The system type on which the [`builder`](#attr-builder) executable is meant to be run.
|
||||
|
||||
A necessary condition for Nix to build derivations locally is that the `system` attribute matches the current [`system` configuration option].
|
||||
It can automatically [build on other platforms](../advanced-topics/distributed-builds.md) by forwarding build requests to other machines.
|
||||
It can automatically [build on other platforms](@docroot@/language/derivations.md#attr-builder) by forwarding build requests to other machines.
|
||||
|
||||
[`system` configuration option]: @docroot@/command-ref/conf-file.md#conf-system
|
||||
|
||||
|
||||
@ -780,9 +780,13 @@ void DerivationGoal::tryToBuild()
|
||||
|
||||
void DerivationGoal::tryLocalBuild() {
|
||||
throw Error(
|
||||
"unable to build with a primary store that isn't a local store; "
|
||||
"either pass a different '--store' or enable remote builds."
|
||||
"\nhttps://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html");
|
||||
R"(
|
||||
Unable to build with a primary store that isn't a local store;
|
||||
either pass a different '--store' or enable remote builds.
|
||||
|
||||
For more information check 'man nix.conf' and search for '/machines'.
|
||||
)"
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
|
||||
@ -331,13 +331,23 @@ void Worker::run(const Goals & _topGoals)
|
||||
if (awake.empty() && 0U == settings.maxBuildJobs)
|
||||
{
|
||||
if (getMachines().empty())
|
||||
throw Error("unable to start any build; either increase '--max-jobs' "
|
||||
"or enable remote builds."
|
||||
"\nhttps://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html");
|
||||
throw Error(
|
||||
R"(
|
||||
Unable to start any build;
|
||||
either increase '--max-jobs' or enable remote builds.
|
||||
|
||||
For more information run 'man nix.conf' and search for '/machines'.
|
||||
)"
|
||||
);
|
||||
else
|
||||
throw Error("unable to start any build; remote machines may not have "
|
||||
"all required system features."
|
||||
"\nhttps://nixos.org/manual/nix/stable/advanced-topics/distributed-builds.html");
|
||||
throw Error(
|
||||
R"(
|
||||
Unable to start any build;
|
||||
remote machines may not have all required system features.
|
||||
|
||||
For more information run 'man nix.conf' and search for '/machines'.
|
||||
)"
|
||||
);
|
||||
|
||||
}
|
||||
assert(!awake.empty());
|
||||
|
||||
@ -270,9 +270,121 @@ public:
|
||||
Setting<std::string> builders{
|
||||
this, "@" + nixConfDir + "/machines", "builders",
|
||||
R"(
|
||||
A semicolon-separated list of build machines.
|
||||
For the exact format and examples, see [the manual chapter on remote builds](../advanced-topics/distributed-builds.md)
|
||||
)"};
|
||||
A semicolon- or newline-separated list of build machines.
|
||||
|
||||
In addition to the [usual ways of setting configuration options](@docroot@/command-ref/conf-file.md), the value can be read from a file by prefixing its absolute path with `@`.
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> This is the default setting:
|
||||
>
|
||||
> ```
|
||||
> builders = @/etc/nix/machines
|
||||
> ```
|
||||
|
||||
Each machine specification consists of the following elements, separated by spaces.
|
||||
Only the first element is required.
|
||||
To leave a field at its default, set it to `-`.
|
||||
|
||||
1. The URI of the remote store in the format `ssh://[username@]hostname`.
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> `ssh://nix@mac`
|
||||
|
||||
For backward compatibility, `ssh://` may be omitted.
|
||||
The hostname may be an alias defined in `~/.ssh/config`.
|
||||
|
||||
2. A comma-separated list of [Nix system types](@docroot@/contributing/hacking.md#system-type).
|
||||
If omitted, this defaults to the local platform type.
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> `aarch64-darwin`
|
||||
|
||||
It is possible for a machine to support multiple platform types.
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> `i686-linux,x86_64-linux`
|
||||
|
||||
3. The SSH identity file to be used to log in to the remote machine.
|
||||
If omitted, SSH will use its regular identities.
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> `/home/user/.ssh/id_mac`
|
||||
|
||||
4. The maximum number of builds that Nix will execute in parallel on the machine.
|
||||
Typically this should be equal to the number of CPU cores.
|
||||
|
||||
5. The “speed factor”, indicating the relative speed of the machine as a positive integer.
|
||||
If there are multiple machines of the right type, Nix will prefer the fastest, taking load into account.
|
||||
|
||||
6. A comma-separated list of supported [system features](#conf-system-features).
|
||||
|
||||
A machine will only be used to build a derivation if all the features in the derivation's [`requiredSystemFeatures`](@docroot@/language/advanced-attributes.html#adv-attr-requiredSystemFeatures) attribute are supported by that machine.
|
||||
|
||||
7. A comma-separated list of required [system features](#conf-system-features).
|
||||
|
||||
A machine will only be used to build a derivation if all of the machine’s required features appear in the derivation’s [`requiredSystemFeatures`](@docroot@/language/advanced-attributes.html#adv-attr-requiredSystemFeatures) attribute.
|
||||
|
||||
8. The (base64-encoded) public host key of the remote machine.
|
||||
If omitted, SSH will use its regular `known_hosts` file.
|
||||
|
||||
The value for this field can be obtained via `base64 -w0`.
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> Multiple builders specified on the command line:
|
||||
>
|
||||
> ```console
|
||||
> --builders 'ssh://mac x86_64-darwin ; ssh://beastie x86_64-freebsd'
|
||||
> ```
|
||||
|
||||
> **Example**
|
||||
>
|
||||
> This specifies several machines that can perform `i686-linux` builds:
|
||||
>
|
||||
> ```
|
||||
> nix@scratchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy 8 1 kvm
|
||||
> nix@itchy.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy 8 2
|
||||
> nix@poochie.labs.cs.uu.nl i686-linux /home/nix/.ssh/id_scratchy 1 2 kvm benchmark
|
||||
> ```
|
||||
>
|
||||
> However, `poochie` will only build derivations that have the attribute
|
||||
>
|
||||
> ```nix
|
||||
> requiredSystemFeatures = [ "benchmark" ];
|
||||
> ```
|
||||
>
|
||||
> or
|
||||
>
|
||||
> ```nix
|
||||
> requiredSystemFeatures = [ "benchmark" "kvm" ];
|
||||
> ```
|
||||
>
|
||||
> `itchy` cannot do builds that require `kvm`, but `scratchy` does support such builds.
|
||||
> For regular builds, `itchy` will be preferred over `scratchy` because it has a higher speed factor.
|
||||
|
||||
For Nix to use substituters, the calling user must be in the [`trusted-users`](#conf-trusted-users) list.
|
||||
|
||||
> **Note**
|
||||
>
|
||||
> A build machine must be accessible via SSH and have Nix installed.
|
||||
> `nix` must be available in `$PATH` for the user connecting over SSH.
|
||||
|
||||
> **Warning**
|
||||
>
|
||||
> If you are building via the Nix daemon (default), the Nix daemon user account on the local machine (that is, `root`) requires access to a user account on the remote machine (not necessarily `root`).
|
||||
>
|
||||
> If you can’t or don’t want to configure `root` to be able to access the remote machine, set [`store`](#conf-store) to any [local store](@docroot@/store/types/local-store.html), e.g. by passing `--store /tmp` to the command on the local machine.
|
||||
|
||||
To build only on remote machines and disable local builds, set [`max-jobs`](#conf-max-jobs) to 0.
|
||||
|
||||
If you want the remote machines to use substituters, set [`builders-use-substitutes`](#conf-builders-use-substituters) to `true`.
|
||||
)",
|
||||
{}, false};
|
||||
|
||||
Setting<bool> alwaysAllowSubstitutes{
|
||||
this, false, "always-allow-substitutes",
|
||||
|
||||
Loading…
Reference in New Issue
Block a user