mirror of
https://github.com/NixOS/nixpkgs.git
synced 2024-12-03 20:33:21 +00:00
c2b898da76
Passing `-l$NIX_BUILD_CORES` improperly limits the overall system load. For a build machine which is configured to run `$B` builds where each build gets `total cores / B` cores (`$C`), passing `-l $C` to make will improperly limit the load to `$C` instead of `$B * $C`. This effect becomes quite pronounced on machines with 80 cores, with 40 simultaneous builds and a cores limit of 2. On a machine with this configuration, Nix will run 40 builds and make will limit the overall system load to approximately 2. A build machine with this many cores can happily run with a load approaching 80. A non-solution is to oversubscribe the machine, by picking a larger `$C`. However, there is no way to divide the number of cores in a way which fairly subdivides the available cores when `$B` is greater than 1. There has been exploration of passing a jobserver in to the sandbox, or sharing a jobserver between all the builds. This is one option, but relatively complicated and only supports make. Lots of other software uses its own implementation of `-j` and doesn't support either `-l` or the Make jobserver. For the case of an interactive user machine, the user should limit overall system load using `$B`, `$C`, and optionally systemd's cpu/network/io limiting features. Making this change should significantly improve the utilization of our build farm, and improve the throughput of Hydra. |
||
|---|---|---|
| .. | ||
| androidndk-pkgs | ||
| beam-modules | ||
| bower-modules/generic | ||
| chez-modules | ||
| compilers | ||
| coq-modules | ||
| dhall-modules | ||
| dotnet-modules | ||
| em-modules/generic | ||
| embedded | ||
| gnuradio-modules | ||
| guile-modules | ||
| haskell-modules | ||
| idris-modules | ||
| interpreters | ||
| java-modules | ||
| libraries | ||
| lisp-modules | ||
| lisp-modules-new | ||
| lua-modules | ||
| misc | ||
| mobile | ||
| nim-packages | ||
| node-packages | ||
| ocaml-modules | ||
| octave-modules | ||
| perl-modules | ||
| pharo | ||
| php-packages | ||
| python2-modules | ||
| python-modules | ||
| quickemu | ||
| r-modules | ||
| ruby-modules | ||
| scheme-modules/scheme-bytestructures | ||
| tools | ||
| web | ||