mirror of
https://github.com/NixOS/nixpkgs.git
synced 2024-12-15 18:23:09 +00:00
c2b898da76
Passing `-l$NIX_BUILD_CORES` improperly limits the overall system load. For a build machine which is configured to run `$B` builds where each build gets `total cores / B` cores (`$C`), passing `-l $C` to make will improperly limit the load to `$C` instead of `$B * $C`. This effect becomes quite pronounced on machines with 80 cores, with 40 simultaneous builds and a cores limit of 2. On a machine with this configuration, Nix will run 40 builds and make will limit the overall system load to approximately 2. A build machine with this many cores can happily run with a load approaching 80. A non-solution is to oversubscribe the machine, by picking a larger `$C`. However, there is no way to divide the number of cores in a way which fairly subdivides the available cores when `$B` is greater than 1. There has been exploration of passing a jobserver in to the sandbox, or sharing a jobserver between all the builds. This is one option, but relatively complicated and only supports make. Lots of other software uses its own implementation of `-j` and doesn't support either `-l` or the Make jobserver. For the case of an interactive user machine, the user should limit overall system load using `$B`, `$C`, and optionally systemd's cpu/network/io limiting features. Making this change should significantly improve the utilization of our build farm, and improve the throughput of Hydra. |
||
|---|---|---|
| .. | ||
| arch | ||
| bcompare | ||
| blackbox | ||
| bumpver | ||
| commit-formatter | ||
| commitizen | ||
| cvs | ||
| cvs2svn | ||
| cvs-fast-export | ||
| cvsps | ||
| cvsq | ||
| cz-cli | ||
| danger-gitlab | ||
| datalad | ||
| dvc | ||
| fnc | ||
| fornalder | ||
| fossil | ||
| gerrit | ||
| git-and-tools | ||
| git-backup | ||
| git-crecord | ||
| git-lfs | ||
| git-repo | ||
| git-review | ||
| git-sizer | ||
| git-up | ||
| gitea | ||
| github-desktop | ||
| gitkraken | ||
| gitlab | ||
| gitlab-triage | ||
| gitless | ||
| gitolite | ||
| gitoxide | ||
| gitqlient | ||
| gitstats | ||
| gogs | ||
| gomp | ||
| got | ||
| gource | ||
| josh | ||
| jujutsu | ||
| meld | ||
| mercurial | ||
| monotone | ||
| monotone-viz | ||
| mr | ||
| nbstripout | ||
| nitpick | ||
| p4 | ||
| p4d | ||
| p4v | ||
| peru | ||
| pijul | ||
| rabbitvcs | ||
| rapidsvn | ||
| rcs | ||
| rcshist | ||
| redmine | ||
| reposurgeon | ||
| smartgithg | ||
| sourcehut | ||
| sparkleshare | ||
| src | ||
| sublime-merge | ||
| subversion | ||
| tailor | ||
| tkrev | ||
| tortoisehg | ||
| vcprompt | ||
| vcsh | ||
| verco | ||
| yadm | ||