Commit Graph

781 Commits

Author SHA1 Message Date
Silvan Mosberger
fbbe972898 Parallel GH actions workflow for Nixpkgs eval
Motivated by ofborg struggling [1] and its evaluations taking too long,
inspired by Jörg's initial PR [2]
and Adam's previous attempt to parallelise Nixpkgs evaluation [3],
this PR contains initial work to relief ofborg from its evaluation duty
by using GitHub Actions to evaluate Nixpkgs.

For now this doesn't take care of all of what ofborg does, such as
requesting appropriate reviewers or labeling mass rebuilds, but this can
be follow-up work.

[1]: https://discourse.nixos.org/t/infrastructure-announcement-the-future-of-ofborg-your-help-needed/56025?u=infinisil
[2]: https://github.com/NixOS/nixpkgs/pull/352808
[3]: https://github.com/NixOS/nixpkgs/pull/269403

Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
Co-Authored-By: Adam Joseph <adam@westernsemico.com>
2024-11-20 10:35:56 +01:00
github-actions[bot]
fe71abdc48
Merge staging-next into staging 2024-11-04 12:06:38 +00:00
Doron Behar
b83db93b6b
libtinfo: init at 6.4 (#285347) 2024-11-04 09:11:05 +02:00
Doron Behar
c3b147d34a tests.buildFHSEnv.libtinfo: simplify using libtinfo 2024-11-03 01:08:36 +02:00
Maximilian Bosch
aaeeef5b6c
stdenv: fix custom hardening settings when using __structuredAttrs = true;
Replaces / Closes #353131

A while ago `postgresql` switched to using structured attrs[1]. In the
PR it was reported that this made postgresql notably slower when
importing SQL dumps[2].

After a bit of debugging it turned out that the hardening was entirely
missing and the following combination of settings was the culprit:

    hardeningEnable = [ "pie" ];
    __structuredAttrs = true;

I.e. the combination of custom hardening settings and structured attrs.

What happened here is that internally the default and enabled hardening
flags get written into `NIX_HARDENING_ENABLE`. However, the value is a list
and the setting is not in the `env` section. This means that in the
structured-attrs case we get something like

    declare -ax NIX_HARDENING_ENABLE=([0]="bindnow" [1]="format" [2]="fortify" [3]="fortify3" [4]="pic" [5]="relro" [6]="stackprotector" [7]="strictoverflow" [8]="zerocallusedregs" [9]="pie")

i.e. an actual array rather than a string with all hardening flags being
space-separated which is what the hardening code of the cc-wrapper
expects[3].

This only happens if `hardeningEnable` or `hardeningDisable` are
explicitly set by a derivation: if none of those are set,
`NIX_HARDENING_ENABLE` won't be set by `stdenv.mkDerivation` and the
default hardening flags are configured by the setup hook of the
cc-wrapper[4].

In other words, this _only_ applies to derivations that have both custom
hardening settings _and_ `__structuredAttrs = true;`.

All values of `NIX_HARDENING_ENABLE` are well-known, so we don't have to
worry about escaping issues. Just forcing it to a string by
concatenating the list everytime solves the issue without additional
issues like eval errors when inheriting `env` from a structuredAttrs
derivation[5]. The price we're paying is a full rebuild.

[1] https://github.com/NixOS/nixpkgs/pull/294504
[2] https://github.com/NixOS/nixpkgs/pull/294504#issuecomment-2451482522
[3] cf3e5d3744/pkgs/build-support/cc-wrapper/add-hardening.sh (L9)
[4] cf3e5d3744/pkgs/build-support/cc-wrapper/setup-hook.sh (L114)
[5] 1e84a7fb95
2024-11-02 22:30:01 +01:00
github-actions[bot]
6e6e33bd99
Merge staging-next into staging 2024-11-01 18:05:02 +00:00
Philip Taron
8d9fcbc47a
tests/srcOnly: init (#347548) 2024-11-01 09:13:09 -07:00
github-actions[bot]
336ffcf0e4
Merge staging-next into staging 2024-10-31 18:04:59 +00:00
Artturin
e99719836d
tests.cc-wrapper.default: reduce rebuild amount from 429 to 319 (#351768) 2024-10-31 19:48:51 +02:00
github-actions[bot]
8f11852505
Merge staging-next into staging 2024-10-30 18:05:08 +00:00
Tristan Ross
fb01d572b5
release-lib: build packages without meta.platforms on all platforms (#351919) 2024-10-30 09:35:04 -07:00
Emily
9f36206578 tests.build-environment-info: add more information
The user information would have helped diagnose the empty GECOS issue
we had with the Hydra Darwin builders recently and the OS information
seems generally nice to have.
2024-10-29 19:42:46 +00:00
Emily
f594f608dc tests.nixos-functions: don’t omit derivations on non‐Linux
Hydra can handle packages that `throw` but not attributes that
disappear depending on platform, and it seems okay for evaluation to
fail here for non‐Linux platforms.
2024-10-29 19:42:46 +00:00
github-actions[bot]
0ad7ce5327
Merge staging-next into staging 2024-10-29 00:14:56 +00:00
Philip Taron
9a18caa5cd
bintools-wrapper: fix late‐bound command references (#351531) 2024-10-28 07:05:32 -07:00
Philip Taron
b37bbeb14f
tests.cc-wrapper.supported: remove obsolete Darwin conditionals (#351535) 2024-10-28 07:05:03 -07:00
github-actions[bot]
ca29a063ab
Merge staging-next into staging 2024-10-28 06:05:39 +00:00
Emily
298b92814b tests.build-environment-info: init
We just had a marathon session trying to fix `ulimit -n` on Hydra and
have no easy way to test that it worked.
2024-10-28 02:03:31 +00:00
Artturin
5f2d5d77da tests.cc-wrapper.default: reduce rebuild amount from 429 to 319
On native the `emulator` is now `execline` which adds 110 more dependencies.
2024-10-28 00:50:16 +02:00
Emily
6730550f7b tests.cc-wrapper.supported: remove obsolete Darwin conditionals
LLVM 9 is gone, and the `gcc11Stdenv` test passes fine now.
2024-10-26 23:14:48 +01:00
Emily
04ba4d1a3b bintools-wrapper: fix late‐bound command references
This ports 985072e2ce to bintools-wrapper.

Co-authored-by: Artturin <Artturin@artturin.com>
Co-authored-by: IlyaNiklyaev <ilya.niklyaev@gmail.com>
2024-10-27 00:32:46 +03:00
Artturin
62f2fc1ae5 tests.cc-wrapper: Add NIX_CC_USE_RESPONSE_FILE check 2024-10-21 21:41:12 +03:00
Artturin
8a874c4dc8 tests-cc-wrapper: Ignore gcc aliases 2024-10-18 23:45:22 +03:00
Artturin
8de11b252d tests.cc-wrapper: Run CC CXX with an empty path
Ensure our wrappers hardcode all the necessary tools
2024-10-18 23:45:22 +03:00
Tristan Ross
959200a01e
{clang-sierraHack{,-stdenv},tests.macOSSierraShared}: drop (#346730) 2024-10-14 11:11:56 -07:00
Gavin John
1a1a4ebd7d
tests/srcOnly: init 2024-10-11 17:55:49 -07:00
Emily
b2cb2a8d44 {clang-sierraHack{,-stdenv},tests.macOSSierraShared}: drop
This is some thing introduced in 2017 to work around a problem that
no longer seems to exist. Nothing uses it except its own test, which
these days passes even with the standard `clangStdenv`.
2024-10-05 21:29:07 +01:00
github-actions[bot]
653bcc0ea9
Merge master into staging-next 2024-10-05 18:04:03 +00:00
sternenseemann
b341506a8d Merge branch master into haskell-updates
Conflicts from #341407 resolved.
2024-09-27 11:41:25 +02:00
Artturin
f0e657f3b1 Merge branch 'master' into staging-next 2024-09-25 06:05:01 +03:00
Artturin
e0464e4788 treewide: replace stdenv.is with stdenv.hostPlatform.is
In preparation for the deprecation of `stdenv.isX`.

These shorthands are not conducive to cross-compilation because they
hide the platforms.

Darwin might get cross-compilation for which the continued usage of `stdenv.isDarwin` will get in the way

One example of why this is bad and especially affects compiler packages
https://www.github.com/NixOS/nixpkgs/pull/343059

There are too many files to go through manually but a treewide should
get users thinking when they see a `hostPlatform.isX` in a place where it
doesn't make sense.

```
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv.is" "stdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv'.is" "stdenv'.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "clangStdenv.is" "clangStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "gccStdenv.is" "gccStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenvNoCC.is" "stdenvNoCC.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "inherit (stdenv) is" "inherit (stdenv.hostPlatform) is"
fd --type f "\.nix" | xargs sd --fixed-strings "buildStdenv.is" "buildStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "effectiveStdenv.is" "effectiveStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "originalStdenv.is" "originalStdenv.hostPlatform.is"
```
2024-09-25 00:04:37 +03:00
github-actions[bot]
bb260f91c4
Merge master into haskell-updates 2024-09-18 00:15:34 +00:00
github-actions[bot]
73cf025711
Merge staging-next into staging 2024-09-17 12:05:50 +00:00
nicoo
423da96e75 test/release: do not add .git to the store
This is slow, and currently takes 5 GiB of extra space on every invocation.
2024-09-17 09:44:23 +00:00
nicoo
493442c470 tests: avoid copying .git into the store 2024-09-17 09:39:51 +00:00
github-actions[bot]
35f1a5b515
Merge master into haskell-updates 2024-09-14 00:15:12 +00:00
github-actions[bot]
5c9c7359c3
Merge staging-next into staging 2024-09-14 00:13:49 +00:00
Sergei Trofimovich
88950412bc gcc49, gcc49Stdenv, gfortran49: remove old implementation
gcc-4.9.4 was released in Aug 3, 2016, 8 years ago. It's a branch that
went out of support years ago. Numerous bugs never get backported to
this version.

Let's remove it.
2024-09-12 10:32:19 +01:00
Wolfgang Walther
bc0395ee6f
auto-patchelf: refactor structuredAttrs support
stdenv now provides better tooling to support structuredAttrs without
depending on $__structuredAttrs itself.
2024-09-11 19:29:39 +02:00
Wolfgang Walther
2a2b88eb55
auto-patchelf: fix test for hook
Fix suggested by Yann Hamdaoui in
https://github.com/NixOS/nixpkgs/pull/340858#issuecomment-2344155304.
2024-09-11 19:29:39 +02:00
github-actions[bot]
14ba90a8e8
Merge master into haskell-updates 2024-09-11 00:15:39 +00:00
Alex Tunstall
eeeeb555e2 tests.cross.sanity: Add GHC to catch regressions
This is to ensure that Haskell users on platforms that lack official
bindists still have a convenient means of getting GHC running natively.

In my admittedly somewhat limited testing on RISC-V, GHC 8.10.7 is able
to bootstrap native builds for 9.2.8 and 9.4.5. GHC 9.2.8 and 9.4.5 are
unable to bootstrap themselves and 9.6.2 when cross-compiled.

If you're looking at this commit to see whether you can safely upgrade
the compiler used here to remove 8.10, please try cross-compiling 9.0 or
later and then booting a native GHC with it.
2024-09-08 23:50:05 +02:00
github-actions[bot]
de15243c9a
Merge master into staging-next 2024-09-07 12:04:51 +00:00
David McFarland
14c908cdc9 tests.dotnet: add test for supported nugetDeps values 2024-09-06 16:47:36 -03:00
K900
23b4832da0 Merge remote-tracking branch 'origin/master' into staging-next 2024-09-04 08:53:48 +03:00
Philip Taron
89cbfde96d
nixpkgs-vet: update CI, docs, and release to 0.1.4
Everything gets moved into the `ci/` top-level directory.

We keep behind `maintainers/scripts/check-by-name.sh` and `pkgs/test/check-by-name/pinned-version.txt` as they are going to cause CI errors and confusion until we get all the way through the various channels.
They'll be removed in about a week or so.
2024-09-03 13:53:25 -07:00
Anderson Torres
f72e74d7ae installShellFiles: migrate tests to tests subdirectory
Rationale: Since RFCs 140 and 146, the old category-based hierarchy is
deprecated and obsolete, and a new approach took place: packages should be as
self-contained as possible.

This paradigm is reflected in many new strict checks that prohibit a package to
refer to files outside its directory tree.

Following this spirit, this commit essentially moves nixpkgs
pkgs/test/default.nix to ./tests/default.nix.

Further, to keep the top-level `tests` attribute, a green alias is kept in the
place of older file.
2024-08-28 18:51:06 -03:00
Philip Taron
0d1b26857b
treewide: support structuredAttrs in setup hooks (part 2) (#335666) 2024-08-28 14:07:58 -07:00
Artturin
740804dd5f
patch-shebangs: don't patch shebangs with bash builtins (#330188) 2024-08-27 03:54:10 +03:00
github-actions[bot]
132f2322d0
Merge master into staging-next 2024-08-26 12:05:25 +00:00