Allows detecting whether attributes are overridden in all-packages.nix.
In a future commit we'll use this to detect empty arguments being set in
all-packages.nix and complain about that.
When specifying the `builder` attribute in `stdenv.mkDerivation`, this
will be effectively transformed into
builtins.derivation {
builder = stdenv.shell;
args = [ "-e" builder ];
}
This also means that `default-builder.sh` is never sourced and as a
result it's not guaranteed that `$NIX_ATTRS_SH_FILE` is set to a correct
location[1].
Also, we need to source `.attrs.sh` to source `$stdenv`. So, the
following is done now:
* If `$NIX_ATTRS_SH_FILE` points to a correct location, then use it.
Directly using `.attrs.sh` is problematic for `nix-shell(1)` usage
(see previous commit for more context), so prefer the environment
variable if possible.
* Otherwise, if `.attrs.sh` exists, then use it. See [1] for when this
can happen.
* If neither applies, it can be assumed that `__structuredAttrs` is
turned off and thus nothing needs to be done.
[1] It's possible that it doesn't exist at all - in case of Nix 2.3 or
it can point to a wrong location on older Nix versions with a bug in
`__structuredAttrs`.
This fixes a bug where `patchShebangs` crashes when trying to patch
files that contain only a shebang (e.g. `#!/bin/bash`) (and nothing
else) and do not end with a newline.
Such file can be produced using `printf "#!/bin/bash" > example` or
`echo -n "#!/bin/bash" > example`.
I don't understand why one would want to create such files, as they do
literally nothing, but the chromium tarball we are using started
shipping some 🫠
Full reproducer:
```nix
with import <nixpkgs> { };
stdenv.mkDerivation {
dontUnpack = true;
name = "patch-shebangs-no-trailing-newline-reproducer";
postPatch = ''
printf "#!/bin/bash" > reproducer
chmod +x reproducer
patchShebangs reproducer
'';
}
```
```
❯ nix-build reproducer.nix
this derivation will be built:
/nix/store/vmbshdkdk4a0bayw3wi21wvxyhzpcsy2-patch-shebangs-no-trailing-newline-reproducer.drv
building '/nix/store/vmbshdkdk4a0bayw3wi21wvxyhzpcsy2-patch-shebangs-no-trailing-newline-reproducer.drv'...
patching sources
patching script interpreter paths in reproducer
/nix/store/vr6wwdxkmyy44sg0gwxi10b8fc5zhwz0-stdenv-linux/setup: line 144: pop_var_context: head of shell_variables not a function context
error: builder for '/nix/store/vmbshdkdk4a0bayw3wi21wvxyhzpcsy2-patch-shebangs-no-trailing-newline-reproducer.drv' failed with exit code 1;
last 3 log lines:
> patching sources
> patching script interpreter paths in reproducer
> /nix/store/vr6wwdxkmyy44sg0gwxi10b8fc5zhwz0-stdenv-linux/setup: line 144: pop_var_context: head of shell_variables not a function context
For full logs, run 'nix log /nix/store/vmbshdkdk4a0bayw3wi21wvxyhzpcsy2-patch-shebangs-no-trailing-newline-reproducer.drv'.
```
We seem to have enough tests to run into this now:
error: creating symlink from '/private/tmp/nix-build-nixpkgs-check-by-name.drv-0/source/test-tmp/var/nix/gcroots/profiles' to '/private/tmp/nix-build-nixpkgs-check-by-name.drv-0/source/test-tmp/var/nix/profiles': File exists
On Darwin, /tmp is sometimes a symlink to /private/tmp, which couldn't
be handled before:
error: access to canonical path '/private/var/folders/xp/9_ry6h9x6l9gh_g32qspz0_40000gp/T/.tmpFbcNO0' is forbidden in restricted mode
This both fixes that and adds a test to make sure it can't happen again
This reverts commit d33a309f2c9f85c28cb897e623503f210057a77a.
commit caused ofborg-eval to fail with
```
$ nix-build pkgs/top-level/release.nix -A tests.cc-wrapper.gcc10Stdenv.aarch64-darwin
error: evaluation aborted with the following error message: 'cannot find attribute `tests.cc-wrapper.gcc10Stdenv''
```
on the 3 other platfomrs the attr is there
ignore `llvmPackages_latest`, it's an alias and it'll be a duplicate.
ignore `llvmPackages_git`, it's not supposed to be used in nixpkgs and
may be broken.
This introduces the `pkgs/by-name` directory as proposed by RFC 140.
Included are:
- The implementation to add packages defined in that directory to the
top-level package scope
- Contributer documentation on how to add packages to it
- A GitHub Actions workflow to check the structure of it on all PRs
most tests use debian-devscripts' hardening-check, so only work on
ELF systems and can only detect a limited subset of flags.
some extra tests actually execute fortify-protected programs and
should be slightly more universally applicable.
- Better filesystem case-sensitivity heuristic
We shouldn't assume that Linux is always case-sensitive.
- Don't include case-sensitive filename in tree
Was used for tests, but this broke channel updates because there's a
check to make sure there's no case-sensitive files!
https://hydra.nixos.org/build/233371356/nixlog/1
This commit does the following two things:
1. Changes the Haskell package used in the extraDependencies test from
releaser to conduit. The reason for this is that conduit is more
of a "core" package in Haskell, and it is more likely to always be
working. (If conduit is not compiling, then large chunks of Hackage
won't be working.)
2. Tighten up the GHCi test to actually use the package from
extraDependencies. It appears that GHCi can fail to import the
package from extraDependencies, and that doesn't cause GHCi to
return an error code, which means the test isn't actually testing
anything.
This new change makes it so that if the package from
extraDependencies is actually not included for some reason, then
this shellFor test should fail.
```
nix-repl> (pkgs.htop.overrideAttrs { pname = "hello-overriden"; }).pname
error:
… while evaluating a branch condition
at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/lib/customisation.nix:86:7:
85| in
86| if builtins.isAttrs result then
| ^
87| result // {
… while calling the 'isAttrs' builtin
at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/lib/customisation.nix:86:10:
85| in
86| if builtins.isAttrs result then
| ^
87| result // {
(stack trace truncated; use '--show-trace' to show the full trace)
error: attempt to call something which is not a function but a set
at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/pkgs/stdenv/generic/make-derivation.nix:58:21:
57| f = self: super:
58| let x = f0 super;
| ^
59| in
```
This commit adds `pkgs.test.cross.sanity`, which is meant to be a
carefully curated list of builds/packages that tend to break when
refactoring our cross-compilation infrastructure.
It should strike a balance between being small enough to fit in
a single eval (i.e. not so large that hydra-eval-jobs is needed)
so we can ask @ofborg to check it, yet should have good examples
of things that often break. So, no buckshot `mapTestOnCross`
calls here.
Makes overrideAttrs usable in the same way that `override` can be used.
It allows the first argument of `overrideAttrs` to be either a function
or an attrset, instead of only a function:
hello.overrideAttrs (old: { postBuild = "echo hello"; })
hello.overrideAttrs { postBuild = "echo hello"; }
Previously only the first example was possible.
Co-authored-by: adisbladis <adisbladis@gmail.com>
Co-authored-by: matthewcroughan <matt@croughan.sh>
This change adds a flag to update shebang paths that point to the Nix
store. This is particularly useful when a cross-compiled package uses
same script at compile-time and run-time, but the interpreter must be
changed since hostPlatform != buildPlatform.
With a recent hackage update, turtle stopped compiling on ghc94. This
commit changes the tests.haskell.incremental test to use the temporary
package instead of turtle.
We shouldn't try compiling to aarch64-unknown-linux-gnu from
aarch64-unknown-linux-gnu, because that tends to confuse things and is
not representative of actual cross compilation usage.
This commit adds a derivation `gcc-stageCompare` to
`pkgs/test/stdenv/default.nix`.
It is important to always build this derivation whenever building
`stdenv`! Because we are using a Nix-driven bootstrap instead of
gcc's built-in `--enable-bootstrap`, the `gcc` derivation no longer
performs the post-self-compilation sanity check. You must build
this derivation in order to perform that sanity check.
The major benefit of this new approach is that the sanity check
(which involves a third compilation of gcc) can be performed
*concurrently* with all packages that depend on `stdenv`, rather
than serially. Since `stdenv` has very little derivation-level
parallelism it cannot take advantage of more than one or perhaps two
builders. If you have three or more builders this commit will
reduce the time-to-rebuild-stdenv by around 20% (one of three gcc
rebuilds is removed from the critical path, and stdenv's build time
is dominated by roughly 3*gcc + 1*binutils + 1*bison-test-suite).
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
Set the executable bit before running the check phase, so that the check
phase can run the script to test its behaviour.
This aligns with what `concatTextFile` is doing.
Also use explicit `if` statements so that we don't silently ignore
`chmod` failures.
with structuredAttrs lists will be bash arrays which cannot be exported
which will be a issue with some patches and some wrappers like cc-wrapper
this makes it clearer that NIX_CFLAGS_COMPILE must be a string as lists
in env cause a eval failure
Some packages are only marked broken on specific platforms, so we
filter those out as well.
Consequently, this might not raise an error if the attribute value
needs to point to a different pkgs attribute, but this is not something
we can detect. For now, we'll have to rely on users of such pkg-config
packages to report that kind of error.
There's really not much we can do about this here.
error: The option `settings.NIXOS_TEST_BOOLEAN.tristate' has conflicting definition values:
- In `structuredExtraConfig': "n"
- In `structuredExtraConfig': "y"
since https://github.com/NixOS/nixpkgs/pull/90065
yes does not silently win over no
Closes#16182
This improves the error message
Error: _assignFirst found no valid variant!
which occurred when the set of outputs was not sufficient to set
the various outputDev, outputBin, etc variables. Specifically, this
would mean that "out" is not among the outputs, which is valid for
a derivation.
This changes the message to something like
error: _assignFirst: could not find a non-empty variable to assign to outputDev. The following variables were all unset or empty: dev out.
If you did not define an "out" output, make sure to define all the specific required outputs: define an output for one of the unset variables.
While this isn't a full explanation of what stdenv can and can not do,
I think it's vast improvement over the 0 bits of information that it
used to provide. This at least gives a clue as to what's going on, and
even suggests a fix, although probably multiple such fixes are required
in an instance where someone starts with a no-out derivation from scratch
(and decide to persist).
Derivations not using `__structuredAttrs` should not attempt to set
environment variables from `env`.
Derivations using `__structuredAttrs` should fail if `env` is not
exportable.
Avoids confusion: `vim-full`'s build-time features are configurable, but both
`vim` and `vim-full` are *customizable* (in the sense of user configuration).
Previously, the cutensor samples could not find the libcutensor.so.1
shared library at runtime. This patch adds cutensor as a buildInput so
the shared library is linked in properly.
* origin/staging-next: (62 commits)
Re-Revert "lua: fix on darwin by using makeBinaryWrapper (#172749)"
openldap: fix cross-compilation
makeBinaryWrapper: fix codesign on aarch64-darwin
python3Packages.ldap: fix linking with openldap 2.5+
Revert "lua: fix on darwin by using makeBinaryWrapper (#172749)"
wine: enable parallel build again
pkgsi686Linux.gdb: fix formatting for 32-bit systems
gtk4: Fix incorrect merge
nixos/openldap: use upstream unit defaults
openldap: update maintainers
openldap: 2.4.58 -> 2.6.2
Revert "Add mingwW64-llvm cross-system."
lua: fix on darwin by using makeBinaryWrapper (#172749)
python310Packages.python-mimeparse: execute tests
pandas: fix darwin build
gtk3: 3.24.33 -> 3.24.33-2022-03-11
gtk4: patch fixing g-c-c crashes
e2fsprogs: patch for CVE-2022-1304
firefox-unwrapped: fix cross compilation
rustc: expose correct llvmPackages for cross compile
...
There are many different versions of the `cudatoolkit` and related
cuda packages, and it can be tricky to ensure they remain compatible.
- `cudaPackages` is now a package set with `cudatoolkit`, `cudnn`, `cutensor`, `nccl`, as well as `cudatoolkit` split into smaller packages ("redist");
- expressions should now use `cudaPackages` as parameter instead of the individual cuda packages;
- `makeScope` is now used, so it is possible to use `.overrideScope'` to set e.g. a different `cudnn` version;
- `release-cuda.nix` is introduced to easily evaluate cuda packages using hydra.
Disable file globbing in --prefix/--suffix, since bash will otherwise
try to find filenames matching the the value to be prefixed/suffixed
if it contains characters considered wildcards, such as `?` and
`*`. We want the value as is, except we also want to split it on on
the separator; hence we can't quote it.
Sanitizers don't seem to be present on aarch64-darwin/macOS 12 (Monterey), so they are removed from the aarch64-darwin tests.
Switching from nativeBuildInputs to buildInputs and adding cc to the deps list caused some strange error messages to go away.
On macOS, /tmp is a symlink to /private/tmp. When performing cd /tmp, and checking cwd - it won't match since it follows the symlink.
This caused test breakage on macOS but not Linux. Instead, use a folder which is not a symlink, and consistent across Linux and macOS.
This allows packagePlatforms to pick up on the overall supported
platforms and schedule builds on Hydra for more than the evaluation
platform (usually x86_64-linux).
Since the rust writer doesn't seem to get fixed on darwin, we'll just
wrap the haskell writer test in our own derivation (which is possible
since tests.writers exposes a bunch of internals via passthru) and
expose it via tests.haskell which are already in mergeable.
Finally a way to test the (hopefully) working haskell writer on darwin
again!
