nordic-dev.net/nixpkgs
nordic-dev.net/nixpkgs
2
0
Fork 0
mirror of https://github.com/NixOS/nixpkgs.git synced 2025-02-11 14:43:47 +00:00
Code Issues Packages Projects Releases Wiki Activity
552,847 Commits 188 Branches 125 Tags 6.5 GiB
abd2811dbd
Commit Graph

1078 Commits

Author SHA1 Message Date
Weijia Wang
74d102ce50 Merge branch 'master' into staging-next 2023-11-12 19:45:20 +01:00
Artturin
c3c31aa798 stdenv: Improve performance 
| stat                   | before          | after           | Δ               | Δ%      |
|------------------------|-----------------|-----------------|-----------------|---------|
| cpuTime                | 513.67          | 507.77          | ↘ 5.90          | -1.15%  |
| envs-bytes             | 20,682,847,968  | 20,628,961,616  | ↘ 53,886,352    | -0.26%  |
| envs-elements          | 1,054,735,104   | 1,051,395,620   | ↘ 3,339,484     | -0.32%  |
| envs-number            | 765,310,446     | 763,612,291     | ↘ 1,698,155     | -0.22%  |
| gc-heapSize            | 53,439,602,688  | 51,711,545,344  | ↘ 1,728,057,344 | -3.23%  |
| gc-totalBytes          | 113,062,066,672 | 112,139,998,240 | ↘ 922,068,432   | -0.82%  |
| list-bytes             | 3,118,249,784   | 3,118,249,784   | 0               |         |
| list-concats           | 52,834,140      | 52,834,140      | 0               |         |
| list-elements          | 389,781,223     | 389,781,223     | 0               |         |
| nrAvoided              | 968,097,988     | 991,889,795     | ↗ 23,791,807    | 2.46%   |
| nrFunctionCalls        | 697,259,792     | 697,259,792     | 0               |         |
| nrLookups              | 510,257,062     | 338,275,331     | ↘ 171,981,731   | -33.70% |
| nrOpUpdateValuesCopied | 1,446,690,216   | 1,446,690,216   | 0               |         |
| nrOpUpdates            | 68,504,034      | 68,504,034      | 0               |         |
| nrPrimOpCalls          | 429,464,805     | 429,464,805     | 0               |         |
| nrThunks               | 1,009,240,391   | 982,109,100     | ↘ 27,131,291    | -2.69%  |
| sets-bytes             | 33,524,722,928  | 33,524,722,928  | 0               |         |
| sets-elements          | 1,938,309,212   | 1,938,309,212   | 0               |         |
| sets-number            | 156,985,971     | 156,985,971     | 0               |         |
| sizes-Attr             | 16              | 16              | 0               |         |
| sizes-Bindings         | 16              | 16              | 0               |         |
| sizes-Env              | 16              | 16              | 0               |         |
| sizes-Value            | 24              | 24              | 0               |         |
| symbols-bytes          | 2,151,298       | 2,151,298       | 0               |         |
| symbols-number         | 159,707         | 159,707         | 0               |         |
| values-bytes           | 30,218,194,248  | 29,567,043,264  | ↘ 651,150,984   | -2.15%  |
| values-number          | 1,259,091,427   | 1,231,960,136   | ↘ 27,131,291    | -2.15%  |

> Accessing the lexical scope directly should be more efficient, yes, because it changes from a binary search (many lookups) to just two memory accesses
> correction: one short linked list + one array access
> oh and you had to do the lexical scope lookup anyway for lib itself
> so it really does save a binary search at basically no extra cost

- roberth

after seeing the stats

> Oooh nice. I did not consider that more of the maybeThunk optimization becomes effective (nrAvoided). Those lookups also caused allocations!

- roberth

Left `lib.generators` and `lib.strings` alone because they're only used
once.
 2023-11-12 10:42:49 +02:00
Alyssa Ross
506dcaabb4 stdenv: fix meson rust cross compilation 2023-11-09 10:02:24 +01:00
github-actions[bot]
fecd99b105
Merge staging-next into staging 2023-11-09 06:01:40 +00:00
github-actions[bot]
4aad65bfdb
Merge master into staging-next 2023-11-09 06:01:11 +00:00
Artturi
71f42b7f1a
Merge pull request #230874 from Ninlives/runPhase 2023-11-07 20:00:47 +02:00
mlatus
707a5c09db stdenv: wrap phase running actions of genericBuild 
Provide a `runPhase` function which wraps the phase running action of
genericBuild. The new function can be used as an interface by `nix
develop`, i.e. `nix develop some#flake --build` may just call `runPhase
build`, which makes its behavior more consistent with `nix build`.
In preparation of fixing https://github.com/NixOS/nix/issues/6202
 2023-11-07 03:31:26 +02:00
Robert Hensing
e143a933f6 check-meta.nix: Fix flake note 
- These new-cli commands can be used with `-f`, in which case they're
  evaluated with pure evaluation disabled.
- Nix 2.4+ is not part of the condition; "flakes" is fully descriptive
  and more relatable.
- Don't suggest that it only enables this variable.
- Just don't say too much.
 2023-10-31 12:44:36 +01:00
github-actions[bot]
96132a216a
Merge master into staging-next 2023-10-26 18:01:02 +00:00
nicoo
87c22100a6 stdenv.mkDerivation: Reject MD5 hashes 
While there is no fetcher or builder (in nixpkgs) that takes an `md5` parameter,
for some inscrutable reason the nix interpreter accepts the following:
```nix
fetchurl {
  url = "https://www.perdu.com";
  hash = "md5-rrdBU2a35b2PM2ZO+n/zGw==";
}
```

Note that neither MD5 nor SHA1 are allowed by the syntax of SRI hashes.
 2023-10-25 23:14:15 +00:00
Maximilian Bosch
8bc5104a6e
treewide: refactor .attrs.sh detection 
When specifying the `builder` attribute in `stdenv.mkDerivation`, this
will be effectively transformed into

    builtins.derivation {
      builder = stdenv.shell;
      args = [ "-e" builder ];
    }

This also means that `default-builder.sh` is never sourced and as a
result it's not guaranteed that `$NIX_ATTRS_SH_FILE` is set to a correct
location[1].

Also, we need to source `.attrs.sh` to source `$stdenv`. So, the
following is done now:

* If `$NIX_ATTRS_SH_FILE` points to a correct location, then use it.
  Directly using `.attrs.sh` is problematic for `nix-shell(1)` usage
  (see previous commit for more context), so prefer the environment
  variable if possible.

* Otherwise, if `.attrs.sh` exists, then use it. See [1] for when this
  can happen.

* If neither applies, it can be assumed that `__structuredAttrs` is
  turned off and thus nothing needs to be done.

[1] It's possible that it doesn't exist at all - in case of Nix 2.3 or
    it can point to a wrong location on older Nix versions with a bug in
    `__structuredAttrs`.
 2023-10-04 18:36:57 +02:00
Maximilian Bosch
53c5f08415
stdenv: refactor .attrs.sh detection 
Relying on `.attrs.sh` to exist in `$NIX_BUILD_TOP` is problematic
because that's not compatible with how `nix-shell(1)` behaves. It places
`.attrs.{json,sh}` into a temporary directory and makes them accessible via
`$NIX_ATTRS_{SH,JSON}_FILE` in the environment[1]. The sole reason that
`nix-shell(1)` still works with structured-attrs enabled derivations
is that the contents of `.attrs.sh` are sourced into the
shell before sourcing `$stdenv/setup` (if `$stdenv` exists) by `nix-shell`.

However, the assumption that two files called `.attrs.sh` and
`.attrs.json` exist in `$NIX_BUILD_TOP` is wrong in an interactive shell
session and thus an inconsistency between shell debug session and actual
builds which can lead to unexpected problems.

To be precise, we currently have the following problem: an expression
like

  with import ./. {};
  runCommand "foo" { __structuredAttrs = true; foo.bar = [ 1 2 3 ]; }
    ''
      echo "''${__structuredAttrs@Q}"
      touch $out
    ''

prints `1` in its build-log. However when building interactively in a
`nix-shell`, it doesn't.

Because of that, I'm considering to propose a full deprecation of
`$NIX_BUILD_TOP/.attrs.{json,sh}`. A first step is to only mention the
environment variables, but not the actual paths anymore in Nix's
manual[2]. The second step - this patch - is to fix nixpkgs' stdenv
accordingly.

Please note that we cannot check for `-e "$NIX_ATTRS_JSON_FILE"` because
certain outdated Nix minors (that are still in the range of supported
Nix versions in `nixpkgs`) have a bug where `NIX_ATTRS_JSON_FILE` points
to the wrong file while building[3].

Also, for compatibility with Nix 2.3 which doesn't provide these
environment variables at all we still need to check for the existence of
.attrs.json/.attrs.sh here. As soon as we bump nixpkgs' minver to 2.4,
this can be dropped.

Finally, dropped the check for ATTRS_SH_FILE because that was never
relevant. In nix#4770 the ATTRS_SH_FILE variable was introduced[4] and
in a review iteration prefixed with NIX_[5]. In other words, these
variables were never part of a release and you'd only have this problem
if you'd use a Nix from a git revision of my branch from back then. In
other words, that's dead code.

[1] https://github.com/nixos/nix/pull/4770#issuecomment-834718851
[2] https://github.com/NixOS/nix/pull/9032
[3] https://github.com/NixOS/nix/issues/6736
[4] 3944a120ec
[5] 27ce722638
 2023-10-04 18:35:30 +02:00
Artturi
d5139e3017
Merge pull request #245909 from Artturin/setupshchanges2 2023-09-04 20:41:16 +03:00
Artturin
1db1e3c467 stdenv: Fix possible issues discovered with 
```
set -o errexit -o nounset -o pipefail
shopt -s inherit_errexit
```

in `stdenv/default-builder.sh`
 2023-09-02 03:25:36 +03:00
Artturin
c25f7f894f stdenv: Add hack to fix cmake canExecute cross-compilation 
Fixes `pkgsCross.musl64.llvmPackages_16.clang.cc` on `x86_64-linux`,
which used to fail with `/bin/sh: clang-tblgen: not found`.

Same hack is used in other projects:
https://github.com/search?q=%2FCMAKE_CROSSCOMPILING_EMULATOR.%2B%5C%2Fusr%5C%2Fbin%5C%2Fenv%2F+NOT+is%3Afork&type=code

Comment from 30435a9d0f/build/cmake/HostLinuxToolchain.cmake (L64)

> Required to run host Linux executables during the build itself.
> An example would be https://gitub.com/KhronosGroup/Vulkan-Loader and
> its "asm_offset" program.
>
> NOTE: Alternatives have been tried unsuccessfully, i.e.:
>
>  With $(set CMAKE_CROSSCOMPILING_EMULATOR), the build fails because
>  the CMake ninja/Make script tries to find the executable in the current
>  path, as in:
>
>    [3/16] Generating gen_defines.asm
>    FAILED: loader/gen_defines.asm
>    cd /tmp/cc/build-Vulkan-Loader/loader && asm_offset GAS
>    /bin/sh: asm_offset: command not found
>    ninja: build stopped: subcommand failed.
>
> With $(set CMAKE_CROSSCOMPILING_EMULATOR ""), the build fails because
> the shell cannot find the "" program as in:
>
>    [3/16] Generating gen_defines.asm
>    FAILED: loader/gen_defines.asm
>    cd /tmp/cc/build-Vulkan-Loader/loader && "" /tmp/cc/build-Vulkan-Loader/loader/asm_offset GAS
>    /bin/sh: : command not found
>    ninja: build stopped: subcommand failed.
>
> It seems that the root of the problem comes from how the CMake function
> cmCustomCommandGenerator::GetArgc0Location() computes the target
> executable's location. At this point it's unclear whether this is a CMake
> bug or a feature.

Risicle discovered this hack.

Co-authored-by: Robert Scott <code@humanleg.org.uk>
 2023-08-29 22:12:06 +03:00
Artturin
87db45704f stdenv: Print _allFlags debug output to stderr 
In the default `fixupPhase` the output of `substituteAllStream` is
streamed to setup-hook.

`stdenv.cc.bintools.overrideAttrs { NIX_DEBUG = 6; }`

With `NIX_DEBUG` contains:

```
@expandResponseParams@ -> /nix/store/yl01rd58vp4m8bbhkihpk132cprfmx6f-expand-response-params/bin/expand-response-params
...
```
 2023-08-10 17:35:07 +03:00
github-actions[bot]
0031ae7742
Merge staging-next into staging 2023-08-06 00:03:36 +00:00
github-actions[bot]
c90f3611dc
Merge master into staging-next 2023-08-06 00:02:21 +00:00
Jan Malakhovski
85d4248a4f
treewide: fix some comments (#247365) 
* pkgs/top-level/splice.nix: fix broken comment

* stdenv: use a value that actually works in the comment there
 2023-08-05 23:01:26 +02:00
Vladimír Čunát
2141d9879a
Revert "stdenv: use improved strip.sh for aarch64-linux" 
This reverts commit 39919b8f21.
The parent merge resolved this more properly.
 2023-07-31 11:20:15 +02:00
Vladimír Čunát
39919b8f21
stdenv: use improved strip.sh for aarch64-linux 
Adapted from PR #246164
TODO: clean up / use it everywhere on the next rebuild.
 2023-07-30 21:18:30 +02:00
Artturin
9816ca55c2 stdenv: Make condition clearer 
-z checks for a empty string
-n checks for a not empty string

It makes more sense to run the chmod if the string is not empty
 2023-07-28 04:20:26 +03:00
Vladimír Čunát
58392652f0
Merge #224822: hardening flags: enable fortify3 by default 
...into staging
 2023-07-06 10:33:24 +02:00
Artturin
42c94d06fd stdenv: fix overriding with attrset when finalAttrs isn't used 
```
nix-repl> (pkgs.htop.overrideAttrs { pname = "hello-overriden"; }).pname
error:
       … while evaluating a branch condition

         at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/lib/customisation.nix:86:7:

           85|     in
           86|       if builtins.isAttrs result then
             |       ^
           87|         result // {

       … while calling the 'isAttrs' builtin

         at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/lib/customisation.nix:86:10:

           85|     in
           86|       if builtins.isAttrs result then
             |          ^
           87|         result // {

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: attempt to call something which is not a function but a set

       at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/pkgs/stdenv/generic/make-derivation.nix:58:21:

           57|           f = self: super:
           58|             let x = f0 super;
             |                     ^
           59|             in
```
 2023-07-03 21:34:50 +03:00
Robert Hensing
1f5a0b0135
Merge pull request #239005 from Artturin/setorfunoverr 
stdenv: let overrideAttrs accept attrset OR function
 2023-07-01 15:27:23 +02:00
Artturin
0fdae31531 stdenv: let overrideAttrs accept attrset OR function 
Makes overrideAttrs usable in the same way that `override` can be used.
It allows the first argument of `overrideAttrs` to be either a function
or an attrset, instead of only a function:

hello.overrideAttrs (old: { postBuild = "echo hello"; })
hello.overrideAttrs { postBuild = "echo hello"; }

Previously only the first example was possible.

Co-authored-by: adisbladis <adisbladis@gmail.com>
Co-authored-by: matthewcroughan <matt@croughan.sh>
 2023-06-30 23:28:00 +03:00
Robert Scott
db3e94c3b2 hardening flags: enable fortify3 by default 2023-06-25 11:50:46 +01:00
Artturin
d88b362808 stdenv: finish removal of deprecated string/null Flags 
forgot to do it before 23.05 but it's ok
 2023-06-22 20:37:06 +03:00
Artturin
ac031478cb stdenv: use lib.isX instead of typeOf Y == X 
more readable
 2023-06-22 12:31:52 +03:00
Robert Hensing
dae6b98a90
Merge pull request #234883 from sternenseemann/nix-structured-attrs 
stdenv/setup.sh: deal with Nix < 2.4 structured attrs
 2023-05-31 19:41:58 +02:00
sternenseemann
1a29857b8a stdenv/setup.sh: deal with Nix < 2.4 structured attrs 
Nix does not (as far it is documented) guarantee that NIX_ATTRS_*_FILE
is set, the only [documented] guarantee seems to be:

> […] made available to the builder via the file .attrs.json in the
> builder’s temporary directory.

This guarantee is of course affected by https://github.com/NixOS/nix/issues/6736,
so it seems to be prudent to fall back to the Nix 2.3 style ATTRS_*_FILE
env vars before defaulting to the expected location in case neither is
available.

See also:

- https://github.com/NixOS/nixpkgs/pull/214937#discussion_r1178101895
- https://github.com/nixos/nixpkgs/commit/afef6588e250

[documented]: https://nixos.org/manual/nix/stable/language/advanced-attributes.html#adv-attr-structuredAttrs
 2023-05-29 23:12:24 +02:00
Sergei Trofimovich
c3bdae6ce5
Merge pull request #223861 from eliasnaur/no-nix-lib64-in-self-rpath 
stdenv: don't set NIX_LIB*_IN_SELF_RPATH by default
 2023-05-17 07:08:57 +01:00
Robert Hensing
9524f57dd5
Merge pull request #230666 from alyssais/inputDerivation-passAsFile 
stdenv: fix inputDerivation with passAsFile
 2023-05-09 20:36:29 +02:00
Alyssa Ross
a295c4566e
stdenv: fix inputDerivation with passAsFile 
passAsFile passes the values of Nix bindings to the builder as
files, so if those values contained references, they wouldn't end up
in the inputDerivation output.  To fix that, append the contents of
every such passed file to the output.

We only have shell builtins in this derivation, so we can't use cat.
The only way I know of appending the contents of one file to another
using only shell builtins is as I've done here, but it requires
putting the contents of the file on echo's argv.  This might end up
causing problems with large files.  Regardless, I think we should try
this, as a failure is better than silently producing an incorrect
result like the previous behavior.
 2023-05-09 14:00:57 +00:00
Alyssa Ross
cd0bccbbe0
stdenv: fix inputDerivation with allowed refs 
Same motivation as for the disallowed references.
 2023-05-08 12:02:59 +00:00
Sergei Trofimovich
72d71dd641 stdenv: do not pass __contentAddressed = false as environment variable 
`nix-2.4+` automatically filters `__contentAddressed` out of the
environment. But not `nix-2.3`. This make `.drv` to differ between
unset and `__contentAddressed = false` derivations.

This change makes them equal by filtering out `__contentAddressed`
unless it's set to `true`.
 2023-05-02 22:57:54 +01:00
Emily Trau
2ad68a7d90 stdenv: factor out meta attr augmentation for reusability 
stdenv: check if separating commonMeta from assertValidity fixes laziness issue

```
error: cannot coerce null to a string

at /var/lib/ofborg/checkout/repo/38dca4e3aa6bca43ea96d2fcc04e8229/mr-est/ofborg-evaluator-3/pkgs/development/coq-modules/coq-lsp/default.nix:34:60:

    33|     homepage = "https://github.com/ejgallego/coq-lsp";
    34|     changelog = "https://github.com/ejgallego/coq-lsp/blob/${defaultVersion}/CHANGES.md";
```
 2023-05-01 20:18:11 +03:00
Sandro
b04d4bad27
Merge pull request #216992 from SuperSandro2000/stdenvNative-fix-eval 
{bintools,cc}-wrapper: don't fallback to version = null
 2023-04-14 11:22:20 +02:00
Sandro Jäckel
b0ee79f9fd
mkDerivation: show meaningful error when version is set to null 
instead of `Cannot coerce null to string`
 2023-04-12 22:08:36 +02:00
Elias Naur
88fd6601d8 stdenv: remove the NIX_LIB*_IN_SELF_RPATH environment variables 
The NIX_LIB64|32_IN_SELF_RPATH environment variables control whether
to add lib64 and lib32 to rpaths. However, they're set depending
on the build paltform, not the target platform and thus their values
are incorrect for for cross-builds.

On the other hand, setting them according to the build platform introduce
pointless differences in build outputs; see #221350 for details.

This change fixes the issues by boldly removes the NIX_LIB*_IN_SELF_RPATH
facility altogether, in the hope that it is no longer necessary. They
were introduced in 2009, long before nixpkgs had good support for
cross-builds.

Fixes #221350
 2023-03-31 10:47:16 -06:00
Vladimír Čunát
12dd95fbb1
Merge branch 'master' into staging-next 2023-03-24 09:07:41 +01:00
Adam Joseph
13507da345 check-meta.nix: fix self-contradictory error messages 
See https://github.com/NixOS/nixpkgs/pull/222792#pullrequestreview-1356114111

You can't just `lib.filter _ lib.systems.all` -- that throws away
important information, leading to nixpkgs disagreeing with itself
like this:

```
$ NIXPKGS_ALLOW_BROKEN=1 nix-instantiate . -A pkgsStatic.systemd
error: Package ‘systemd-252.5’ in ... is only supported on ... x86_64-linux but not on requested x86_64-linux, refusing to evaluate.
```

After:

```
$ NIXPKGS_ALLOW_BROKEN=1 nix-instantiate . -A pkgsStatic.systemd
error: Package ‘systemd-252.5’ in ... is not available on the requested hostPlatform:
         hostPlatform.config = "x86_64-unknown-linux-musl"
         package.meta.platforms = [
           "aarch64-linux"
           "armv5tel-linux"
           "armv6l-linux"
           "armv7a-linux"
           "armv7l-linux"
           "i686-linux"
           "m68k-linux"
           "microblaze-linux"
           "microblazeel-linux"
           "mipsel-linux"
           "mips64el-linux"
           "powerpc64-linux"
           "powerpc64le-linux"
           "riscv32-linux"
           "riscv64-linux"
           "s390-linux"
           "s390x-linux"
           "x86_64-linux"
         ]
         package.meta.badPlatforms = [
           {
             isStatic = true;
             parsed = { };
           }
         ]
       , refusing to evaluate.
```
 2023-03-24 00:15:57 -07:00
Weijia Wang
b46c97e1dc stdenv: generalise showPlatforms 2023-03-23 23:47:11 +02:00
github-actions[bot]
307b719414
Merge master into staging-next 2023-03-23 18:01:20 +00:00
Weijia Wang
2a7e1e0228 stdenv: fix error with patterned platforms 2023-03-23 17:27:49 +02:00
Sandro
979c6c1fe4
Merge pull request #213780 from SuperSandro2000/check-meta-platform 
stdenv: show supported and requested platforms when check meta fails
 2023-03-23 15:01:34 +01:00
github-actions[bot]
455127ad5e
Merge master into staging-next 2023-03-16 18:01:20 +00:00
Bernardo Meurer
6e55733359
Merge pull request #219747 from Stunkymonkey/deprecate-isNull 2023-03-16 11:10:22 -03:00
Martin Weinelt
19680e9902
Merge pull request #217568 from trofi/stdenv-parallel-install 
stdenv/generic/setup.sh: enable parallel installs for parallel builds
 2023-03-15 17:10:19 +00:00
Felix Buehler
d10e69c86b treewide: deprecate isNull 
https://nixos.org/manual/nix/stable/language/builtins.html#builtins-isNull
 2023-03-06 22:40:04 +01:00