nordic-dev.net/nixpkgs
nordic-dev.net/nixpkgs
2
0
Fork 0
mirror of https://github.com/NixOS/nixpkgs.git synced 2024-12-27 16:15:05 +00:00
Code Issues Packages Projects Releases Wiki Activity
235,807 Commits 186 Branches 125 Tags 9.3 GiB
20ec6b70ce
Commit Graph

766 Commits

Author SHA1 Message Date
Matthew Bauer
a52e317200 check-meta: use system tuple in platforms 
Fixes #60345
 2019-04-30 12:59:03 -04:00
Matthew Bauer
87944c3125
Merge pull request #56744 from matthewbauer/macos-10-12 
Update macOS to 10.12
 2019-04-26 22:20:03 -04:00
Matthew Bauer
ec7d72a57d setup.sh: make sure initialPath goes at end of HOST_PATH 
We want initialPath to have lowest precedence.

In addition, unset _PATH and _HOST_PATH as they shouldn’t be needed
after final PATH and HOST_PATH are set.
 2019-04-26 21:54:49 -04:00
Matthew Bauer
7488a367af
Merge pull request #56555 from matthewbauer/wasm 
Initial WebAssembly/WASI cross-compilation support
 2019-04-23 22:44:33 -04:00
John Ericson
1a7a96a093 stdenv, compiler-rt: Compress WASI conditionals 2019-04-23 21:48:58 -04:00
Matthew Bauer
9abff4af4f wasm: init cross target 
Adds pkgsCross.wasm32 and pkgsCross.wasm64. Use it to build Nixpkgs
with a WebAssembly toolchain.

stdenv/cross: use static overlay on isWasm

isWasm doesn’t make sense dynamically linked.
 2019-04-23 21:48:57 -04:00
Matthew Bauer
7f23d9fc5f
Merge pull request #59755 from matthewbauer/patches-in-make-derivation 
make-derivation: put patches in all derivations
 2019-04-23 18:33:04 -04:00
volth
b0193379e5
make-derivation.nix: minor 
it seems to change nothing (.platform is always there), just to be consisted with the style of other checks
 2019-04-21 20:05:13 +00:00
volth
b1c3581b46
make-derivation.nix: @matthewbauer's review 2019-04-21 19:16:28 +00:00
volth
5d87bc2650 fix bootstrap when platform.gcc.arch=="skylake" 2019-04-21 16:37:49 +00:00
Matthew Bauer
c7ccb9f197 make-derivation: put patches in all derivations 
This puts patches in all derivations even if it unspecified by the
derivation. By default it will be an empty list. This simplifies
overrides, as we can now assume that patches is a valid name so that
this works:

self: super: {
  mypkg = super.pkg.overrideAttrs (o: {
    patches = o.patches ++ [ ./my-very-own.patch ];
  });
}

That is, you don’t need to provide a default "or []", make-derivation
provides one for you.

Unfortunately, this is a mass rebuild.
 2019-04-16 22:51:26 -04:00
Graham Christensen
817c933878
check-env: don't execute check-meta.nix 15,000 times 
Generated from https://github.com/NixOS/nix/pull/2761:

```
                                                                                          ns     calls ns/call
- /home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:22:5 591200 15026 39.3451
+ /home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:22:5 8744   308   28.3896
```

more, generated by:

```
$ NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 nix-instantiate ./pkgs/top-level/release.nix -A unstable > before 2>&1
$ jq -r '.functions | map((.name + ":" + .file + ":" + (.line|tostring) + ":" + (.column|tostring) + " " + (.count|tostring))) | .[]' before | sort  > before.list
```

applying this patch, then:

```
$ NIX_SHOW_STATS=1 NIX_COUNT_CALLS=1 nix-instantiate ./pkgs/top-level/release.nix -A unstable > after 2>&1
$ jq -r '.functions | map((.name + ":" + .file + ":" + (.line|tostring) + ":" + (.column|tostring) + " " + (.count|tostring))) | .[]' after | sort  > after.list
```

and then diffing before.list and after.list to get:

```
                                                                                                        calls
- :/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:4:1               7513
+ :/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:4:1               154

- mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:23                7513
+ mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:23                154

- mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:26                7513
+ mutuallyExclusive:/home/grahamc/projects/github.com/NixOS/nixpkgs/lib/lists.nix:658:26                154

- onlyLicenses:/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:21:18 15026
+ onlyLicenses:/home/grahamc/projects/github.com/NixOS/nixpkgs/pkgs/stdenv/generic/check-meta.nix:21:18 308
```

The following information is from `NIX_SHOW_STATS=1 GC_INITIAL_HEAP_SIZE=4g nix-env -f ./outpaths.nix -qaP --no-name --out-path --arg checkMeta true`:

| stat                       | before         | after          | Δ               | Δ%      |
|:---------------------------|---------------:|---------------:|:----------------|--------:|
| **cpuTime**                |        179.915 |        145.543 | 🡖 34.372        | -19.10% |
| **envs-bytes**             |  3,900,878,824 |  3,599,483,208 | 🡖 301,395,616   |  -7.73% |
| **envs-elements**          |    214,426,071 |    185,881,709 | 🡖 28,544,362    | -13.31% |
| **envs-number**            |    136,591,891 |    132,026,846 | 🡖 4,565,045     |  -3.34% |
| **gc-heapSize**            | 11,400,048,640 | 12,314,890,240 | 🡕 914,841,600   |   8.02% |
| **gc-totalBytes**          | 25,976,902,560 | 24,510,740,176 | 🡖 1,466,162,384 |  -5.64% |
| **list-bytes**             |  1,665,290,080 |  1,665,290,080 | 0               |         |
| **list-concats**           |      7,264,417 |      7,264,417 | 0               |         |
| **list-elements**          |    208,161,260 |    208,161,260 | 0               |         |
| **nrAvoided**              |    191,359,386 |    179,693,661 | 🡖 11,665,725    |  -6.10% |
| **nrFunctionCalls**        |    119,665,062 |    116,348,547 | 🡖 3,316,515     |  -2.77% |
| **nrLookups**              |     80,996,257 |     76,069,825 | 🡖 4,926,432     |  -6.08% |
| **nrOpUpdateValuesCopied** |    213,930,649 |    213,930,649 | 0               |         |
| **nrOpUpdates**            |     12,025,937 |     12,025,937 | 0               |         |
| **nrPrimOpCalls**          |     88,105,604 |     86,451,598 | 🡖 1,654,006     |  -1.88% |
| **nrThunks**               |    196,842,044 |    175,126,701 | 🡖 21,715,343    | -11.03% |
| **sets-bytes**             |  7,678,425,776 |  7,285,767,928 | 🡖 392,657,848   |  -5.11% |
| **sets-elements**          |    310,241,340 |    294,373,227 | 🡖 15,868,113    |  -5.11% |
| **sets-number**            |     29,079,202 |     27,601,310 | 🡖 1,477,892     |  -5.08% |
| **sizes-Attr**             |             24 |             24 | 0               |         |
| **sizes-Bindings**         |              8 |              8 | 0               |         |
| **sizes-Env**              |             16 |             16 | 0               |         |
| **sizes-Value**            |             24 |             24 | 0               |         |
| **symbols-bytes**          |     16,474,666 |     16,474,676 | 🡕 10            |   0.00% |
| **symbols-number**         |        376,426 |        376,427 | 🡕 1             |   0.00% |
| **values-bytes**           |  6,856,506,288 |  6,316,585,560 | 🡖 539,920,728   |  -7.87% |
| **values-number**          |    285,687,762 |    263,191,065 | 🡖 22,496,697    |  -7.87% |

The following information is from `NIX_SHOW_STATS=1 GC_INITIAL_HEAP_SIZE=4g nix-instantiate ./nixos/release-combined.nix -A tested`:

| stat                       | before         | after          | Δ               | Δ%     |
|:---------------------------|---------------:|---------------:|:----------------|-------:|
| **cpuTime**                |        256.071 |        237.531 | 🡖 18.54         | -7.24% |
| **envs-bytes**             |  7,111,004,192 |  7,041,478,520 | 🡖 69,525,672    | -0.98% |
| **envs-elements**          |    346,236,940 |    339,588,487 | 🡖 6,648,453     | -1.92% |
| **envs-number**            |    271,319,292 |    270,298,164 | 🡖 1,021,128     | -0.38% |
| **gc-heapSize**            |  8,995,291,136 | 10,110,009,344 | 🡕 1,114,718,208 | 12.39% |
| **gc-totalBytes**          | 37,172,737,408 | 36,878,391,888 | 🡖 294,345,520   | -0.79% |
| **list-bytes**             |  1,886,162,656 |  1,886,163,472 | 🡕 816           |  0.00% |
| **list-concats**           |      6,898,114 |      6,898,114 | 0               |        |
| **list-elements**          |    235,770,332 |    235,770,434 | 🡕 102           |  0.00% |
| **nrAvoided**              |    328,829,821 |    326,618,157 | 🡖 2,211,664     | -0.67% |
| **nrFunctionCalls**        |    240,850,845 |    239,998,495 | 🡖 852,350       | -0.35% |
| **nrLookups**              |    144,849,632 |    142,126,339 | 🡖 2,723,293     | -1.88% |
| **nrOpUpdateValuesCopied** |    251,032,504 |    251,032,504 | 0               |        |
| **nrOpUpdates**            |     17,903,110 |     17,903,110 | 0               |        |
| **nrPrimOpCalls**          |    140,674,913 |    139,485,975 | 🡖 1,188,938     | -0.85% |
| **nrThunks**               |    294,643,131 |    288,678,022 | 🡖 5,965,109     | -2.02% |
| **sets-bytes**             |  9,464,322,192 |  9,456,172,048 | 🡖 8,150,144     | -0.09% |
| **sets-elements**          |    377,474,889 |    377,134,877 | 🡖 340,012       | -0.09% |
| **sets-number**            |     50,615,607 |     50,616,875 | 🡕 1,268         |  0.00% |
| **sizes-Attr**             |             24 |             24 | 0               |        |
| **sizes-Bindings**         |              8 |              8 | 0               |        |
| **sizes-Env**              |             16 |             16 | 0               |        |
| **sizes-Value**            |             24 |             24 | 0               |        |
| **symbols-bytes**          |      3,147,102 |      3,147,064 | 🡖 38            | -0.00% |
| **symbols-number**         |         82,819 |         82,819 | 0               |        |
| **values-bytes**           | 11,147,448,768 | 10,996,111,512 | 🡖 151,337,256   | -1.36% |
| **values-number**          |    464,477,032 |    458,171,313 | 🡖 6,305,719     | -1.36% |
 2019-04-11 19:21:38 -04:00
Frederik Rietdijk
2fcb11a244 Merge staging-next into master 2019-03-01 09:06:20 +01:00
Matthew Bauer
b86e62d30d llvm: support cross compilation with useLLVM flag 
You can build (partially) with LLVM toolchain using the useLLVM flag.
This works like so:

  nix-build -A hello --arg crossSystem '{ system =
    "aarch64-unknown-linux-musl"; useLLVM = true }'

also don’t separate debug info in lldClang

It doesn’t work currently with that setup hook. Missing build-id?
 2019-02-26 19:45:35 -05:00
Daiderd Jordan
7ec53a932f
stdenv: only set __darwinAllowLocalNetworking on darwin 
This is a darwin only nix attribute for sandbox builds, it can be
ignored on other platforms to avoid unnecessary rebuilds.
 2019-02-14 21:38:08 +01:00
Matthew Bauer
5c09d977c7 Merge remote-tracking branch 'origin/master' into staging 2019-02-09 12:14:06 -05:00
Matthew Bauer
c6f8f8d98d make-derivation: only modify name when name is given 
This preserves Nix’s native error handling of missing name:

  error: derivation name missing
 2019-02-04 09:15:03 -05:00
Vincent Laporte
0d17ecce2c
mkDerivation: cleaner handling of the name argument 2019-02-01 16:02:42 +00:00
Vladimír Čunát
8ba516664b
Merge branch 'staging-next' into staging 2019-02-01 09:42:53 +01:00
Vladimír Čunát
5effa4e0f9
Merge branch 'master' into staging-next 
Comments on conflicts:
- llvm: d6f401e1 vs. 469ecc70 - docs for 6 and 7 say the default is
  to build all targets, so we should be fine
- some pypi hashes: they were equivalent, just base16 vs. base32
 2019-02-01 09:22:29 +01:00
Matthew Bauer
8020bd6869
Merge pull request #35884 from dtzWill/fix/man-in-outputsToInstall 
default to including "man" in outputsToInstall
 2019-01-31 14:55:12 -05:00
Matthew Bauer
e2fe4c2d49 make-derivation: fix ordering of conditionals 
cross should have higher precedence
 2019-01-28 11:38:30 -05:00
Matthew Bauer
9fd1c170cc make-derivation: try to fix stdenv.cc == null conditional 
sometimes this gets an infinite recursion error
 2019-01-28 10:41:19 -05:00
Matthew Bauer
7e589e5594 make-derivation: fix position in trace 
For a long time now, tracing has been broken in Nixpkgs. So when you
have an eval error you would get something like this:

  error: while evaluating the attribute 'buildInputs' of the derivation 'hello-2.10' at /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:185:11:
  while evaluating 'chooseDevOutputs' at /home/mbauer/nixpkgs/lib/attrsets.nix:474:22, called from undefined position:
  while evaluating 'optionals' at /home/mbauer/nixpkgs/lib/lists.nix:257:5, called from /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:132:17:

This is coming from how Nix handles string context and how
make-derivation messes with the "name" attribute. This commit should
restore the old behavior so you get a nice line number like:

  error: while evaluating the attribute 'buildInputs' of the derivation 'hello-2.10' at /home/mbauer/nixpkgs/pkgs/applications/misc/hello/default.nix:4:3:
  while evaluating 'chooseDevOutputs' at /home/mbauer/nixpkgs/lib/attrsets.nix:474:22, called from undefined position:
  while evaluating 'optionals' at /home/mbauer/nixpkgs/lib/lists.nix:257:5, called from /home/mbauer/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:132:17:

NOTE: This will still be broken for cross compilation due to the
prefixes we are adding to name.
 2019-01-28 09:25:47 -05:00
Matthew Bauer
d54fbbebbb
Merge pull request #49552 from matthewbauer/setup-dedupe 
setup.sh: avoid running the same hook twice
 2019-01-27 18:22:41 -05:00
Matthew Bauer
9172c1eee2 setup.sh: avoid running the same hook twice 
In strictDeps=false, we don’t want the same package hook to be run
twice, otherwise we get duplicates in some flags.

Fixes #41340
 2019-01-27 17:49:13 -05:00
Matthew Bauer
ae16dd1a15
stdenv/make-derivation: don't hide broken packages 
This behavior ended up breaking the handleEvalIssue functionality by hiding those packages. So something like this:

$ nix-env -iA nixpkgs.zoom-us

would silently fail, without telling the user how to fix it! Regardless, this "bug" should be handled in Nix - not Nixpkgs.

Fixes #38952.
 2019-01-27 17:02:17 -05:00
Matthew Bauer
bd62ac6b88
Merge pull request #54652 from matthewbauer/shell-in-flags-array 
setup.sh: put SHELL in flagsArray
 2019-01-27 13:55:54 -05:00
Matthew Bauer
329913f733 setup.sh: put SHELL in flagsArray 
We don’t want to modify makeFlags, that is given to us by our
environment. Adding to it could lead to duplicates after repeated use.

Fixes #27533
 2019-01-26 21:14:36 -05:00
Matthew Bauer
8babcc3d44
Merge branch 'master' into fix/man-in-outputsToInstall 2019-01-26 10:51:12 -05:00
Frederik Rietdijk
a1a5ea5943 stdenv: make checkInputs native 
We can't run the checkPhase when build != host, so we may as well make
the checkInputs native.

This signicantly improves the situation of Python packages when enabling
strictDeps.
 2019-01-13 14:43:18 +01:00
Orivej Desh (NixOS)
9a21967f0a
stdenv: prune libtool files by default (#51767) 
See the motivation in fd97db43bc (#41819).
 2019-01-11 13:20:46 +00:00
Daniel Goertzen
1c10efc912 add generic x86_32 support (#52634) 
* add generic x86_32 support

- Add support for i386-i586.
- Add `isx86_32` predicate that can replace most uses of `isi686`.
- `isi686` is reinterpreted to mean "exactly i686 arch, and not say i585 or i386".
- This branch was used to build working i586 kernel running on i586 hardware.

* revert `isi[345]86`, remove dead code

- Remove changes to dead code in `doubles.nix` and `for-meta.nix`.
- Remove `isi[345]86` predicates since other cpu families don't have specific model predicates.

* remove i386-linux since linux not supported on that cpu
 2019-01-06 12:57:36 -06:00
Jörg Thalheim
1b146a8c6f
treewide: remove paxutils from stdenv 
More then one year ago we removed grsecurity kernels from nixpkgs:
https://github.com/NixOS/nixpkgs/pull/25277

This removes now also paxutils from stdenv.
 2018-12-22 12:55:05 +01:00
Graham Christensen
a375d4bfc3
stdenv: shorten evaluation errors when in Hydra 
Hydra's page showing evaluation errors is about a mile long, showing
buckets of user-friendly errors, like this:

    in job ‘seyren.aarch64-linux’:
    Package ‘oraclejre-8u191’ in /nix/store/fa9zzkbljkvdavwzirkrr5irg25ymbjl-source/pkgs/development/compilers/oraclejdk/jdk-linux-base.nix:71 has an unfree license (‘unfree’), refusing to evaluate.

    a) For `nixos-rebuild` you can set
      { nixpkgs.config.allowUnfree = true; }
    in configuration.nix to override this.

    b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
      { allowUnfree = true; }
    to ~/.config/nixpkgs/config.nix.

    in job ‘jetbrains.webstorm.x86_64-linux’:
    Package ‘webstorm-2018.3.1’ in /nix/store/fa9zzkbljkvdavwzirkrr5irg25ymbjl-source/pkgs/applications/editors/jetbrains/default.nix:230 has an unfree license (‘unfree’), refusing to evaluate.

    a) For `nixos-rebuild` you can set
      { nixpkgs.config.allowUnfree = true; }
    in configuration.nix to override this.

    b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
      { allowUnfree = true; }
    to ~/.config/nixpkgs/config.nix.

This makes it extremely hard to find actual issues in the output. This
patch set makes the output much more condensed in Hydra:

    Failed to evaluate nifticlib-2.0.0: «unsupported»: is not supported on ‘x86_64-apple-darwin’
    Failed to evaluate dmd-2.081.2: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate dmdBuild-2.081.2: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate ldc-1.11.0: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate ldcBuild-1.11.0: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate ldc-0.17.5: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
    Failed to evaluate ldcBuild-0.17.5: «unsupported»: is not supported on ‘aarch64-unknown-linux-gnu’
 2018-12-18 10:50:53 -05:00
Piotr Bogdan
44c9c27d54 stdenv: prune libtool files by default 2018-12-09 22:45:15 +00:00
Jan Tojnar
c5881ec2c9
Merge remote-tracking branch 'upstream/master' into staging 2018-11-30 20:09:45 +01:00
Matthew Bauer
28e2277305 make-derivation: remove selfConsistent check 
version is set in lots of places but might not need to be in a name.

Alternative to #50364.
 2018-11-30 17:58:33 +01:00
Frederik Rietdijk
1828a5c5ba Merge master into staging-next 2018-11-30 17:46:21 +01:00
Matthew Bauer
e6834171b9
Merge pull request #51183 from matthewbauer/make-derivation-no-assertions 
Fix breaking changes to make-derivation
 2018-11-29 19:35:55 -06:00
Vaibhav Sagar
b5504e26b0 pkgs/stdenv/generic/make-derivation.nix: s/targetPlatform/hostPlatform/ 2018-11-29 17:54:59 -05:00
Matthew Bauer
a56fe056ec make-derivation: don’t add host suffix if there is no c compiler 
Some trivial builders use the name attr to choose the exec name
produced. For example nixos-install,

  nixos-install = makeProg {
    name = "nixos-install";
    src = ./nixos-install.sh;
    nix = config.nix.package.out;
    path = makeBinPath [ nixos-enter ];
  };

When cross compiling, this puts the prog in,

  /bin/nixos-install-powerpc64le-unknown-linux-gnu
 2018-11-28 12:44:10 -06:00
Matthew Bauer
d64f288154 make-derivation: don’t assert on separateDebugInfo 
When separateDebugInfo = true & !stdenv.hostPlatform.isLinux, we
always gave an error. There is no reason to do this. Instead, just
don’t add separate debug info when we aren’t on Linux.
 2018-11-28 12:29:00 -06:00
Matthew Bauer
13e45aeed3 make-derivation: remove selfConsistent check 
version is set in lots of places but might not need to be in a name.

Alternative to #50364.
 2018-11-28 12:26:21 -06:00
Vladimír Čunát
a5de78b7d7
Merge branch 'master' into staging-next 2018-11-26 10:28:00 +01:00
Matthew Bauer
7eeb02d47b Revert "make-derivation: add disallowedReferences in strictDeps" 
This reverts commit 8dbfb61e46.
Also reverts commit fc99c337ed.

Fixes #50915
 2018-11-25 15:53:28 -06:00
Frederik Rietdijk
c31cb577ae Merge master into staging-next 2018-11-22 09:57:08 +01:00
Matthew Bauer
fc99c337ed make-derivation: don’t disallow propagated native inputs 
propagateNativeBuildInputs will end up going in the output derivation.
This case is allowed to end up in references because of that. Sorry
for the disruption!

Fixes #50865
 2018-11-21 09:39:47 -06:00
Daiderd Jordan
c9223a17bc
Revert "patch-shebangs: use --build for auto patch shebangs" 
Completely breaks darwin. Every package in the stdenv that has shebangs
in the output will end up with references to bootstrap-tools.

This reverts commit eb7c50a993.
 2018-11-14 23:37:31 +01:00
Matthew Bauer
f19bb8321a
Merge pull request #35304 from volth/patch-97 
[staging] substitute() print warning if does nothing
 2018-11-12 12:52:51 -06:00
Frederik Rietdijk
3b052406ea Merge staging-next into staging 2018-11-12 19:01:36 +01:00
Frederik Rietdijk
daf3297cb4 Merge master into staging-next 2018-11-12 18:59:08 +01:00
Léo Gaspard
2986ce16a8
meta.tests: rename into passthru.tests 
Nix currently rejects derivations in `meta` values. This works around
that limitation by using `passthru` instead.

Closes https://github.com/NixOS/nixpkgs/issues/50230
 2018-11-11 23:11:46 +09:00
Léo Gaspard
cae5598611
meta.tests: drop meta.needsVMSupport 
Its job is already handled by `requiredSystemFeatures`
 2018-11-11 23:11:46 +09:00
Léo Gaspard
83b27f60ce
tests: split into a separate all-tests.nix file 
This will make the list much easier to re-use, eg. for `nixosTests`

The drawback is that this approaches makes the
```
nix-build release.nix -A tests.opensmtpd.x86_64-linux
```
command about twice as slow (3s to 6s): it now has to evaluate `nixpkgs`
once for each architecture, instead of just having the hardcoded list of
tests that allowed to say “ok just evaluate for x86_64-linux”.

On the other hand, complete evaluation of `release.nix` should be much
faster because we no longer import `nixpkgs` for each test: testing with
the following command went from 30s to 18s, and that's just for a few
tests.
```
time nix-instantiate --eval --strict nixos/release.nix -A tests.nat
```
I initially wanted to test on the whole `release.nix`, but there are too
many broken tests and it takes too long to eval them all, especially
compared to the fact that the current implementation breaks some setup.

Given developers can just `nix-build nixos/tests/my-test.nix`, it sounds
like an overall win.
 2018-11-11 23:11:46 +09:00
Frederik Rietdijk
1d3bff25db Merge staging-next into staging 2018-11-11 14:28:08 +01:00
Frederik Rietdijk
53d00c3351 Merge master into staging-next 2018-11-10 11:08:54 +01:00
Matthew Bauer
c8aff96110
Merge pull request #49608 from matthewbauer/cross-patch-shebangs-2 
Restore cross-patch-shebangs branch
 2018-11-07 13:37:02 -06:00
Matthew Bauer
6d531f3541 make-derivation: enable pie hardening with musl 
Fixes #49071

On ld.gold, we produce broken executables when linking with the Musl
libc. This appears to be a known bug when using ld.gold and Musl. This
thread describes the workaround as enabling PIE when using ld.gold and
Musl:

https://www.openwall.com/lists/musl/2015/05/01/5

By default we don’t enable PIE to avoid breaking things. But in the
Musl case we are breaking things by not enabling PIE. So this adds a
special case for defaultHardeningFlags which keeps the pie hardening
for everything. Any packages that break with PIE can add the pie flag
to disableHardeningFlags array (a no-op for now on anything but Musl).
 2018-11-07 15:24:51 +01:00
Patrick Hilhorst
320c9c10de
make-derivation: use pname-version as default name if both are present 2018-11-06 00:04:21 +01:00
Frederik Rietdijk
322f87137c Merge master into staging-next 2018-11-04 11:33:17 +01:00
Frederik Rietdijk
abea6f461a Revert "Merge pull request #49398 from Synthetica9/implement-rfc0035" to fix eval 
This reverts commit 3fc7d5eb83, reversing
changes made to 1fddf2b689.

The idea is good, however, before enforcing, make sure all occurences
are fixed.
 2018-11-04 11:02:43 +01:00
Frederik Rietdijk
cb4ff927a1 Merge master into staging-next 2018-11-04 08:49:24 +01:00
Matthew Bauer
8dbfb61e46 make-derivation: add disallowedReferences in strictDeps 
When strictDeps = true, we don’t want native build inputs to end up in
the output. For instance gcc is a builtin native build input and
should only show up in an output if it is also listed in buildInputs.

/cc @ericson2314
 2018-11-02 19:31:51 -05:00
Matthew Bauer
eb7c50a993 patch-shebangs: use --build for auto patch shebangs 
In strictDeps=false, autoPatchshebangs should use
--build (corresponding to PATH) to lookup commands. This restores the
previous behavior of patchshebangs so that we don’t break stuff that
isn’t careful in the buildInputs vs. nativeBuildInputs distinction.
Unfortunately this won’t work under cross compilation.
 2018-11-02 00:27:14 -05:00
Patrick Hilhorst
1f7fc09176
make-derivation: use a more descriptive assert message 
As suggested by @Profpatsch
 2018-10-30 14:33:14 +01:00
Léo Gaspard
02e1f00ffd
dovecot, opensmtpd: add link to test in meta.tests 
Rationale
---------

Currently, tests are hard to discover. For instance, someone updating
`dovecot` might not notice that the interaction of `dovecot` with
`opensmtpd` is handled in the `opensmtpd.nix` test.

And even for someone updating `opensmtpd`, it requires manual work to go
check in `nixos/tests` whether there is actually a test, especially
given not so many packages in `nixpkgs` have tests and this is thus most
of the time useless.

Finally, for the reviewer, it is much easier to check that the “Tested
via one or more NixOS test(s)” has been checked if the file modified
already includes the list of relevant tests.

Implementation
--------------

Currently, this commit only adds the metadata in the package. Each
element of the `meta.tests` attribute is a derivation that, when it
builds successfully, means the test has passed (ie. following the same
convention as NixOS tests).

Future Work
-----------

In the future, the tools could be made aware of this `meta.tests`
attribute, and for instance a `--with-tests` could be added to
`nix-build` so that it also builds all the tests. Or a `--without-tests`
to build without all the tests. @Profpatsch described in his NixCon talk
such systems.

Another thing that would help in the future would be the possibility to
reasonably easily have cross-derivation nix tests without the whole
NixOS VM stack. @7c6f434c already proposed such a system.

This RFC currently handles none of these concerns. Only the addition of
`meta.tests` as metadata to be used by maintainers to remember to run
relevant tests.
 2018-10-30 21:31:39 +09:00
Patrick Hilhorst
c7e026bec4
make-derivation: use lib.assertMsg 
As suggested by @Profpatsch
 2018-10-29 18:25:59 +01:00
Patrick Hilhorst
5be927db14
make-derivation: use ? instead of builtins.hasAttr 
As suggested by @edolstra
 2018-10-29 15:17:13 +01:00
Patrick Hilhorst
2962f94fec
make-derivation: add check that the name is consistent with pname and version 2018-10-29 14:58:12 +01:00
Patrick Hilhorst
149a55eca7
make-derivation: get position info from version 2018-10-29 14:51:22 +01:00
Patrick Hilhorst
efca8b4b97
make-derivation: use pname-version as default name if both are present 2018-10-29 14:51:08 +01:00
Matthew Bauer
b3041b4455 make-derivation: set CMAKE_SYSTEM_* when cross compiling 
Uses uname data to find what to set these variables:

- CMAKE_SYSTEM_NAME
- CMAKE_SYSTEM_PROCESSOR
- CMAKE_SYSTEM_VERSION
- CMAKE_HOST_SYSTEM_NAME
- CMAKE_HOST_SYSTEM_PROCESSOR
- CMAKE_HOST_SYSTEM_VERSION
 2018-10-16 21:50:37 -05:00
Matthew Bauer
93834fe194
Merge pull request #47230 from bhipple/fix/licenses 
Remove dead code from stdenv check-meta license logic
 2018-10-05 22:58:27 -05:00
aszlig
b25b6e0c75
stdenv: Improve ELF detection for isELF 
The isELF function only checks whether ELF is contained within the first
4 bytes of the file, which is a bit fuzzy and will also return
successful if it's a text file starting with ELF, for example:

  ELF headers
  -----------

  Some text here about ELF headers...

So instead, we're now doing a precise match on \x7fELF.

Signed-off-by: aszlig <aszlig@nix.build>
Acked-by: @Ericson2314
Closes: https://github.com/NixOS/nixpkgs/pull/47244
 2018-09-25 06:55:18 +02:00
Benjamin Hipple
0b9d9ab256 Remove dead code from stdenv check-meta license logic 
The `unfree` and `unfreeRedistributable` licenses both have `free = false`,
which will trigger the first portion of logic. This removes dead code to
simplify the logic.

As a follow-up, I plan to add an attribute `redistributable = [true|false]`,
which can be used by Hydra to determine whether a given package with a given
license can be included in the channel.
 2018-09-23 12:48:02 -04:00
John Ericson
7319013ea1 Merge remote-tracking branch 'upstream/master' into staging 2018-09-18 16:55:42 -04:00
John Ericson
35378f0141
Merge pull request #46857 from obsidiansystems/darwin-to-linux-prep 
misc pkgs: various cross fixes in preparation for darwin->linux
 2018-09-18 16:52:30 -04:00
John Ericson
2111e7b742 mkDerivation: Make separateDebugInfo assertion lazier to match other assertions 
This is needed to access attributes of derivations on platforms where
they cannot be built.
 2018-09-18 16:25:19 -04:00
Graham Christensen
b80c9ce4a9
stdenv: Validate meta.outputsToInstall 
If meta.outputsToInstall is set to include absent outputs, various
tools break including channel updates and nix-env.

    grahamc@Morbo> nix-env -i -f . -A elf-header-real
    installing 'elf-header'
    error: this derivation has bad 'meta.outputsToInstall'

This patch verifies each value in meta.outputsToInstall is a valid
output. It validates this condition only if checkMeta is true.

    grahamc@Morbo> nix-build . -A elf-header-real
    error: Package ‘elf-header’ in /home/grahamc/projects/nixpkgs/pkgs/development/libraries/elf-header/default.nix:36 has invalid meta.outputsToInstall, refusing to evaluate.

    The package elf-header has set meta.outputsToInstall to: bin

    however elf-header only has the outputs: out

    and is missing the following ouputs:

      - bin

    (use '--show-trace' to show detailed location information)

Note, now the nix-env experience is decidedly worse for users who have
checkMeta set to true:

    grahamc@Morbo> nix-env -i -f . -A elf-header-real; echo $?
    0

though since this is already an issue for unfree, broken, unsupported,
and insecure validity problems I'm not sure we should do something
different here.
 2018-09-18 10:38:44 -04:00
Matthew Bauer
ba5717a6f5 stdenv: fix HOST_PATH change 
a4630c65ca was incorrect in assuming $SHELL would be a path to the
bash derivation. In fact $SHELL will be a path to the bash executable.

Unfortunately this did not fix the original issue. So instead, we just
have to reuse initialPath can be added like PATH is.

Sorry for the inconvenience! I hadn’t thought through the effects of
the last commit.

/cc @copumpkin @ericson2314
 2018-09-17 14:18:06 -05:00
Matthew Bauer
a4630c65ca stdenv: add shell to HOST_PATH for backwards compatibility 
To avoid breaking things, we need to make sure SHELL goes into
HOST_PATH. This reflects my changes to patch-shebangs to make it cross
compilation ready. When a script is patched from the Nix store it now
looks to HOST_PATH to get the targeted machine’s executables.
Unfortunately, this only works in native builds.
 2018-09-16 15:58:21 -05:00
Jan Malakhovski
b7bd0561be Merge branch 'master' into staging 2018-09-08 22:08:32 +00:00
John Ericson
773233ca77 top-level, stdenv: Make system and stdenv.system describe the hostPlatform. 
Intuitively, one cares mainly about the host platform: Platforms differ
in meaningful ways but compilation is morally a pure process and
probably doesn't care, or those difference are already abstracted away.
@Dezgeg also empirically confirmed that > 95% of checks are indeed of
the host platform.

Yet these attributes in the old cross infrastructure were defined to be
the build platform, for expediency. And this was never before changed.
(For native builds build and host coincide, so it isn't clear what the
intention was.)

Fixing this doesn't affect native builds, since again they coincide. It
also doesn't affect cross builds of anything in Nixpkgs, as these are no
longer used. It could affect external cross builds, but I deem that
unlikely as anyone thinking about cross would use more explicit
attributes for clarity, all the more so because the rarity of inspecting
the build platform.
 2018-09-06 08:33:51 -04:00
John Ericson
51907d257c stdenv, neovim: Use lib.warn for deprecation warnings 2018-09-05 11:40:29 -04:00
Jan Malakhovski
c63ca0a431 stdenv: implement enableParallelChecking option 
Works similarly to `enableParallelBuilding`, but is set by default when
`enableParallelBuilding` is set. In my experience most packages that build
fine in parallel also check fine in parallel.
 2018-09-05 01:14:39 +00:00
John Ericson
06cd7c15a3 mkDerivation: Fix cross compilation 
Derivations where drawing their `system` attribute from `hostPlatform`
instead of `buildPlatform`. Fix that, and add an explanatory commment.

Fixes #45993
 2018-09-03 23:18:24 -04:00
John Ericson
82110ae656 stdenv: Better message for deprecated isArm 
The message should say what to do instead.
 2018-08-31 16:01:58 -04:00
John Ericson
2c2f1e37d4 reewide: Purge all uses stdenv.system and top-level system 
It is deprecated and will be removed after 18.09.
 2018-08-30 17:20:32 -04:00
Jörg Thalheim
9efffe0135 hurd: cleanup unmaintained target 
This has been not touched in 6 years. Let's remove it to cause less
problems when adding new cross-compiling infrastructure.
This also simplify gcc significantly.
 2018-08-28 22:18:02 +01:00
Jan Malakhovski
eb5b313b16 stdenv: shadow outputs 2018-08-05 19:24:46 +00:00
Jan Malakhovski
a53504f4a4 stdenv: shadow doCheck and doInstallCheck 2018-08-05 19:24:32 +00:00
John Ericson
df5d1673ea
Merge pull request #44367 from obsidiansystems/no-stdenv-is-cross 
treewide: Remove stdenv.isCross
 2018-08-02 16:36:01 -04:00
John Ericson
34da7e2ce2 treewide: Remove stdenv.isCross 
I *want* cross-specific overrides to be verbose, so I rather not have
this shorthand. This makes the syntactic overhead more proportional to
the maintainence cost. Hopefully this pushes people towards fewer
conditionals and more abstractions.
 2018-08-02 15:01:58 -04:00
James Deikun
bd63de114e stdenv/build-support: support .tbz and .txz tarballs 2018-07-30 15:30:16 -04:00
Frederik Rietdijk
f59e6f4037 Merge master into staging 2018-07-26 16:18:53 +02:00
John Ericson
87b4a5d6a7 stdenv: Remove crossAttrs, now that it is no longer used 
Fix #33302
 2018-07-24 20:01:43 -04:00
Frederik Rietdijk
099c13da1b Merge staging-next into master (#44009) 
* substitute(): --subst-var was silently coercing to "" if the variable does not exist.

* libffi: simplify using `checkInputs`

* pythonPackges.hypothesis, pythonPackages.pytest: simpify dependency cycle fix

* utillinux: 2.32 -> 2.32.1

https://lkml.org/lkml/2018/7/16/532

* busybox: 1.29.0 -> 1.29.1

* bind: 9.12.1-P2 -> 9.12.2

https://ftp.isc.org/isc/bind9/9.12.2/RELEASE-NOTES-bind-9.12.2.html

* curl: 7.60.0 -> 7.61.0

* gvfs: make tests run, but disable

* ilmbase: disable tests on i686. Spooky!

* mdds: fix tests

* git: disable checks as tests are run in installcheck

* ruby: disable tests

* libcommuni: disable checks as tests are run in installcheck

* librdf: make tests run, but disable

* neon, neon_0_29: make tests run, but disable

* pciutils: 3.6.0 -> 3.6.1

Semi-automatic update generated by https://github.com/ryantm/nixpkgs-update tools. This update was made based on information from https://repology.org/metapackage/pciutils/versions.

* mesa: more include fixes

mostly from void-linux (thanks!)

* npth: 1.5 -> 1.6

minor bump

* boost167: Add lockfree next_prior patch

* stdenv: cleanup darwin bootstrapping

Also gets rid of the full python and some of it's dependencies in the
stdenv build closure.

* Revert "pciutils: use standardized equivalent for canonicalize_file_name"

This reverts commit f8db20fb3a.
Patching should no longer be needed with 3.6.1.

* binutils-wrapper: Try to avoid adding unnecessary -L flags

(cherry picked from commit f3758258b8895508475caf83e92bfb236a27ceb9)
Signed-off-by: Domen Kožar <domen@dev.si>

* libffi: don't check on darwin

libffi usages in stdenv broken darwin. We need to disable doCheck for that case.

* "rm $out/share/icons/hicolor/icon-theme.cache" -> hicolor-icon-theme setup-hook

* python.pkgs.pytest: setupHook to prevent creation of .pytest-cache folder, fixes #40273

When `py.test` was run with a folder as argument, it would not only
search for tests in that folder, but also create a .pytest-cache folder.
Not only is this state we don't want, but it was also causing
collisions.

* parity-ui: fix after merge

* python.pkgs.pytest-flake8: disable test, fix build

* Revert "meson: 0.46.1 -> 0.47.0"

With meson 0.47.0 (or 0.47.1, or git)
things are very wrong re:rpath handling
resulting in at best missing libs but
even corrupt binaries :(.

When we run patchelf it masks the problem
by removing obviously busted paths.
Which is probably why this wasn't noticed immediately.

Unfortunately the binary already
has a long series of paths scribbled
in a space intended for a much smaller string;
in my testing it was something like
lengths were 67 with 300+ written to it.

I think we've reported the relevant issues upstream,
but unfortunately it appears our patches
are what introduces the overwrite/corruption
(by no longer being correct in what they assume)

This doesn't look so bad to fix but it's
not something I can spend more time on
at the moment.

--

Interestingly the overwritten string data
(because it is scribbled past the bounds)
remains in the binary and is why we're suddenly
seeing unexpected references in various builds
-- notably this is is the reason we're
seeing the "extra-utils" breakage
that entirely crippled NixOS on master
(and probably on staging before?).

Fixes #43650.

This reverts commit 305ac4dade.

(cherry picked from commit 273d68eff8)
Signed-off-by: Domen Kožar <domen@dev.si>
 2018-07-24 15:04:48 +01:00
John Ericson
611eefb067
Merge pull request #44025 from obsidiansystems/stdenv-deps-typo 
mkDerivation: depsHostBuild -> depsHostHost
 2018-07-23 16:09:21 -04:00
John Ericson
ce2637c97a mkDerivation: depsHostBuild -> depsHostHost 
`depsHostBuild` is not a thing, would never be a thing per the rules,
and isn't used anywhere. This is just my typo, hitherto unnoticed
because "host -> host" dependencies are by far the most obscure form.
 2018-07-23 15:22:48 -04:00