Commit Graph

2227 Commits

Author SHA1 Message Date
Artturin
f002ffed9a treewide: enable strictDeps in bootstrap packages 2022-05-22 16:40:26 +03:00
Artturin
6d54fe622d stdenv: start deprecating non-list configureFlags
the motivation for this is to simplify stdenv and ease the job of
reviewers due to them needing to tell contributors about the defacto
rule that configureFlags should be a list of strings
2022-05-18 20:32:33 +03:00
Artturi
22f6a17b22
Merge pull request #172570 from Artturin/epbbydefault 2022-05-13 20:46:33 +03:00
Artturin
a17e1b8809 config.enableParallelBuildingByDefault: init option
useful for testing if some builds could be faster
2022-05-11 16:41:30 +03:00
Linus Heckemann
676e4d42d6
Merge pull request #157106 from lheckemann/better-error-for-bad-buildinputs
stdenv/make-derivation: check that all inputs are of an appropriate type
2022-05-11 11:28:11 +02:00
github-actions[bot]
f39e774f11
Merge master into staging-next 2022-05-09 00:02:27 +00:00
John Ericson
db0c4f14ac
Merge pull request #172022 from elikoga/patch-2
Update link from old nix manual to current stable one
2022-05-08 18:26:19 -04:00
elikoga
04f6a432c4
Update link from old nix manual to current stable one
https://github.com/NixOS/rfcs/pull/72
2022-05-08 05:20:34 +02:00
Artturin
2d019e0931 stdenv: deprecate unused adapters
stdenv: deprecate addCoverageInstrumentation adapter

this used to be used in nixos/tests but it hasn't been used in nixpkgs
for years

stdenv: deprecate replaceMaintainersField adapter

it was added in 2009 in 01e98e49b1
by nbp
there are no uses of it in nixpkgs now

stdenv: deprecate validateLicenses adapter

it was added in 2009 in b29073af25

unfreePredicate is now handled in ./pkgs/stdenv/generic/check-meta.nix
so this is unnecessary
2022-05-07 22:36:10 +03:00
Linus Heckemann
b3e8855999 make-derivation: allow strings in build input lists 2022-05-07 10:42:50 +02:00
github-actions[bot]
16684f8bd3
Merge master into staging-next 2022-05-04 12:01:10 +00:00
Artturi
f0e48dd98b
Merge pull request #119942 from hercules-ci/mkDerivation-overridable-recursive-attributes 2022-05-04 14:31:50 +03:00
github-actions[bot]
ee7e3f30f3
Merge staging-next into staging 2022-05-04 00:02:57 +00:00
ckie
4def222ea4
stdenv/check-meta: add a "maintainerless" warning
This warning logs when a package has no maintainers. It will stay silent
if `meta.maintainers` is not set at all, only complaining when it is an
empty list. In the future a separate warning could be added to allow for
that stricter behavior. Or this warning could be changed.
2022-05-03 22:29:14 +03:00
ckie
3a34b6c820
stdenv/check-meta: add an eval warning option
This will be used in the next commit in this patch series.
2022-05-03 22:29:12 +03:00
ckie
5e420c2455
stdenv/check-meta: turn validity.valid into a str
This will allow for adding more validity types in the future, such as a
warning type. (which is in the next commit in this series)

This is NOT a breaking change because validity.valid is never exposed
outside of `stdenv.mkDerivation`.
2022-05-03 22:28:25 +03:00
github-actions[bot]
51a899bdc1
Merge staging-next into staging 2022-05-03 18:10:46 +00:00
Artturin
9f05fc6661 config.allowUnsupportedSystem: define as option 2022-05-02 20:39:43 +03:00
Artturin
9f473092f8 config.allowBroken: define as option 2022-05-02 17:20:44 +03:00
Artturin
1c49b81263 config.allowUnfree: define as option 2022-05-02 17:20:25 +03:00
Robert Hensing
0e00acafe9 stdenv.mkDerivation: public -> finalPackage 2022-05-02 08:49:33 +02:00
Robert Hensing
37ab5b4396 mkDerivation: Add error hint for infinite recursion 2022-05-02 08:49:32 +02:00
Robert Hensing
41b3688ba1 make-derivation.nix: Remove unnecessary TODO
The goal was to keep the number of calls to a minimum, but that
would come at the cost of clobbering git blame and possibly not
improving readability.
2022-05-02 08:49:31 +02:00
Robert Hensing
6d7efb3a16 stdenv.mkDerivation: Make self more overlay-like; use self.public
`self` is now arguments, like `super`. The final package is in
`self.public`.
2022-05-02 08:49:31 +02:00
Robert Hensing
a4e7085227 stdenv.mkDerivation: Allow overriding of recursive definitions
See updated manual for further explanation.
2022-05-02 08:49:26 +02:00
Ben Siraphob
02dbca8c2b
Merge pull request #161270 from Stunkymonkey/darwin-phases 2022-04-30 10:58:05 -06:00
Artturin
33415135b0 config.contentAddressedByDefault: init option 2022-04-27 23:21:32 +03:00
Artturin
d76021402d config.strictDepsByDefault: init option
so i won't have to change make-derivation.nix when i wan't to make this
true
2022-04-27 23:21:21 +03:00
github-actions[bot]
deb60260a7
Merge master into staging-next 2022-04-17 06:01:07 +00:00
Adam Joseph
9d60e3dd29 stdenv: use disallowedRequisites to check forbidden requisites 2022-04-13 23:13:33 -07:00
Adam Joseph
9d46a1025a pkgs/stdenv/linux: add powerpc64le bootstrap-files
This PR adds Hydra-generated bootstrap tarballs for powerpc64le-linux.
I'll be following the script established in PR to tarballs.nixos.org.

Files came from this Hydra build:

  https://hydra.nixos.org/build/172142499

Which used nixpkgs revision 49a83445c2
to instantiate:

  /nix/store/gj272sd56gsj6qpyzh4njpfzwdhviliz-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu.drv

and then built:

  /nix/store/n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu

I downloaded these files from Hydra with the following commands:

```
STOREPATH=n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu
OPTIONS="--option binary-caches https://cache.nixos.org"
nix store add-file \
  --name bootstrap-tools.tar.xz  \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz
nix store add-path \
  --name busybox \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox
```

As @lovesegfault requested in #151399, here are the the `sha256sum`s
of all the `on-server` components for extra verification:

```
$ sha256sum /nix/store/n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/*
036d062869f7accf0ad89714d12029469dfe6af504f9b226d61eb7d808ad4735  /nix/store/n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/bootstrap-tools.tar.xz
6bc05832180f5075f4458c32eb0a5e2b673f605884dce01822be474f4e0a63ee  /nix/store/n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/busybox
```

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2022-04-11 18:35:44 -07:00
github-actions[bot]
3dc8bd98b9
Merge staging-next into staging 2022-04-06 00:02:41 +00:00
Artturi
e2166cd22b
Merge pull request #166827 from Artturin/addtimeprint 2022-04-05 22:13:58 +03:00
Vladimír Čunát
b4729bad3d
Merge #165406: staging-next 2022-03-23 2022-04-05 20:34:08 +02:00
Robert Hensing
559ac3c9e7
Merge pull request #166383 from hercules-ci/always-sanitize-derivation-name
Always sanitize derivation name
2022-04-05 20:05:33 +02:00
Linus Heckemann
235fe92e42 make-derivation: allow nested lists in buildInputs
This isn't really desirable in general, but given that Nix itself
currently relies on this behaviour and that we don't want to break
backwards compatibility we should support it for now, maybe
deprecating it in the future.
2022-04-05 12:20:38 +02:00
Martin Weinelt
bf1914e12a Merge remote-tracking branch 'origin/staging-next' into staging 2022-04-05 04:05:02 +02:00
Vladimír Čunát
6fd0d1bb4b
stdenv bootstrap tools: adjust with glibc-2.34 (2nd try)
This time without rebuilding stdenvs.
2022-04-04 20:40:42 +02:00
Vladimír Čunát
e66ed266f8
Revert "stdenv bootstrap tools: adjust with glibc-2.34"
This reverts commit c1ef87e8d0.
I didn't realize that it would rebuild all linux stdenvs;
I certainly don't want to incur such a rebuild at this point.
2022-04-04 20:40:41 +02:00
Vladimír Čunát
c1ef87e8d0
stdenv bootstrap tools: adjust with glibc-2.34
https://hydra.nixos.org/build/171494041
2022-04-04 19:54:28 +02:00
Linus Heckemann
dbdacbca8e Merge remote-tracking branch 'upstream/nixos-unstable' into better-error-for-bad-buildinputs 2022-04-02 23:12:56 +02:00
github-actions[bot]
ffa96ab0c0
Merge staging-next into staging 2022-04-02 18:01:37 +00:00
github-actions[bot]
fc17fe6417
Merge master into staging-next 2022-04-02 18:01:07 +00:00
Linus Heckemann
93681a52a5 stdenv: check that all inputs are of an appropriate type
Fixes #24462
2022-04-02 19:09:10 +02:00
7c6f434c
8188f10752
Merge pull request #166430 from alyssais/openssl-static-retry
treewide: use lib.getLib for OpenSSL libraries
2022-04-02 12:59:55 +00:00
Artturin
1e2a288f0e stdenv: print the time the phase took if it was longer than 30s
will be useful for finding why a build runs for a long time on hydra
because of tests or the build etc etc
2022-04-02 00:42:02 +03:00
Alyssa Ross
fd78240ac8
treewide: use lib.getLib for OpenSSL libraries
At some point, I'd like to make another attempt at
71f1f4884b ("openssl: stop static binaries referencing libs"), which
was reverted in 195c7da07d.  One problem with my previous attempt is
that I moved OpenSSL's libraries to a lib output, but many dependent
packages were hardcoding the out output as the location of the
libraries.  This patch fixes every such case I could find in the tree.
It won't have any effect immediately, but will mean these packages
will automatically use an OpenSSL lib output if it is reintroduced in
future.

This patch should cause very few rebuilds, because it shouldn't make
any change at all to most packages I'm touching.  The few rebuilds
that are introduced come from when I've changed a package builder not
to use variable names like openssl.out in scripts / substitution
patterns, which would be confusing since they don't hardcode the
output any more.

I started by making the following global replacements:

    ${pkgs.openssl.out}/lib -> ${lib.getLib pkgs.openssl}/lib
    ${openssl.out}/lib -> ${lib.getLib openssl}/lib

Then I removed the ".out" suffix when part of the argument to
lib.makeLibraryPath, since that function uses lib.getLib internally.

Then I fixed up cases where openssl was part of the -L flag to the
compiler/linker, since that unambigously is referring to libraries.

Then I manually investigated and fixed the following packages:

 - pycurl
 - citrix-workspace
 - ppp
 - wraith
 - unbound
 - gambit
 - acl2

I'm reasonably confindent in my fixes for all of them.

For acl2, since the openssl library paths are manually provided above
anyway, I don't think openssl is required separately as a build input
at all.  Removing it doesn't make a difference to the output size, the
file list, or the closure.

I've tested evaluation with the OfBorg meta checks, to protect against
introducing evaluation failures.
2022-03-30 15:10:00 +00:00
Robert Hensing
7e0ef81015 stdenv.mkDerivation: Always sanitize derivation name 2022-03-30 10:29:19 +02:00
Bernardo Meurer
e2d150f6b1
Merge pull request #134917 from NixOS/binutils-2.37
binutils: 2.35.1 -> 2.38
2022-03-29 10:16:36 -07:00
Bernardo Meurer
8e5d2ead61
binutils: rename gold to enableGold 2022-03-28 18:50:11 -07:00
Bernardo Meurer
7bac80fef8
stdenv: fix binutils' bootstrap
Co-authored-by: TredwellGit <tredwell@tutanota.com>
2022-03-28 18:50:04 -07:00
github-actions[bot]
b2fc63a844
Merge staging-next into staging 2022-03-26 12:02:50 +00:00
github-actions[bot]
811b90f085
Merge master into staging-next 2022-03-26 12:01:14 +00:00
Adam Joseph
13e63f67dd make-bootstrap-tools-cross: fix typo which prevents hydra eval
When implementing this reviewer request:

  https://github.com/NixOS/nixpkgs/pull/161158#discussion_r822256070

to move mips64el-unknown-linux-* from platforms.nix to examples.nix, I
neglected to update the reference in make-bootstrap-tools-cross.nix.
As a result, the hydra jobs to generate a bootstrap tarball for
mips64el are not running:

  https://hydra.nixos.org/eval/1750828?filter=mips&compare=1750620&full=#tabs-errors

This commit fixes the problem, so the hydra job can run.  Once it
does, I will submit a PR adding the trusted bootstrap tarball hash to
pkgs/stdenv/linux/bootstrap-files/.

Co-authored-by: sterni <sternenseemann@systemli.org>
2022-03-26 12:03:00 +01:00
github-actions[bot]
8357b101a5
Merge staging-next into staging 2022-03-25 12:02:05 +00:00
Vladimír Čunát
0a8b4eddd2
Merge branch 'master' into staging-next 2022-03-25 10:16:56 +01:00
Artturin
33cce15e42 treewide: remove meta.repositories
there's no documentation for meta.repositories and its not widely used
2022-03-24 23:56:14 +02:00
sternenseemann
b5cad4d4a4 stdenv/setup.sh: make sure $sourceRoot has +x before cd-ing
This change is prompted by the following, admittedly cursed tarball:

```
> curl https://registry.npmjs.org/char-regex/-/char-regex-1.0.2.tgz 2>/dev/null \
  | tar -ztv
drw-rw-rw- 0/0               0 2020-02-18 10:50 package
-rw-rw-rw- 0/0             297 2020-02-18 10:50 package/index.d.ts
-rw-rw-rw- 0/0            1920 2020-02-18 10:50 package/index.js
-rw-rw-rw- 0/0            1092 2020-01-31 11:31 package/LICENSE
-rw-rw-rw- 0/0             937 2020-02-18 10:51 package/package.json
-rw-rw-rw- 0/0             713 2020-02-18 10:50 package/README.md
```

The minimal reproducer for the issue is the following derivation trying
to work around the uid 0 issue with `dontMakeSourcesWritable = true`:

```nix
{ stdenv, fetchurl }:

stdenv.mkDerivation {
  name = "test";

  src = fetchurl {
    sha1 = "d744358226217f981ed58f479b1d6bcc29545dcf";
    url = "https://registry.npmjs.org/char-regex/-/char-regex-1.0.2.tgz";
  };

  dontMakeSourcesWritable = true;

  installPhase = ''
    cp -R . $out
  '';
}
```

This currently fails in the following way:

```
these derivations will be built:
  /nix/store/pc3jbydl0xcc8nrndf5xkf7hdhpgpb41-test.drv
building '/nix/store/pc3jbydl0xcc8nrndf5xkf7hdhpgpb41-test.drv'...
unpacking sources
unpacking source archive /nix/store/v9p98kqplf4kflmy91p0687xlvr6klb1-char-regex-1.0.2.tgz
source root is package
find: 'package/index.d.ts': Permission denied
find: 'package/index.js': Permission denied
find: 'package/LICENSE': Permission denied
find: 'package/package.json': Permission denied
find: 'package/README.md': Permission denied
/nix/store/6c47azxacncswc1pllzj28zfzqw40d7c-stdenv-linux/setup: line 1311: cd: package: Permission denied
builder for '/nix/store/pc3jbydl0xcc8nrndf5xkf7hdhpgpb41-test.drv' failed with exit code 1
error: build of '/nix/store/pc3jbydl0xcc8nrndf5xkf7hdhpgpb41-test.drv' failed
```

As you can see, the issue is that `$sourceRoot` isn't executable,
prohibiting the call to `cd`. This can be fixed by running
`chmod +x "${sourceRoot}"` before `cd` regardless of
`dontMakeSourcesWritable` in `unpackPhase` since if `chmod` fails, `cd`
would fail as well and we are out of options.

Verified that the workaround works locally.

Another thing to investigate is investigating if we should use
`--no-same-owner` for `tar` and if it helps in this case as well.
See also <https://github.com/Profpatsch/yarn2nix/issues/56>.
2022-03-24 11:13:38 +01:00
github-actions[bot]
691557fc47
Merge staging-next into staging 2022-03-18 00:03:09 +00:00
Adam Joseph
12371a51e6 lib/systems: add mips64el definitions
MIPS has a large space of {architecture,abi,endianness}; this commit
adds all of them to lib/systems/platforms.nix so we can be done with
it.

Currently lib/systems/inspect.nix has a single "isMips" predicate,
which is a bit ambiguous now that we will have both mips32 and mips64
support, with the latter having two ABIs.  Let's add four new
predicates (isMips32, isMips64, isMips64n32, and isMips64n64) and
treat the now-ambiguous isMips as deprecated in favor of the
more-specific predicates.  These predicates are used mainly for
enabling/disabling target-specific workarounds, and it is extremely
rare that a platform-specific workaround is needed, and both mips32
and mips64 need exactly the same workaround.

The separate predicates (isMips64n32 and isMips64n64) for ABI
distinctions are, unfortunately, useful.  Boost's user-scheduled
threading (used by nix) does does not currently supports mips64n32,
which is a very desirable ABI on routers since they rarely have
more than 2**32 bytes of DRAM.
2022-03-10 20:30:16 -08:00
github-actions[bot]
c15f51235d
Merge staging-next into staging 2022-03-07 12:02:15 +00:00
Maximilian Bosch
856376bb20
Merge pull request #162197 from trofi/fix-gcc-wrapper-in-stdenv
pkgs/stdenv/linux/default.nix: restore dropped gcc-wrapper
2022-03-01 18:35:49 +01:00
Sandro
800e9e9994
Merge pull request #161079 from SuperSandro2000/exit-handler 2022-03-01 14:41:56 +01:00
Felix Buehler
63a37b844c darwin: deprecate phases 2022-02-28 10:58:08 +01:00
Sergei Trofimovich
32c30ae43c pkgs/stdenv/linux/default.nix: restore dropped gcc-wrapper
7459a40 `stdenv-bootstrap: force using new libc from stage2` overrode
`gcc-wrapper`. 91fa7657 `stdenv: revert gcc hack` dod not restore the initial
value.

As a result attempt to build glibc-2.35 fails early on perl as:

   ld: cannot find -lnsl: No such file or directory
   collect2: error: ld returned 1 exit status
   I can't compile the test program.
   (The supplied flags or libraries might be incorrect.)

The change restores `gcc-wrapper` as it was before 7459a40.
2022-02-28 09:42:20 +00:00
Maximilian Bosch
91fa7657d1
stdenv: revert gcc hack
See https://github.com/NixOS/nixpkgs/pull/133431#discussion_r805339569
2022-02-27 15:10:58 +01:00
Maximilian Bosch
7459a4021c
stdenv-bootstrap: force using new libc from stage2
This fix is needed to work around linker-errors such as

    undefined reference to `__libc_csu_fini'

which I got in almost every derivation which is part of stage2. The
reasoning behind this is that the startup-code was simplified[1] and
thus `__libc_csu_fini` doesn't exist anymore.

A workable solution is to use a newer libc which properly links in
stage3. And actually this seems expected given the rationale for stage3:

    # Construct a third stdenv identical to the 2nd, except that this
    # one uses the rebuilt Glibc from stage2.  It still uses the recent
    # binutils and rest of the bootstrap tools, including GCC.

So this patch basically overrides the libraries inside `gcc-unwrapped` -
which is basically the bootstrap tools and thus also contains the libc
used in stage3 - with the shared objects from the freshly built libc
from stage2.

[1] https://sourceware.org/pipermail/libc-alpha/2021-March/123079.html
2022-02-27 10:25:33 +01:00
Sandro Jäckel
b7da6c7da7
stdenv, dep-licenses.sh: do not skip handling of other exit traps
See https://github.com/akinomyoga/ble.sh/issues/179
2022-02-20 16:27:24 +01:00
Naïm Favier
1ffdf02435
stdenv/check-meta: remove onlyLicenses check 2022-02-17 13:09:24 +01:00
github-actions[bot]
335510eb84
Merge master into staging-next 2022-02-08 00:01:49 +00:00
zowoq
f5227f0643 bootstrap-tools: disable iconv for the hello test 2022-02-08 08:57:11 +10:00
Dmitry Kalinkin
3087088c41
Merge branch 'staging-next' into staging
Conflicts:
	pkgs/top-level/aliases.nix
	pkgs/top-level/python-aliases.nix
2022-02-01 21:37:39 -05:00
matthewcroughan
7bea56b425 stdenv/check-meta: add note for Flake usage
Flake users that use a command like `nix build nixpkgs#hello` on a
broken/insecure package will not be able to use an environment variable
to override that behavior, unless they pass `--impure` to the command.

Co-authored-by: pkharvey <kayharvey@protonmail.com>
2022-02-01 13:23:46 -05:00
github-actions[bot]
ce60c22080
Merge staging-next into staging 2022-01-30 06:01:52 +00:00
matthewcroughan
b0c0e0d7eb stdenv: introduce withCFlags
Adds an easy method of appending compiler flags to your stdenv via a
list.

Co-authored-by: tomberek <tomberek@users.noreply.github.com>
Co-authored-by: Gytis Ivaskevicius <gytis02.21@gmail.com>
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
2022-01-29 21:23:12 -05:00
Jan Tojnar
2da5ce4ab3 Merge branch 'staging-next' into staging
; Conflicts:
;	pkgs/development/python-modules/fakeredis/default.nix
2022-01-28 14:05:11 +01:00
Sandro
27cccd4e49
Merge pull request #151363 from Stunkymonkey/doc-updateWalker 2022-01-27 14:06:36 +01:00
Felix Buehler
59c55f4558 update-walker: remove because unused 2022-01-26 21:46:59 +01:00
Sandro Jäckel
24880b690b stdenv: fix shellcheck complaining about things not being posix compliant 2022-01-23 03:50:23 +01:00
Sandro Jäckel
37fdba0b4f stdenv: restore bash options -e/-u to the values they where before
Source https://github.com/akinomyoga/ble.sh/issues/169#issuecomment-1019049032

Author: akinomyoga
2022-01-23 03:50:09 +01:00
Dmitry Kalinkin
7673650020
stdenv/darwin: fix for curl with zstd and idn2
The `curlMinimal` is to be used throughout the early bootstrap
stages. The final stage will allow the new references of the `curl`.

Fixes: 29526bc2 ('curl: IDN support requires libidn2 package')
2022-01-13 19:45:30 -05:00
John Ericson
baf290023c
Merge pull request #153790 from sternenseemann/cross-bootstrap-logic
Cross bootstrapping logic fixes benefitting llvmPackages
2022-01-08 12:39:30 -08:00
sternenseemann
766f5ffb76 llvmPackages_*: respect cc for target when choosing C++ flavour
llvmPackages_*.clang should check the default compiler for the package
set it is targeting (targetPackages.stdenv.cc) instead of the compiler
that has been used to build it (stdenv.cc) in order to get some sense of
whether to use libc++ or libstdc++.

Since we are now inspecting targetPackages in the llvmPackages.clang
attribute, we need to avoid using it in the cross stdenv — which just
forces us to explicitly request libcxxClang for darwin instead of
relying on the clang attribute to pick it for us.

We also need to do something similar for targetPackages.stdenv.cc: Here
the llvmPackages.clang logic would work as we want (inspect
targetPackages.stdenv.cc and if it doesn't exist, make the choice based
on stdenv.cc), but it gets locked in a cycle with the previous package.
We can easily break this, however: We know that the previous set had
clang and the next one doesn't exist, so we'd choose libcxxClang any day
of the week.
2022-01-07 14:42:41 +01:00
Vladimír Čunát
0b7de2ab41
Merge #151399: stdenv: update aarch64 bootstrap-files
...into staging
2022-01-04 20:08:39 +01:00
Naïm Favier
f2065d81ad stdenv/generic: introduce shellDryRun
Add `shellDryRun` to the generic stdenv and substitute it for uses of
`${stdenv.shell} -n`. The point of this layer of abstraction is to add
the flag `-O extglob`, which resolves #126344 in a more direct way.
2021-12-27 20:30:01 -05:00
github-actions[bot]
e38de71da8
Merge master into staging-next 2021-12-25 00:01:28 +00:00
rembo10
eefebccc12 cmake make-derivation: fix hostPlatform uname release when cross compiling 2021-12-24 13:18:42 +05:30
Dmitry Kalinkin
1a9297102c
Merge branch 'master' into staging-next 2021-12-23 16:58:33 -05:00
Bernardo Meurer
ba3cc463ca
Merge pull request #147942 from oxalica/test/riscv-bootstrap-tools
stdenv: bootstrap riscv64 (Step 2)
2021-12-23 21:40:31 +00:00
David Guibert
8e7acfc587 aarch64: update bootstrap-files
fixes #112086

Fetch bootstrap files built by Hydra and re-add them as FODs to skip
currently dangling URLs. They are already available on
https://cache.nixos.org

https://hydra.nixos.org/job/nixpkgs/trunk/stdenvBootstrapTools.aarch64-linux.dist ->
https://hydra.nixos.org/build/160330465#tabs-buildinputs

These commands have been run to get the output hashes as FODs:

nix store add-file --name bootstrap-tools.tar.xz  $(nix-store -r /nix/store/9y4qnwflnxkjrqnfbzhsjv6zq2r0m48z-stdenv-bootstrap-tools --option binary-caches https://cache.nixos.org)/on-server/bootstrap-tools.tar.xz
nix store add-path --name busybox                 $(nix-store -r /nix/store/9y4qnwflnxkjrqnfbzhsjv6zq2r0m48z-stdenv-bootstrap-tools --option binary-caches https://cache.nixos.org)/on-server/busybox
2021-12-20 10:53:19 +01:00
github-actions[bot]
209e8d0932
Merge master into staging-next 2021-12-11 00:01:48 +00:00
Guillaume Girol
4a8f9970e2
Merge pull request #147544 from cab404/fix-stdenv-override
stdenv: move overriden stdenv in closure
2021-12-10 20:29:30 +00:00
github-actions[bot]
f11307e4b2
Merge staging-next into staging 2021-11-30 00:02:29 +00:00
oxalica
a5851f18c5
stdenv: add bootstrap files for riscv64
Bootstrap files are from
https://hydra.nixos.org/build/159891432
and
https://hydra.nixos.org/build/159891436
2021-11-30 05:03:30 +08:00
Vladimir Serov
523c701c0b
stdenv: move overriden stdenv in closure
Before that, base stdenv passed non-makeOverridable version of itself
inside. This cause it to be lost on package-name.stdenv.
2021-11-27 01:23:32 +03:00
oxalica
8cf96b9426
stdenv: patch autoconf script for riscv in stage 2 2021-11-26 00:32:33 +08:00
oxalica
fc3ea54fb3
make-bootstrap-tools: produce libatomic on riscv platform 2021-11-26 00:32:33 +08:00
toonn
5d23e617ec stdenv: Pass standalone argument for libcxxabi 2021-11-24 15:17:13 -08:00
toonn
63a3ead084 stdenv: Darwin no longer needs 10.11 workaround
Co-authored-by: happysalada <raphael@megzari.com>
2021-11-24 15:17:13 -08:00
toonn
77fc5f8c14 bootstrap-tools: Introduce getLib for lib outputs
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
Co-authored-by: Jörg Thalheim <joerg@thalheim.io>
2021-11-24 15:17:13 -08:00
toonn
cc767e1d05 darwin.stdenv: Update to LLVM 11 and clang 11.1.0 2021-11-24 15:17:13 -08:00
toonn
84454c3233 darwin.Libsystem: Patch TargetConditionals.h
`TargetConditionals.h` was missing several definitions, like
`TARGET_OS_TV` that are part of SDK 10.12 at least. And one that doesn't
seem to occur in any SDK afaict, `TARGET_OS_EMBEDDED_OTHER`.

I added the definitions from SDK 10.12 verbatim and defined
`TARGET_OS_EMBEDDED_OTHER` to be equal to `0`.

I think none of this works if `darwin.Libsystem` is used to build for
linux or iOS though so maybe this needs a more thorough fix?

This reverts 336d82617f because it's no
longer necessary.
2021-11-24 15:17:13 -08:00
toonn
e07eef85e8 xnu: Fix python3 patch
This reverts 7da313e10a because it's no
longer necessary.
2021-11-24 15:17:13 -08:00
Daiderd Jordan
7e4880c21b stdenv: update darwin bootstrap tools for llvm11
Build from bdffd0b276.

https://hydra.nixos.org/build/124753463
(cherry picked from commit f4a08349ed43a36ad5489a4ab548fcf0c129215d)
2021-11-24 15:17:13 -08:00
Alyssa Ross
2ebeb02a99 stdenv/setup: tell libtool about library paths
Packages that use libtool run it as a wrapper around the linker.
Before calling the linker, libtool will determine what libraries would
be linked, and check if there's a corresponding libtool
archive (libfoo.la) file in the same directory .  This file
contains extra information about the library.  This is especially
important for static linking, because static archives don't contain
dependency information, so we need libtool to use the .la files to
figure out which libraries actually need to be linked against.

But in Nixpkgs, this has never worked.  libtool isn't able to find any
libraries, because only the compiler wrapper knows how to find them,
and the compiler wrapper is opaque to libtool.  This is why
pkgsStatic.util-linuxMinimal doesn't build prior to this patch — it
depends on libpam, which depends on libaudit, and if libtool can't
find the .la file, nothing will tell the linker to also link against
libaudit when linking libpam.  (It was previously possible to build a
static util-linux, because linux-pam only recently had the audit
dependency added.)

There are a couple of ways we could fix this, so that libtool knows
where to look for .la files.

 * Set LD_LIBRARY_PATH/DYLD_LIBRARY_PATH/whatever, which libtool will
   examine.  This would have major side effects though, because the
   dynamic linker looks at it too.

 * Inject libtool scripts with the appropriate information.  That's
   what I've done here.  It was the obvious choice because we're
   already finding and modifying the libtool scripts, to remove paths
   outside the Nix store that libtool might check in unsandboxed
   builds.  Instead of emptying out the system paths, we can
   repopulate it with our own library paths.

(We can't use a wrapper like we do for other tools in Nixpkgs, because
libtool scripts are often distributed in source tarballs, so we can't
just add a wrapped version of libtool as a dependency.  That's why
there's already the fixLibtool function in stdenv.)

With this change, libtool is able to discover .la files, and
pkgsStatic.util-linuxMinimal can build again, linking correctly
against libpam and libaudit.
2021-11-23 21:33:16 +00:00
Artturi
0809a3a44b
Merge pull request #137209 from milahu/patch-10 2021-10-25 20:32:06 +03:00
Rick van Schijndel
6a848de95d stdenv: add llvm-config to meson cross file
This is e.g. required to get mesa cross-compiling, but is useful for everyone depending on libllvm
2021-10-13 20:44:10 +02:00
sterni
5d0972c63b
Merge pull request #138289 from sternenseemann/fix-non-gnu-strip
stdenv: move --enable-deterministic-archives flag into GNU wrapper
2021-10-07 12:29:59 +02:00
happysalada
a634fbe065 stdenv: add shopt inherit_errexit 2021-10-06 00:42:58 +09:00
happysalada
84e4715a14 stdenv: use named ref to clarify intent 2021-10-06 00:42:29 +09:00
Emily
20a6d8a241 libxml2: use libiconv on Darwin
This fixes e.g. python3Packages.beautifulsoup4, which has tests relying
on the wider encoding support enabled by libiconv.

Fixes #137678.
2021-10-02 10:05:17 +09:00
Ryan Burns
40299257cc Merge branch 'master' into staging-next 2021-09-23 18:43:00 -07:00
Michael Stone
470466fdbd
sigtool: 4a3719b4 -> 2a13539d (#138453)
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2021-09-23 17:14:07 +02:00
sternenseemann
11fe2fc3cf stdenv: move --enable-deterministic-archives flag into GNU wrapper
`--enable-deterministic-archives` is a GNU specific strip flag and
causes other strip implementations (for example LLVM's, see #138013)
to fail. Since strip failures are ignored, this means that stripping
doesn't work at all in certain situation (causing unnecessary
dependencies etc.).

To fix this, no longer pass `--enable-deterministic-archives`
unconditionally, but instead add it in a GNU binutils specific strip
wrapper only.

`commonStripFlags` was only used for this flag, so we can remove
it altogether.

Future work could be to make a generic strip wrapper, with support for
nix-support/strip-flags-{before,after} and NIX_STRIP_FLAGS_{BEFORE,AFTER}.
This possibly overkill and unnecessary though -- also with the
additional challenge of incorporating the darwin strip wrapper somehow.
2021-09-18 15:11:17 +02:00
Winter
8cff7796d7
stdenv: re-add isMachO helper function (#138334) 2021-09-17 23:09:06 -04:00
Vladimír Čunát
183cd6b09f
Merge #138186: stdenv: remove isMachO helper function
...into staging-next
2021-09-17 18:31:26 +02:00
Winter
f8edf7720d stdenv: remove isMachO helper function
This reverts commit 488395c0f8.

Currently, `nix print-dev-env` fails to execute if this function is present, because of its use of hex literals.
Until this issue (https://github.com/NixOS/nix/issues/5262) is solved, we should revert this to prevent breakage.
2021-09-16 16:56:12 -04:00
happysalada
33518fcb45 stdenv/setup.sh: fix read -N 0 for bash 5
somehow `read -N 0` behavior changed in bash 5. `read -d ''` has identical behavior
the purpose of the function is to read stdin and exit 1 on a null byte (i.e. if stdin is the content of a binary)

(cherry picked from commit 5d0acf20f8)
2021-09-12 09:48:54 +09:00
milahu
5e2f703e83
unpackFile: ignore timestamp warnings 2021-09-09 20:13:21 +02:00
happysalada
4c92bb8bdf stdenv: fix nix_build_cores guess
- use builtin arithmetic instead of external expr
- simplify logic with bash builtins
2021-09-07 00:36:55 +09:00
happysalada
02c142a2dd stdenv: fix showBuildStats
- remove going through another file
- use builtin instead of external cat
- improve echo formatting
2021-09-06 22:51:33 +09:00
Martin Weinelt
7f732aca66 Revert "Merge remote-tracking branch 'origin/python-unstable' into staging-next"
This reverts commit b041b2e1b2, reversing
changes made to 5b6c2380ad.
2021-09-05 15:02:25 +02:00
happysalada
e32bf6f4f2 stdenv setup.sh: remove combined [ in favor of [[
[ ... ] && [ ...] -> [[ ... && ... ]]
2021-08-30 10:27:45 +09:00
happysalada
fd89fb6248 stdenv: remove bash version compatibility hack 2021-08-30 10:26:56 +09:00
happysalada
13049cd33e stdenv: remove combined command conditional 2021-08-30 10:26:14 +09:00
happysalada
2fa9facc49 stdenv: arithmetic fixes 2021-08-30 10:25:40 +09:00
happysalada
1200c8175c stdenv: reduce io 2021-08-30 10:24:30 +09:00
happysalada
54475daa27 stdenv: declare missing variables 2021-08-30 10:24:00 +09:00
Sebastián Mancilla
488395c0f8
stdenv: add isMachO helper function (#133808)
Detect if a binary is a Mach-O file.
2021-08-21 15:33:03 -04:00
John Ericson
3edba5edfc pkgsStatic: Finally obviate overlay! 2021-08-20 06:09:48 +00:00
John Ericson
904625852d pkgsStatic: Inline more of static overlay 2021-08-19 21:55:06 +00:00
John Ericson
221ca6f2ff
Merge pull request #134463 from Ericson2314/stdenv-adapter-latebind
stdenv: Fix overriding + `overrideAttrs`
2021-08-18 19:01:54 -04:00
John Ericson
b4cc2a2479 pkgs/stdenv/make-derivation: Reindent
We previously make it just be the function, not a single-item attrset,
without deindenting to make a readable diff. No we deindent.
2021-08-18 17:22:52 +00:00
John Ericson
f110a182a6 stdenv: Fix overriding + overrideAttrs
The old stdenv adapters were subtly wrong in two ways:

 - `overrideAttrs` leaked the original, unoverridden `mkDerivation`.

 - `stdenv.override` would throw away any manually-set `mkDerivation`
   from a stdenv reverting to the original.

Now, `mkDerivation` is controlled (nearly directly) via an argument, and
always correctly closes over the final ("self") stdenv. This means the
adapters can work entirely via `.override` without any manual `stdenv //
...`, and both those issues are fixed.

Note hashes are changed, because stdenvs no previously overridden like
`stdenvNoCC` and `crossLibcStdenv` now are. I had to add some
`dontDisableStatic = true` accordingly. The flip side however is that
since the overrides compose, we no longer need to override anything but
the default `stdenv` from which all the others are created.
2021-08-18 17:22:50 +00:00
github-actions[bot]
1b4a28fed4
Merge master into staging-next 2021-08-07 12:01:25 +00:00
Artturin
40944bbab7 stdenv/check-meta: add maxSilent
Hydra supports it
https://github.com/NixOS/hydra/blob/master/src/hydra-eval-jobs/hydra-eval-jobs.cc#L172
2021-08-07 02:18:59 +03:00
Alyssa Ross
0901dfb214 stdenv/setup: force libtool to skip dep checks
When we "fix" libtool, we empty out its system library path to avoid
it discovering libraries in e.g. /usr when the sandbox is disabled.
But this also means that the checks libtool does to make sure it can
find the libraries its supposed to be linking to won't work.  On Linux
and Darwin, this isn't a problem, because libtool doesn't actually
perform any checks, but it is on at least NetBSD and Cygwin[1].

So, we force libtool not to do these checks on any platform, bringing
the more exotic platforms into line with the existing behaviour on
Linux and Darwin.

Without this change, lots of library packages produce warnings like
this in their build output on the platforms with checks by default:

    *** Warning: linker path does not have real file for library -lz.
    *** I have the capability to make that library automatically link in when
    *** you link to this library.  But I can only do this if you have a
    *** shared version of the library, which you do not appear to have
    *** because I did check the linker path looking for a file starting
    *** with libz but no candidates were found. (...for regex pattern test)
    *** The inter-library dependencies that have been dropped here will be
    *** automatically added whenever a program is linked with this library
    *** or is declared to -dlopen it.

And dependent packages break because libtool doesn't link their
transitive dependencies.  So making this change fixes _lots_ of
packages on those platforms.

[1]: https://git.savannah.gnu.org/cgit/libtool.git/tree/m4/libtool.m4?id=544fc0e2c2a03129a540aebef41ad32bfb5c06b8#n3445
2021-07-27 20:42:31 +00:00
Jörg Thalheim
7bf8b97d68
Merge pull request #130601 from happysalada/stdenv_cosmetic
Stdenv cosmetic
2021-07-23 09:45:43 +01:00
happysalada
4fc50527f8 stdenv: typo 2021-07-19 14:49:47 +09:00
Vladimír Čunát
4bd38c330f
Revert #127736: stdenv changes towards an alternative shell
At least for now.  Such changes are risky (we have very many packages),
and apparently it needs more testing/review without blocking other
changes.

This reverts the whole range 4d0e3984918^..8752c327377,
except for one commit that got reverted in 6f239d7309 already.
(that MR didn't even get its merge commit)
2021-07-17 20:39:47 +02:00
Vladimír Čunát
88a6ee04bf
Merge #129854: stdenv/darwin: finalize URLs (into staging-next) 2021-07-16 17:41:02 +02:00
Vladimír Čunát
2a44031d56
stdenv/make-bootstrap-tools: nuke yet another header
This is needed after glibc bump from PR #111616.
2021-07-16 16:15:32 +02:00
Vladimír Čunát
f47ee76a6a
stdenv/darwin: switch to the official bootstrap URLs 2021-07-16 10:44:04 +02:00
Vladimír Čunát
3b437b95df
Merge #129854: stdenv/darwin: update bootstrap tools
... for x86_64-darwin (into staging-next)
It wouldn't bootstrap otherwise.

Unfortunately we still haven't managed to get the tarballs
on the proper URLs, but GitHub should be reliable enough
and surely almost noone will bootstrap themselves anyway.
2021-07-16 10:26:10 +02:00
Vladimír Čunát
6f239d7309
Revert "setup.sh: fatal: This word should yield a string, but it contains an array"
This reverts commit bf99a819a1.
It caused regressions in some packages; see:
https://github.com/NixOS/nixpkgs/commit/bf99a819a160
2021-07-16 10:00:29 +02:00
Vladimír Čunát
d6a51653d7
stdenv/darwin: update bootstrap tools for x86_64-darwin
Updated python3 will block bootstrapping otherwise.

Verification instructions:
$ xdg-open https://hydra.nixos.org/build/144118577#tabs-details
$ nix build /nix/store/sh9cd0rarr8a2db1zdr4fzl4qsmvz5jp-stdenv-bootstrap-tools
$ cat /nix/store/sh9cd0rarr8a2db1zdr4fzl4qsmvz5jp-stdenv-bootstrap-tools/nix-support/hydra-build-products
$ nix hash-path /nix/store/9h4d7s313wv3gkfwi493yr1wvdsz9lf2-stdenv-bootstrap-tools/on-server/{sh,bzip2,mkdir,cpio}
sha256-igMAVEfumFv/LUNTGfNi2nSehgTNIP4Sg+f3L7u6SMA=
sha256-K3rhkJZipudT1Jgh+l41Y/fNsMkrPtiAsNRDha/lpZI=
sha256-VddFELwLDJGNADKB1fWwWPBtIAlEUgJv2hXRmC4NEeM=
sha256-SWkwvLaFyV44kLKL2nx720SvcL4ej/p2V/bX3uqAGO0=
$ nix hash-file /nix/store/9h4d7s313wv3gkfwi493yr1wvdsz9lf2-stdenv-bootstrap-tools/on-server/bootstrap-tools.cpio.bz2
sha256-b65dXbIm6o6s6U8tAiGpR6SMfvfn/VFcZgTHBetJZis=
2021-07-10 17:03:40 +02:00
Vladimír Čunát
7c25c1fd88
Merge branch 'glibc-2.33' into staging 2021-07-06 16:17:13 +02:00
Vladimír Čunát
5a71fa8447
stdenv bootstrap: hack around glibc version mismatch
With this we shouldn't need other workarounds for the LTO problems.
2021-07-06 16:02:11 +02:00
Raphael Megzari
cf5882f27c Update pkgs/stdenv/generic/setup.sh:use [[ instead of [
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2021-07-06 09:27:18 +09:00
Raphael Megzari
2c3b847d29 Update pkgs/stdenv/generic/setup.sh: use [[ instead of [
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2021-07-06 09:27:18 +09:00