Commit Graph

2568 Commits

Author SHA1 Message Date
Artturin
88f36d2694 gzip: make reproducible when GZIP_NO_TIMESTAMPS is set
the logic can be tested with

```
$ GZIP_NO_TIMESTAMPS=1 && echo "${GZIP_NO_TIMESTAMPS:+-n }"
-n
$ unset GZIP_NO_TIMESTAMPS && echo "${GZIP_NO_TIMESTAMPS:+-n }"

```
2023-01-12 01:30:29 +02:00
github-actions[bot]
b3d2428df4
Merge staging-next into staging 2023-01-11 20:59:01 +00:00
github-actions[bot]
a113c6f06f
Merge master into staging-next 2023-01-11 20:58:29 +00:00
Adam Joseph
098c6b0bec check-meta(hasUnsupportedPlatform): use lib.meta.availableOn
`hasUnsupportedPlatform` was not updated with #37395, so it does not
understand attrsets in `meta.[bad]platforms`.  In particular,
attrsets in `meta.badPlatforms` will "fail open" and be ignored.

Let's use `lib.meta.availableOn` instead of duplicating its logic.

Thanks to @alyssais for [noticing][1].

[1][https://github.com/NixOS/nixpkgs/pull/194148#discussion_r990817610]

Co-authored-by: sternenseemann <sternenseemann@systemli.org>
2023-01-11 19:31:52 +00:00
Adam Joseph
74df5ad72b stdenv: build gettext only once
Right now we build gettext several times during the bootstrap.
Gettext's build process is "embarrassingly serial", so avoiding
rebuilding it speeds things up considerably.
2023-01-10 13:29:02 -08:00
Artturi
386f2bb148
Merge pull request #209255 from Artturin/stdenvupdate1
stdenv: don't fail installPhase on missing makefile
2023-01-10 03:48:17 +02:00
Sergei Trofimovich
8a4e6b7e1b stdenv/darwin: rebuild gawk earlier: in stage4 instead of final stage
`gawk-5.1.1 -> 5.2.1` update (https://github.com/NixOS/nixpkgs/pull/207478)
started failing `stdenv` reference checks as `gawk` now leaks
`bootstrapTools` reference:

    `gawk` -> `gettext` -> `libiconv` -> `bootstrapTools`.

The change rebuild `gawk` in `stage4` to pull rebuilt tools.
2023-01-06 06:48:35 +00:00
Artturin
314b03125f stdenv: don't fail installPhase on missing makefile
otherwise the build just fails with 'make: *** No rule to make target 'install'.  Stop.'

and update buildPhase message

i don't know if the 'makefile may have been created in buildPhase' is
true but i guess it might be possible
2023-01-06 03:01:39 +02:00
github-actions[bot]
789ae2c93d
Merge staging-next into staging 2023-01-04 18:01:36 +00:00
Sergei Trofimovich
b470a6b212 linux/bootstrap-tools: move libstdc++ out of default library search path
This change allows building new gcc during bootstrap without fear of
pulling in outdated libstdc++.so after g++ switched from bootstrapTools
to freshly built g++.

Noticed when tried to add early bootstrap stage to rebuild `gcc` before
`glibc` is fully untangled from `bootstrapTools` as a failure to built
`binutils`:

    ld: dwp.o: in function `__gnu_cxx::new_allocator<gold::Dwp_output_file::Contribution>::allocate(unsigned long, void const*)':
    /nix/store/...-gcc-11.3.0/include/c++/11.3.0/ext/new_allocator.h:116: undefined reference to `std::__throw_bad_array_new_length()'

The change survives existing bootstrap and unblockes early `gcc` bootstrap.
2023-01-04 16:02:22 +00:00
piegames
aa8ead30fd
Merge pull request #175495 from amjoseph-nixpkgs/pr/sourceProvenance/correct-meta-typecheck
check-meta.nix: make non-source consistent with documentation
2023-01-04 15:31:51 +01:00
github-actions[bot]
03e830eb69
Merge staging-next into staging 2023-01-03 00:02:53 +00:00
John Ericson
a6b1de7902
Merge pull request #208478 from trofi/comment-stdenv-bootstrap
stdenv/linux: document some tips in debugging stdenv bootstrap tower
2023-01-02 17:28:43 -05:00
Sergei Trofimovich
062e14b561 stdenv/linux: document some tips in debugging stdenv bootstrap tower
Just a few comments added:

- added a few one-liners to explore which tools are rebuilt at each
  stdenv iteration during bootstrap
- explicitly listed available toolchains and their sources for on each
  bootstrap step: glibc, binutils, gcc, coreutils.
- added mention of static libraries linked into gcc

Co-authored-by: Adam Joseph <54836058+amjoseph-nixpkgs@users.noreply.github.com>
2023-01-02 21:55:32 +00:00
Adam Joseph
607d59fa9e check-meta.nix: make non-source consistent with documentation
The documentation for `meta.sourceProvenance` in
`doc/stdenv/meta.chapter.md` says: "the `meta.sourceProvenance`
attribute should be a list containing one or more value..."

Let's update check-meta.nix to require that `meta.sourceProvenance` is
a list, as the documentation says, rather than a single element.

Adding two extra keystrokes `[` and `]` when filling out this field is
an insignificant burden for package authors, and being able to assume
that the `meta.sourceProvenance` field is always a list greatly
simplifies any code that acts on the value of this field.

Since `meta.sourceProvenance` was just merged a few hours ago now is
the easiest time to fix this: nobody is using the feature yet.
2023-01-01 18:21:11 -08:00
Jan Tojnar
5810109b42 Merge branch 'staging-next' into staging
- readline6 attribute removed from all-packages.nix in d879125d61
- readline attribute was bumped to readline82 in 50adabdd60
2023-01-02 03:04:32 +01:00
Naïm Favier
4af22aab8e
stdenv/check-meta: do deep type checks
Use a wrapper around `mergeDefinitions` to type-check values deeply, so
that e.g. `maintainers = [ 42 ];` is an error.
2023-01-01 14:10:42 +01:00
github-actions[bot]
dc7ebb0163
Merge staging-next into staging 2022-12-18 18:01:41 +00:00
figsoda
ec8cb34358 treewide: fix typos 2022-12-17 19:39:44 -05:00
Sergei Trofimovich
428107f837 stdenv: set enableParallelBuilding explicitly if enableParallelBuildingByDefault is set
Without the change we don't propagate `enableParallelBuilding = true`
and leave most builds sequential.

Noticed on `mythtv` package which did not specify parallelism and
`config.enableParallelBuildingByDefault = true` had no effect.
2022-12-17 22:23:24 +00:00
Naïm Favier
0b067316d4
stdenv: use intersectAttrs instead of intersectLists
Better complexity.
2022-12-17 12:55:44 +01:00
Naïm Favier
84eebc0fe4
Merge pull request #205944 from ncfavier/structured-attrs-env 2022-12-15 13:27:54 +01:00
Naïm Favier
e14de22618
stdenv: handle env gracefully
Derivations not using `__structuredAttrs` should not attempt to set
environment variables from `env`.

Derivations using `__structuredAttrs` should fail if `env` is not
exportable.
2022-12-15 13:27:11 +01:00
Artturin
630bb71ac5 stdenv: sort defaultNativeBuildInputs alphabetically 2022-12-13 01:12:20 +02:00
Artturin
b3717f6c14 stdenv: remove now unneeded linux conditional 2022-12-13 01:04:29 +02:00
Artturin
7866db71cc stdenv/generic: fix todo 2022-12-13 00:34:48 +02:00
Artturin
11c3127e38 stdenv: detect the type of variable in {prepend,append}ToVar
stdenv: error if using {prepend,append}ToVar on associative array

i don't know how to prepend to associative array
2022-12-10 04:42:36 +02:00
Artturin
89dc806f13 what to do about attrs.env or {} maybe have a empty env attrset always
so no need to use `or`
2022-12-08 21:11:35 +02:00
Artturin
02e3f51d27 darwin: use // for binutils-unwrapped and cctools to preserve the other
attributes
2022-12-08 21:09:02 +02:00
Artturin
bb914d8676 stdenv: export system pname name version for substituteAll
so we don't have to add these to the env attrset
2022-12-08 21:09:02 +02:00
Artturin
734d7df235 allow derivation attributes in env
derivations can be coerced to their output paths
2022-12-08 06:13:19 +02:00
Artturin
8ad0103a34 config.structuredAttrsByDefault: add option 2022-12-08 06:13:19 +02:00
Artturin
1c4820efdd work around a nix bug 2022-12-08 06:13:19 +02:00
Artturin
238a6053c4 stdenv: support opt-in __structuredAttrs
Co-authored-by: Robin Gloster <mail@glob.in>

stdenv: print message if structuredAttrs is enabled

stdenv: add _append

reduces the chance of a user doing it wrong

fix nix develop issue

output hooks don't work yet in nix develop though

making $outputs be the same on non-structuredAttrs and structuredAttrs
is too much trouble.

lets instead make a function that gets the output names

reading environment file '/nix/store/2x7m69a2sm2kh0r6v0q5s9z1dh41m4xf-xz-5.2.5-env-bin'
nix: src/nix/develop.cc:299: std::string Common::makeRcScript(nix::ref<nix::Store>, const BuildEnvironment&, const Path&): Assertion `outputs != buildEnvironment.vars.end()' failed.

use a function to get all output names instead of using $outputs

copy env functionality from https://github.com/NixOS/nixpkgs/pull/76732/commits
2022-12-08 06:13:19 +02:00
Artturi
df3056bfac
Merge pull request #176597 from amjoseph-nixpkgs/pr/remove-redundant-file-buildInput
stdenv/cross: remove now-redundant `file` nativeBuildInput on mingw
2022-11-21 19:22:16 +02:00
github-actions[bot]
77fbd162df
Merge staging-next into staging 2022-11-20 18:01:45 +00:00
Artturin
341e6fd558 splice.nix: start deprecating nativeDrv and crossDrv 2022-11-19 00:04:54 +02:00
Robert Hensing
51518a5fd3 stdenv.tests: Add succeedOnFailure 2022-11-14 19:03:35 +01:00
Sandro Jäckel
9f4143e964
stdenv: fix succeedOnFailure 2022-11-12 19:25:14 +01:00
github-actions[bot]
f52955a521
Merge master into staging-next 2022-10-16 18:01:42 +00:00
piegames
87d738e864
Merge #195120: check-meta.nix: fix checkMetaRecursively option 2022-10-16 15:33:16 +02:00
github-actions[bot]
4f07e7f326
Merge master into staging-next 2022-10-15 06:07:54 +00:00
Robert Scott
692512da6a
Merge pull request #193415 from risicle/ris-nixpkgs-allow-nonsource-fix-1
stdenv/check-meta: fix support for `NIXPKGS_ALLOW_NONSOURCE=1`
2022-10-15 01:25:44 +01:00
arcnmx
466fd1439f check-meta.nix: fix checkMetaRecursively option
In specific cases, combining the `checkMeta` and `checkMetaRecursively`
config options would result in `error: infinite recursion encountered`

fixes #193296
2022-10-13 14:25:57 -07:00
Franz Pletz
3624ac2458
perl: fix build with libxcrypt 2022-10-09 18:07:54 +02:00
Martin Weinelt
253ca4957d Merge remote-tracking branch 'origin/master' into staging-next 2022-10-05 00:44:16 +02:00
Artturin
30cc0cde34 stdenv: complete the deprecation of stdenv.glibc 2022-10-03 20:19:54 +03:00
Robert Scott
d02ac63f4f stdenv/check-meta: fix support for NIXPKGS_ALLOW_NONSOURCE=1 2022-09-28 23:18:33 +01:00
github-actions[bot]
f18d801779
Merge staging-next into staging 2022-09-28 18:05:53 +00:00
aszlig
8f98a6d39b check-meta: Add isHydraChannel
This is needed in order to mark a certain derivation containing a Nix
expression tarball to Hydra so that it is recognised as a channel.

When I first got an evaluation error due to using this meta attribute, I
was under the impression that nobody outside of Vuizvui[1] is using this
feature and that we don't have any occurrence of isHydraChannel in
Nixpkgs.

However, when working around[2] the issue I assumed that it's not
something that should be included in Nixpkgs because we're not using it
there.

It turned out that my assumption was wrong and we *do* use the attribute
in Nixpkgs, namely via releaseTools.channel, which is similar to what
we're doing in Vuizvui.

Since we already include a bunch of undocumented attributes in
metaTypes, it only makes sense to add isHydraChannel as well since it's
actually documented in the Hydra documentation[3].

[1]: https://github.com/openlab-aux/vuizvui
[2]: https://github.com/openlab-aux/vuizvui/commit/e0685e81b3fdc43a272f0
[3]: 53335323ae/doc/manual/src/jobs.md (meta-fields)

Signed-off-by: aszlig <aszlig@nix.build>
2022-09-28 14:12:45 +02:00
github-actions[bot]
e0bea8e017
Merge staging-next into staging 2022-09-26 00:03:58 +00:00
piegames
6762de9a28 check-meta.nix: type checking changes
- Enable metadata checking by default, see https://github.com/NixOS/nixpkgs/pull/25304#issuecomment-298385426
- Check metadata before any other package issues, see https://github.com/NixOS/nixpkgs/issues/191124#issuecomment-1246523976
- Document that type checks only apply to the top level of nested values.

Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2022-09-25 16:37:15 +02:00
Graham Christensen
c2b898da76 treewide: drop -l$NIX_BUILD_CORES
Passing `-l$NIX_BUILD_CORES` improperly limits the overall system load.

For a build machine which is configured to run `$B` builds where each
build gets `total cores / B` cores (`$C`), passing `-l $C` to make will
improperly limit the load to `$C` instead of `$B * $C`.

This effect becomes quite pronounced on machines with 80 cores, with
40 simultaneous builds and a cores limit of 2. On a machine with this
configuration, Nix will run 40 builds and make will limit the overall
system load to approximately 2. A build machine with this many cores
can happily run with a load approaching 80.

A non-solution is to oversubscribe the machine, by picking a larger
`$C`. However, there is no way to divide the number of cores in a way
which fairly subdivides the available cores when `$B` is greater than
1.

There has been exploration of passing a jobserver in to the sandbox,
or sharing a jobserver between all the builds. This is one option, but
relatively complicated and only supports make. Lots of other software
uses its own implementation of `-j` and doesn't support either `-l` or
the Make jobserver.

For the case of an interactive user machine, the user should limit
overall system load using `$B`, `$C`, and optionally systemd's
cpu/network/io limiting features.

Making this change should significantly improve the utilization of our
build farm, and improve the throughput of Hydra.
2022-09-22 16:01:23 -04:00
Stéphan Kochen
6515b56871
darwin.apple_sdk_11_0: use stdenv objc4
Stdenv on aarch64-darwin pulls in (bootstrap-stage4) objc4, unlike
x86_64. However derivations that otherwise depend on objc4 would use a
a different objc4 derivation on top of the final stdenv.

Because this library defines an LLVM module, having multiple instances
of it in the import path will interfere with builds.
2022-09-20 12:14:49 +02:00
Vladimír Čunát
f08ac2c92b
Merge branch 'master' into staging-next 2022-08-29 08:58:52 +02:00
pennae
abd2e87da1 stdenv: remove deprecated adapters 2022-08-26 16:25:44 +02:00
Adam Joseph
51ba233717 pkgs/stdenv/linux: bootstrap-files for third (and final) mips ABI: n32
This PR provides the Hydra-generated bootstrap tarballs for mips64el-linux-gnuabin32.

With this PR we now have the bootstrap-file for all three little-endian ABIs on mips: n64, n32, and o32.  I do not currently plan to do big-endian mips unless some motivation arises; all mips chips are bi-endian and Debian has dropped big-endian support due to lack of interest.

I'll be following the script used in #151399, #168199, and #183487.

Files came from [this](https://hydra.nixos.org/build/188389586#tabs-summary) Hydra build, which used nixpkgs revision 97d9c84e1d to instantiate:

```
/nix/store/hakn8s85s9011v61r6svp5qy8x1y64fv-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32.drv
```

and then built:

```
/nix/store/rjgybpnf3yiqyhvl2n2lx31jf800fii2-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32
```

I downloaded these files from Hydra and prefetched them into the nix store with the following commands:

```
STOREPATH=rjgybpnf3yiqyhvl2n2lx31jf800fii2-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32
OPTIONS="--option binary-caches https://cache.nixos.org --option trusted-public-keys cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
nix store prefetch-file \
  file://$(nix store add-file --name bootstrap-tools.tar.xz  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz)
nix store prefetch-file --executable \
  file://$(nix store add-path --name busybox $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox)
```

These commands produced the following output:

```
Downloaded 'file:///nix/store/w6zzd2fx2vhmjfcf5h5zc01m0swldpbw-bootstrap-tools.tar.xz' to '/nix/store/6w0f0mqblrghvh6yjwcb4xdqq9x50lbl-w6zzd2fx2vhmjfcf5h5zc01m0swldpbw-bootstrap-tools.tar.xz' (hash 'sha256-LWrpN6su2yNVurUyhZP34OiZyzgh7MfN13fIIbou8KI=').
Downloaded 'file:///nix/store/nqagw1kgdz1zlmqi00qfjrmwqk3g3bgd-busybox' to '/nix/store/i361xhbdhhnvg7zd637xpm63vbl80s0s-nqagw1kgdz1zlmqi00qfjrmwqk3g3bgd-busybox' (hash 'sha256-4N3G1qYA7vitjhsIW17pR6UixIuzrq4vZXa8F0/X4iI=').
```

I used the hashes from the output above to create the `fetchurl` invocation which is part of this commit.

I then started the bootstrap with the following command:

```
nix build -L -f . --arg localSystem '(import ./lib).systems.examples.mips64el-linux-gnuabin32' hello
```

As @lovesegfault requested, here are the the `sha256sum`s of all the `on-server` components for extra verification:

```
sha256sum /nix/store/${STOREPATH}/on-server/*
```

which produced the following output:

```
2d6ae937ab2edb2355bab5328593f7e0e899cb3821ecc7cdd777c821ba2ef0a2  /nix/store/rjgybpnf3yiqyhvl2n2lx31jf800fii2-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32/on-server/bootstrap-tools.tar.xz
01633f71135cb9ab1b5ce3ebb67e80cbf288739729bffc1350c1552f6f8df34b  /nix/store/rjgybpnf3yiqyhvl2n2lx31jf800fii2-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabin32/on-server/busybox
```
2022-08-25 16:13:37 -07:00
github-actions[bot]
0b8aa77df1
Merge master into staging-next 2022-08-25 18:01:10 +00:00
Bernardo Meurer
14181ab152
Merge pull request #185182 from amjoseph-nixpkgs/pr/powerpc64le-bootstrapFiles/again
pkgs/stdenv/linux: update powerpc64le bootstrap-files
2022-08-25 10:07:22 -03:00
Matthew Bauer
ec4019f6dc Set inherit_errexit after bash version check
inherit_errexit wasn’t available in bash 3. We have a check to show a
nice error message, but that check is after we set inherit_errexit in
setup.sh. So we can just move this to below the BASH_VERSINFO check.
2022-08-21 23:01:55 -04:00
Sergei Trofimovich
a0219e7fb7
Merge pull request #181536 from trofi/fix-gnumake-guile
gnumake: unconditionally disable guileSupport on bootstrap
2022-08-19 19:22:13 +01:00
Sergei Trofimovich
7f92320f66 make-bootstrap-tools.nix: drop libelf.so from tootstrap tarballs
gcc stopped using libelf in commit 48215350c24 ("re PR lto/46273 (Failed
to bootstrap)") around 2010, before gcc-4.6.0.

Bootstrap tools don't use it either.
2022-08-18 08:23:08 +01:00
Bernardo Meurer
d00f5c5eaa
Merge pull request #183487 from amjoseph-nixpkgs/pr/mipsel/bootstrap-files
stdenv/linux/bootstrap-files: add mipsel-linux (mips32)
2022-08-17 15:14:47 -07:00
Adam Joseph
6122297eab pkgs/stdenv/linux: update powerpc64le bootstrap-files
This PR updates the Hydra-generated bootstrap tarballs for
powerpc64le-linux.  The bootstrap-files referenced prior to this
commit will only bootstrap in a nixpkgs which has
https://github.com/NixOS/nixpkgs/pull/181802.  That PR was closed in
favor of https://github.com/NixOS/nixpkgs/pull/181943, which is a
cleaner solution but which requires regenerating the bootstrap-files.

I'll be following the script established in
https://github.com/NixOS/nixpkgs/pull/151399, which I previously used
in https://github.com/NixOS/nixpkgs/pull/168199.

Files came from [this](https://hydra.nixos.org/build/186237511) Hydra build, which used nixpkgs revision ac43c44478 to instantiate:

```
/nix/store/nhjbza9vlcyhp9zxfz6lwpc3m2ghrpzj-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu.drv
```

and then built:

```
/nix/store/fklpm7fy6cp5wz55w0gd8wakyqvzapjx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu
```

I downloaded these files from Hydra with the following commands:

```
STOREPATH=fklpm7fy6cp5wz55w0gd8wakyqvzapjx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu
OPTIONS="--option binary-caches https://cache.nixos.org --option trusted-public-keys cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
nix store add-file \
  --name bootstrap-tools.tar.xz  \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz
nix store add-path \
  --name busybox \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox
```

I then prefetched them into `/nix/store` with:

```
$ nix store prefetch-file --executable file:///nix/store/p9lz8r81zp3a4sl2qq2v4j69syjzryn2-busybox
Downloaded 'file:///nix/store/p9lz8r81zp3a4sl2qq2v4j69syjzryn2-busybox' to '/nix/store/a42qf2kf5hychcsw5sz0pvghy9vli1im-p9lz8r81zp3a4sl2qq2v4j69syjzryn2-busybox' (hash 'sha256-jtPEAsht4AUAG4MLK8xocQSfveUR4ppU1lS4bGI1VN4=').

$ nix store prefetch-file file:///nix/store/y4530zpk7ia4szf5cdi4zpyy5lpjv3iv-bootstrap-tools.tar.xz
Downloaded 'file:///nix/store/y4530zpk7ia4szf5cdi4zpyy5lpjv3iv-bootstrap-tools.tar.xz' to '/nix/store/kgzyq9q08nll28ccqjcbv8angq5hyvdp-y4530zpk7ia4szf5cdi4zpyy5lpjv3iv-bootstrap-tools.tar.xz' (hash 'sha256-MpIDnpZUK3M17qlnuoxfnK0EgxRosm3TMW1WfPZ1+jU=').
```

And started the bootstrap with the following command:

```
nix build -f . -L hello
```

As @lovesegfault requested in #151399, here are the the `sha256sum`s of all the `on-server` components for extra verification:

```
$ sha256sum /nix/store/fklpm7fy6cp5wz55w0gd8wakyqvzapjx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/*
3292039e96542b7335eea967ba8c5f9cad04831468b26dd3316d567cf675fa35  /nix/store/fklpm7fy6cp5wz55w0gd8wakyqvzapjx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/bootstrap-tools.tar.xz
3d078dff7b4087d82442937667c91dace3321493aae4d3a4160d046b7eabcc2c  /nix/store/fklpm7fy6cp5wz55w0gd8wakyqvzapjx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/busybox
```
2022-08-04 23:05:31 -07:00
github-actions[bot]
2a54df0946
Merge staging-next into staging 2022-08-03 18:02:18 +00:00
Sandro
0e0cc47192
Merge pull request #181550 from reckenrode/update-darwin-bootstrap-tools 2022-08-03 16:12:31 +02:00
Sandro
09cb5e8416
Merge pull request #183507 from Artturin/stdenvmeson1 2022-08-01 23:06:51 +02:00
github-actions[bot]
e8ce2f4a26
Merge master into staging-next 2022-08-01 18:01:20 +00:00
Artturin
9f309c8792 stdenv: mesonFlags: use canExecute in needs_exe_wrapper
this is more accurate
2022-08-01 20:03:47 +03:00
Robert Hensing
12a060dbbc
Merge pull request #183967 from Ericson2314/small-stdenv-cleanup
stdenv: Two small cleanups
2022-08-01 14:37:42 +02:00
github-actions[bot]
15686bdd94
Merge master into staging-next 2022-07-30 18:01:31 +00:00
Alyssa Ross
5330c0a1af
treewide: use isAarch where appropriate 2022-07-30 16:18:27 +00:00
John Ericson
650380ca0b stdenv: Simplify dispatch
This is a change for `powerpc-linux` but that is ancient and I don't
think it matters. The impure bootstrap that was previously assigned to
it has probably bitrotted anyways.
2022-07-29 18:34:02 -04:00
John Ericson
9d59a4df78 stdenv/common-path.nix: move to stdenv/generic/
Do this since it is part of the generic stdenv/`mkDerivation`
infrastructure, rather than being a bootstrapping strategy.
2022-07-29 18:24:00 -04:00
Sergei Trofimovich
f6ee60dde4 gnumake: unconditionally disable guileSupport on bootstrap
Before the change an attempt to use `gnumake.override { guileSupport = true; }`
caused recursion in bootstrap stages as guileSupport pulls in guile and it's
dependencies.

To restore the bootstrap the change unconditionally sets
`guileSupport = false;` for `gnumake`.

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2022-07-29 19:34:09 +01:00
Adam Joseph
efb35dff8a stdenv/linux/bootstrap-files: add mipsel-linux (mips32) 2022-07-28 14:23:11 -07:00
github-actions[bot]
9153131664
Merge staging-next into staging 2022-07-27 12:02:00 +00:00
github-actions[bot]
9ed58a6fb6
Merge master into staging-next 2022-07-27 12:01:18 +00:00
Vladimír Čunát
a0e2bdeb84
Merge #179628: make-bootstrap-tools-cross: add mips32el 2022-07-27 13:12:10 +02:00
Artturi
79e8669003
Merge pull request #179603 from Artturin/subplacemultiple 2022-07-24 04:14:09 +03:00
Artturi
111abd87ca
Merge pull request #168590 from amjoseph-nixpkgs/stdenv-disallowedReferences
stdenv: use disallowedRequisites to check forbidden requisites
2022-07-18 02:51:23 +03:00
github-actions[bot]
8df1eb061a
Merge master into staging-next 2022-07-17 00:02:14 +00:00
Bernardo Meurer
5fed5fe917
Merge pull request #180223 from amjoseph-nixpkgs/pr/mips64el/bootstrap-file-hashes
pkgs/stdenv/linux: add mips64el bootstrap-files
2022-07-16 16:41:08 -07:00
Randy Eckenrode
8d1c1184e6
make-bootstrap-tools: add pbzx and tbd tools on x86_64-darwin 2022-07-14 21:07:28 -04:00
Artturin
ca2ca8b194 stdenv: start deprecating non-list mesonFlags
the motivation for this is to simplify stdenv and ease the job of
reviewers due to them needing to tell contributors about the defacto
rule that mesonFlags should be a list of strings
2022-07-06 15:23:20 +03:00
Adam Joseph
73811b372e pkgs/stdenv/linux: add mips64el bootstrap-files
This PR adds Hydra-generated bootstrap tarballs for mips64el-linux.  I'll be following the script established in https://github.com/NixOS/nixpkgs/pull/151399, which I previously used in https://github.com/NixOS/nixpkgs/pull/168199.

Files came from [this](https://hydra.nixos.org/build/182757245) Hydra build, which used nixpkgs revision ef3fe254f3 to instantiate:

```
/nix/store/a2bvv663wjnyhq8m7v84aspsd3sgf9h6-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64.drv
```

and then built:

```
/nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64
```

I downloaded these files from Hydra with the following commands:

```
STOREPATH=aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64
OPTIONS="--option binary-caches https://cache.nixos.org"
nix store add-file \
  --name bootstrap-tools.tar.xz  \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz
nix store add-path \
  --name busybox \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox
```

I then prefetched them into `/nix/store` with:

```
$ nix store prefetch-file --executable file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox
Downloaded 'file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox' to '/nix/store/ai30ss23914syz6j8m95arkwffbbx44k-busybox' (hash 'sha256-sTE58ofjqAqX3Xtq1g9wDxzIe6Vo//GHbicfqJoivDI=').

$ nix store prefetch-file  file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz
Downloaded 'file:///nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz' to '/nix/store/nr6zjrbwbxcxs6brf042zsyqllwbwj9v-bootstrap-tools.tar.xz' (hash 'sha256-tTgjeXpd2YgnfP4JvRuO0bXd2j8GqzBcd57JI3wH9x0=').
```

And started the bootstrap with the following command (the `--arg localSystem` is needed because #161159 has not merged):

```
nix build -f . -L hello --arg localSystem '(import ./lib).systems.examples.mips64el-linux-gnuabi64'
```

As @lovesegfault requested in #151399, here are the the `sha256sum`s of all the `on-server` components for extra verification:

```
$ sha256sum /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/*
b53823797a5dd988277cfe09bd1b8ed1b5ddda3f06ab305c779ec9237c07f71d  /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/bootstrap-tools.tar.xz
598e05abb69b2c1a0db46585cd2131212077c0937ce2a665daf3811f059ae767  /nix/store/aw3dmsrh22831l83vi3q9apg9qi3x8ms-stdenv-bootstrap-tools-mips64el-unknown-linux-gnuabi64/on-server/busybox
```
2022-07-05 12:16:58 -07:00
Artturin
3173c3b6b6 stdenv: start deprecating non-list cmakeFlags
the motivation for this is to simplify stdenv and ease the job of
reviewers due to them needing to tell contributors about the defacto
rule that cmakeFlags should be a list of strings
2022-07-05 04:05:30 +03:00
Adam Joseph
dc5bdd6852 Revert "stdenv: label the ephemeral coreutils-stage4 package"
This reverts commit 23ea8b35da.
2022-07-02 16:56:08 -07:00
Rick van Schijndel
2fcdf54568
Merge pull request #169378 from amjoseph-nixpkgs/stdenv-makeStaticLibraries-do-what-comment-says
stdenv: make stage3.{gmp,mpfr,mpc,isl} do what the comment says
2022-06-30 07:44:05 +02:00
Adam Joseph
ea32f3e523 make-bootstrap-tools-cross: enable Hydra builds of mips32el bootstrap-files
This one-line commit will cause Hydra to build a bootstrap-files
tarball for 32-bit little-endian MIPS.
2022-06-29 14:48:13 -07:00
Artturin
ba1efa71ae stdenv: substituteInPlace: accept multiple filenames
I don't know if getopt is available everywhere, so I did not use it.

in any case, it can be changed to use getopt in the future if
needed.
2022-06-29 22:03:19 +03:00
github-actions[bot]
5b9ee17283
Merge master into staging-next 2022-06-24 00:02:29 +00:00
Sergei Trofimovich
b0f5f5cc96
Merge pull request #178408 from trofi/add-config.enableParallelBuildingByDefault
config.configurePlatformsByDefault: init option
2022-06-23 21:32:59 +00:00
Rick van Schijndel
adafa1cb42
Merge pull request #178626 from a-m-joseph/pr/168413fallout/mktemp
setup.sh: use six `X` characters in `mktemp` invocation
2022-06-23 06:45:40 +02:00
Rick van Schijndel
f104ffc941
Merge pull request #177789 from a-m-joseph/pr/setup/busyboxfind
setup.sh: use -exec rather than -execdir
2022-06-23 06:40:38 +02:00
Adam Joseph
97bf76b3b3 setup.sh: use six X characters in mktemp invocation
Closes #178625

The `busybox` version of `mktemp` requires exactly six `X` characters
in the argument to `mktemp`, unlike the `coreutils` version of `mktemp`.

Let's accomodate packages, like `epson-escpr2`, which fool `setup.sh`
into using the `busybox` version instead of the `stdenv` version.
2022-06-22 12:46:22 -07:00
github-actions[bot]
30c9243acc
Merge staging-next into staging 2022-06-22 00:03:15 +00:00
Sergei Trofimovich
c534d056fe pkgs/stdenv/generic/make-derivation.nix: add a bug reference to strictDepsByDefault TODO 2022-06-21 19:03:43 +01:00
Sergei Trofimovich
db09d70d33 config.configurePlatformsByDefault: init option
Useful to enable tree-wide occasionally to have incremental progress
towards https://github.com/NixOS/nixpkgs/pull/87909 resolution.
2022-06-21 09:23:13 +01:00
Vladimír Čunát
5ffd19ddbf
Merge #175785: libidn2: hack to avoid referencing bootstrap tools
...into staging
2022-06-17 18:23:25 +02:00
Adam Joseph
64fc13ff1d setup.sh: use -exec rather than -execdir
Busybox's `find` does not support `-execdir`, so let's uses `-exec`
instead, in the spirit of requiring the minimum features needed for
the task.
2022-06-15 12:10:44 -07:00
github-actions[bot]
caada97a27
Merge master into staging-next 2022-06-14 00:02:11 +00:00
Robert Schütz
38c776b679 stdenv/check-meta: support NIXPKGS_ALLOW_NONSOURCE=0 2022-06-13 19:08:17 +00:00
Martin Weinelt
6451e9d4a5 Merge remote-tracking branch 'origin/staging-next' into staging 2022-06-08 22:22:26 +02:00
Alyssa Ross
5dfe7f928e
release-cross.nix: fix cross bootstrap tools eval
In 5643714dea, I changed pkgs/stdenv/linux/make-bootstrap-tools.nix
to take a package set instead of system and localSystem arguments, but
I forgot to update make-bootstrap-tools-cross.nix.

Fixes: 5643714dea ("stdenvBootstrapTools: inherit {cross,local}System")
2022-06-08 17:17:10 +00:00
Adam Joseph
10030672ab stdenv/cross: remove now-redundant file nativeBuildInput on mingw
Since 97c43828fb the `file` package has
been part of stdenv, and no longer needs to be listed explicitly as a
build input.  Let's remove the platform-specific inclusion for mingw64
as suggested by @mehmooda:

  https://github.com/NixOS/nixpkgs/pull/168413#issuecomment-1147370500

I traced the line removed by this commit through the `git blame`; it
was initially added in this commit (and then shuffled around a few
dozen times by refactorings):

  8b292a1b35

The commit message indicates that `libpng-1.6.20` was current at the
time.  Although there are [libpng
archives](https://github.com/glennrp/libpng) available in git form,
the older versions don't have their autoconfery vendored in, so I
can't link to them.  Fortunately the relevant bit hasn't changed since
then:

a37d483651/configure (L5575)

```
mingw* | pw32*)
  # Base MSYS/MinGW do not provide the 'file' command needed by
  # func_win32_libid shell function, so use a weaker test based on 'objdump',
  # unless we find 'file', for example because we are cross-compiling.
  if ( file / ) >/dev/null 2>&1; then
    lt_cv_deplibs_check_method='file_magic ^x86 archive import|^x86 DLL'
    lt_cv_file_magic_cmd='func_win32_libid'
  else
    # Keep this pattern in sync with the one in func_win32_libid.
    lt_cv_deplibs_check_method='file_magic file format (pei*-i386(.*architecture: i386)?|pe-arm-wince|pe-x86-64)'
    lt_cv_file_magic_cmd='$OBJDUMP -f'
  fi
  ;;
```
2022-06-06 11:26:46 -07:00
github-actions[bot]
a1bbd60cb7
Merge staging-next into staging 2022-06-05 18:01:45 +00:00
John Ericson
82707a2593
Merge pull request #176385 from hercules-ci/fix-overlay-style-stdenv-adapters
stdenv/adapters.nix: Fix for overlay style arguments
2022-06-05 12:29:24 -04:00
Robert Hensing
dd770cc211 stdenv/adapters.nix: Fix for overlay style arguments 2022-06-05 13:54:03 +02:00
Robert Hensing
cd88f8613f pkgs/make-derivation.nix: Refactor, hardcode mkDerivationSimple 2022-06-05 13:36:56 +02:00
Robert Hensing
7e3c80f5b7 pkgs/make-derivation.nix: Refactor, inline makeOverlayable 2022-06-05 13:35:04 +02:00
Robert Hensing
1b00adbad1 pkgs/make-derivation.nix: Refactor, introduce let binding 2022-06-05 13:33:35 +02:00
Adam Joseph
02630180fa stdenv: add -stageX markers to gmp, mpfr, libmpc, and isl 2022-06-05 00:35:06 -07:00
Adam Joseph
a9e0d86411 Update pkgs/stdenv/linux/default.nix
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
2022-06-05 00:35:06 -07:00
Adam Joseph
23ea8b35da stdenv: label the ephemeral coreutils-stage4 package
During stdenv bootstrapping, coreutils is built twice.  This makes
troubleshooting very difficult, because both packages have
name="coreutils", so it is a hassle to figure out "which coreutils am
I using / is not building"?

The first of these builds is used only in stage4, and is not part of
the final stdenv.  Let's label that one with a different `name`
attribute to make it obvious which is which.
2022-06-05 00:35:06 -07:00
Adam Joseph
122b6930b0 stdenv: cause makeStaticLibraries usage to agree with usage spec
The usage of `makeStaticLibraries` in stdenv/linux/default.nix is
prefaced by this comment:

  # Link GCC statically against GMP etc.  This makes sense because
  # these builds of the libraries are only used by GCC, so it
  # reduces the size of the stdenv closure.

However "these builds of the libraries are only used by GCC" is not
actually true.  As currently written, the stage4 coreutils links
against these customized, static-ified libraries.

Beside the fact that the code doesn't actually do what it says, this
causes other problems as well.  One example is #168983, which arises
because have a dynamically-linked binary (coreutils) which is built
from statically-linked libraries (libgmp.a); doing this causes mayhem
on platforms where `-fstack-protector` needs an auxiliary
`libssp.{so,a}` library; we end up with link failures because some
parts of the resulting binary want `libssp.so` and other parts want
`libssp_nonshared.a`.

Let's make the code actually do what the comment says, by moving these
definitions into the `gcc-unwrapped` override.  This will cause the
stage4-coreutils to link against libgmp dynamically, rather than
statically.  For this reason this commit depends on the previous
commit, which allows that to be done without creating a forbidden
reference from stdenv-final to the bootstrap-files.
2022-06-05 00:35:06 -07:00
Adam Joseph
7fd749009f stdenv: force gmp to rebuild in stage4 of the bootstrap
As explained in the comment, this ensures that stage4-coreutils does
not leak a reference to the bootstrap-files by way of libgmp.  This
will allow the next patch in this series to build stage4-coreutils
using a dynamically-linked (rather than statically-linked) libgmp.
2022-06-05 00:35:00 -07:00
Vladimír Čunát
088b29159d
libidn2: hack to avoid referencing bootstrap tools
Due to bootstrap tools getting purged from closure of libidn2.dev,
a very large rebuild is caused.
2022-06-05 08:59:20 +02:00
github-actions[bot]
dc4d841894
Merge staging-next into staging 2022-06-02 00:03:03 +00:00
github-actions[bot]
d6b9d24302
Merge master into staging-next 2022-06-02 00:02:21 +00:00
Sandro
4c3e9f091a
check-meta: fix comment (#175517)
* check-meta: fix comment

* Apply suggestions from code review
2022-06-02 01:38:57 +02:00
github-actions[bot]
e8d5ce419a
Merge staging-next into staging 2022-05-31 18:01:50 +00:00
github-actions[bot]
d6ede66f6a
Merge master into staging-next 2022-05-31 18:01:11 +00:00
Alyssa Ross
5643714dea stdenvBootstrapTools: inherit {cross,local}System
It's expected that attributes in the top-level package set will all use
that package set, but this wasn't the case for the bootstrap tools.
This led some very confusing behaviour:

- pkgsMusl.stdenvBootstrapTools would build glibc bootstrap tools
- stdenvBootstrapTools was _always_ cross compiled, even if
  Nixpkgs wasn't, because it always set crossSystem.  This also didn't
  match the behaviour of using make-bootstrap-tools.nix as an
  entrypoint, where crossSystem would default to null.

For the Linux stdenv, I've made the ideal fix, which is to make pkgs an
argument rather than taking the arguments for pkgs, and then
re-importing it.  This means it'll always use exactly the same package
set that's calling it, and should also mean faster eval due to not
importing Nixpkgs twice.

The Darwin stdenv is more complicated, and I'm not able to easily test
it, so I wasn't confident in making the same fix there.  Instead, I've
just made sure crossSystem and localSystem are set to the correct values
so they're not always cross compiled and match the parent package set's.
It would still be preferable if somebody could make Darwin's
make-bootstrap-tools.nix take pkgs as an argument, rather than all the
arguments for pkgs.
2022-05-31 14:32:27 +00:00
github-actions[bot]
2d502cf3a4
Merge staging-next into staging 2022-05-30 12:02:11 +00:00
github-actions[bot]
76b451751c
Merge master into staging-next 2022-05-30 12:01:31 +00:00
Robert Scott
5bb9bf4774 meta.sourceProvenance: inline hasSourceProvenance
it may be what the license handling code does, but it's confusing and not very useful

Co-authored-by: Adam Joseph <54836058+a-m-joseph@users.noreply.github.com>
2022-05-30 16:27:34 +08:00
Robert Scott
7906ea6d9d allowNonSourcePredicate: use example of categorical permissivity
Co-authored-by: Adam Joseph <54836058+a-m-joseph@users.noreply.github.com>
2022-05-30 16:27:34 +08:00
Robert Scott
095eb91533 meta.sourceProvenance: disallow string values
strings complicate reasoning about values and may not be needed with `sourceProvenance`

Co-authored-by: Alexander Foremny <aforemny@posteo.de>
2022-05-30 16:27:34 +08:00
Robert Scott
da9162f667 add mechanism for handling meta.sourceProvenance attributes
heavily based on patterns used by licenses infrastructure, so may
appear overengineered for its initial level of use
2022-05-30 16:27:34 +08:00
Rick van Schijndel
f174277eed
Merge pull request #168413 from a-m-joseph/libtool-purity-fix-put-file-in-bootstrap
fixLibtool(): replace /usr/bin/file in ./configure, add file to common-path.nix
2022-05-28 17:53:43 +02:00
github-actions[bot]
ff43d848bf
Merge staging-next into staging 2022-05-27 18:01:52 +00:00
Artturi
60a0871486
Merge pull request #172391 from Artturin/deprecate-stdenv-glibc 2022-05-27 17:56:19 +03:00
Alyssa Ross
089ff89f49
lib.systems: drop scaleway-c1
These servers apparently no longer exist, since September 2, 2021[1].
If somebody needs this for non-Scaleway machines, they should suggest
its reintroduction with a different name.

[1]: https://news.ycombinator.com/item?id=27192757
2022-05-27 12:02:39 +00:00
Adam Joseph
97c43828fb fixLibtool(): patch ./configure, add file to common-path.nix
libtool's libtool.m4 script assumes that `file` is available, and can
be found at `/usr/bin/file` (this path is hardwired).  Furthermore,
the script with this assumption is vendored into the ./configure
scripts of an enormous number of packages.  Without this commit, you
will frequently see errors like this during the configurePhase with
the sandbox enabled:

  ./configure: line 9595: /usr/bin/file: command not found

Due mostly to luck, this error does not affect native compiles on
nixpkgs' two most popular platforms, x86_64-linux and aarch64-linux.
However it will cause incorrect linker flag detection and a failure to
generate shared libraries for sandboxed cross-builds to a x86_64-linux
host as well as any sandboxed build (cross or native) for the following
hosts: x86_64-freebsd, *-hpux, *-irix, mips64*-linux, powerpc*-linux,
s390x-linux, s390x-tpf, sparc-linux, and *-solaris.

This commit fixes the problem by adding an extra line to fixLibtool()
in pkgs/stdenv/generic/setup.sh.  This extra line will scan the
unpacked source code for executable files named "configure" which
contain the following text:

'GNU Libtool is free software; you can redistribute it and/or modify'

This text is taken to be an indicator of a vendored libtool.m4.  When
it is found, the configure script containing it is subjected to `sed
-i s_/usr/bin/file_file_` which replaces all occurrences of
`/usr/bin/file` with `file`.

Additionally, the `file` package is now considered to be part of
`stdenv`.  It has been added to `common-path.nix` so that the `file`
binary will be found in the `$PATH` of every build, except for the
bootstrap-tools and the first few stages of stdenv boostrapping.

Verified no regressions under:

  nix-build --arg pkgs 'import ./. {}' ./lib/tests/release.nix

This commit allows the following commands to complete, which should
enable Hydra to produce bootstrap-files for mips64el:

  nix-build \
    --option sandbox true \
    --option sandbox-fallback false \
    pkgs/top-level/release-cross.nix \
    -A bootstrapTools.mips64el-linux-gnuabi64.build

  nix-build \
    --option sandbox true \
    --option sandbox-fallback false \
    . \
    -A pkgsCross.mips64el-linux-gnuabi64.nix_2_4
2022-05-26 13:53:36 -07:00
Artturin
1d44ac176c treewide: add enableParallelBuilding's to bootstrap packages so hashes stay the same
when enableParallelBuildingByDefault is enabled

verified with
`nix-diff $(nix eval ".#gcc-unwrapped.drvPath") $(nix eval --expr 'with import ./. { config = { enableParallelBuildingByDefault = true; }; }; gcc-unwrapped.drvPath' --impure)`
2022-05-25 16:03:14 +03:00
Artturin
0c4d65b21e treewide: stdenv.glibc -> glibc 2022-05-25 15:51:20 +03:00
Artturin
a05b581783 stdenv: warn about use of stdenv.glibc
TODO was added in
119920faa6
2022-05-25 15:51:20 +03:00
github-actions[bot]
574f2d4696
Merge staging-next into staging 2022-05-24 00:02:57 +00:00
sternenseemann
41485e7337 stdenv.mkDerivation: be less strict about check execution for cross
Instead of requiring the platforms be equal, use `isCompatible` to
determine if we can execute tests. The upside of this is that we now
can execute tests for natively cross compiled package sets like
pkgsStatic, pkgsLLVM and pkgsCross.musl64 etc.
2022-05-23 21:25:04 +02:00
sternenseemann
acb063701a lib.systems.elaborate: expose canExecute predicate over isCompatible
canExecute is like isCompatible, but also checks that the Kernels are
_equal_, i.e. that both platforms use the same syscall interface. This
is crucial in order to actually be able to execute binaries for the
other platform.

isCompatible is dropped, since it has changed semantically and there's
no use case left in nixpkgs.
2022-05-23 21:25:04 +02:00
Artturin
f002ffed9a treewide: enable strictDeps in bootstrap packages 2022-05-22 16:40:26 +03:00
Artturin
6d54fe622d stdenv: start deprecating non-list configureFlags
the motivation for this is to simplify stdenv and ease the job of
reviewers due to them needing to tell contributors about the defacto
rule that configureFlags should be a list of strings
2022-05-18 20:32:33 +03:00
Artturi
22f6a17b22
Merge pull request #172570 from Artturin/epbbydefault 2022-05-13 20:46:33 +03:00
Artturin
a17e1b8809 config.enableParallelBuildingByDefault: init option
useful for testing if some builds could be faster
2022-05-11 16:41:30 +03:00
Linus Heckemann
676e4d42d6
Merge pull request #157106 from lheckemann/better-error-for-bad-buildinputs
stdenv/make-derivation: check that all inputs are of an appropriate type
2022-05-11 11:28:11 +02:00
github-actions[bot]
f39e774f11
Merge master into staging-next 2022-05-09 00:02:27 +00:00
John Ericson
db0c4f14ac
Merge pull request #172022 from elikoga/patch-2
Update link from old nix manual to current stable one
2022-05-08 18:26:19 -04:00
elikoga
04f6a432c4
Update link from old nix manual to current stable one
https://github.com/NixOS/rfcs/pull/72
2022-05-08 05:20:34 +02:00
Artturin
2d019e0931 stdenv: deprecate unused adapters
stdenv: deprecate addCoverageInstrumentation adapter

this used to be used in nixos/tests but it hasn't been used in nixpkgs
for years

stdenv: deprecate replaceMaintainersField adapter

it was added in 2009 in 01e98e49b1
by nbp
there are no uses of it in nixpkgs now

stdenv: deprecate validateLicenses adapter

it was added in 2009 in b29073af25

unfreePredicate is now handled in ./pkgs/stdenv/generic/check-meta.nix
so this is unnecessary
2022-05-07 22:36:10 +03:00
Linus Heckemann
b3e8855999 make-derivation: allow strings in build input lists 2022-05-07 10:42:50 +02:00
github-actions[bot]
16684f8bd3
Merge master into staging-next 2022-05-04 12:01:10 +00:00
Artturi
f0e48dd98b
Merge pull request #119942 from hercules-ci/mkDerivation-overridable-recursive-attributes 2022-05-04 14:31:50 +03:00
github-actions[bot]
ee7e3f30f3
Merge staging-next into staging 2022-05-04 00:02:57 +00:00
ckie
4def222ea4
stdenv/check-meta: add a "maintainerless" warning
This warning logs when a package has no maintainers. It will stay silent
if `meta.maintainers` is not set at all, only complaining when it is an
empty list. In the future a separate warning could be added to allow for
that stricter behavior. Or this warning could be changed.
2022-05-03 22:29:14 +03:00
ckie
3a34b6c820
stdenv/check-meta: add an eval warning option
This will be used in the next commit in this patch series.
2022-05-03 22:29:12 +03:00
ckie
5e420c2455
stdenv/check-meta: turn validity.valid into a str
This will allow for adding more validity types in the future, such as a
warning type. (which is in the next commit in this series)

This is NOT a breaking change because validity.valid is never exposed
outside of `stdenv.mkDerivation`.
2022-05-03 22:28:25 +03:00
github-actions[bot]
51a899bdc1
Merge staging-next into staging 2022-05-03 18:10:46 +00:00
Artturin
9f05fc6661 config.allowUnsupportedSystem: define as option 2022-05-02 20:39:43 +03:00
Artturin
9f473092f8 config.allowBroken: define as option 2022-05-02 17:20:44 +03:00
Artturin
1c49b81263 config.allowUnfree: define as option 2022-05-02 17:20:25 +03:00
Robert Hensing
0e00acafe9 stdenv.mkDerivation: public -> finalPackage 2022-05-02 08:49:33 +02:00
Robert Hensing
37ab5b4396 mkDerivation: Add error hint for infinite recursion 2022-05-02 08:49:32 +02:00
Robert Hensing
41b3688ba1 make-derivation.nix: Remove unnecessary TODO
The goal was to keep the number of calls to a minimum, but that
would come at the cost of clobbering git blame and possibly not
improving readability.
2022-05-02 08:49:31 +02:00
Robert Hensing
6d7efb3a16 stdenv.mkDerivation: Make self more overlay-like; use self.public
`self` is now arguments, like `super`. The final package is in
`self.public`.
2022-05-02 08:49:31 +02:00
Robert Hensing
a4e7085227 stdenv.mkDerivation: Allow overriding of recursive definitions
See updated manual for further explanation.
2022-05-02 08:49:26 +02:00
Ben Siraphob
02dbca8c2b
Merge pull request #161270 from Stunkymonkey/darwin-phases 2022-04-30 10:58:05 -06:00
Artturin
33415135b0 config.contentAddressedByDefault: init option 2022-04-27 23:21:32 +03:00
Artturin
d76021402d config.strictDepsByDefault: init option
so i won't have to change make-derivation.nix when i wan't to make this
true
2022-04-27 23:21:21 +03:00
github-actions[bot]
deb60260a7
Merge master into staging-next 2022-04-17 06:01:07 +00:00
Adam Joseph
9d60e3dd29 stdenv: use disallowedRequisites to check forbidden requisites 2022-04-13 23:13:33 -07:00
Adam Joseph
9d46a1025a pkgs/stdenv/linux: add powerpc64le bootstrap-files
This PR adds Hydra-generated bootstrap tarballs for powerpc64le-linux.
I'll be following the script established in PR to tarballs.nixos.org.

Files came from this Hydra build:

  https://hydra.nixos.org/build/172142499

Which used nixpkgs revision 49a83445c2
to instantiate:

  /nix/store/gj272sd56gsj6qpyzh4njpfzwdhviliz-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu.drv

and then built:

  /nix/store/n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu

I downloaded these files from Hydra with the following commands:

```
STOREPATH=n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu
OPTIONS="--option binary-caches https://cache.nixos.org"
nix store add-file \
  --name bootstrap-tools.tar.xz  \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz
nix store add-path \
  --name busybox \
  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox
```

As @lovesegfault requested in #151399, here are the the `sha256sum`s
of all the `on-server` components for extra verification:

```
$ sha256sum /nix/store/n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/*
036d062869f7accf0ad89714d12029469dfe6af504f9b226d61eb7d808ad4735  /nix/store/n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/bootstrap-tools.tar.xz
6bc05832180f5075f4458c32eb0a5e2b673f605884dce01822be474f4e0a63ee  /nix/store/n81pljbd8m0xgypm84krc2bnvqgjrfxx-stdenv-bootstrap-tools-powerpc64le-unknown-linux-gnu/on-server/busybox
```

Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2022-04-11 18:35:44 -07:00
github-actions[bot]
3dc8bd98b9
Merge staging-next into staging 2022-04-06 00:02:41 +00:00
Artturi
e2166cd22b
Merge pull request #166827 from Artturin/addtimeprint 2022-04-05 22:13:58 +03:00
Vladimír Čunát
b4729bad3d
Merge #165406: staging-next 2022-03-23 2022-04-05 20:34:08 +02:00
Robert Hensing
559ac3c9e7
Merge pull request #166383 from hercules-ci/always-sanitize-derivation-name
Always sanitize derivation name
2022-04-05 20:05:33 +02:00
Linus Heckemann
235fe92e42 make-derivation: allow nested lists in buildInputs
This isn't really desirable in general, but given that Nix itself
currently relies on this behaviour and that we don't want to break
backwards compatibility we should support it for now, maybe
deprecating it in the future.
2022-04-05 12:20:38 +02:00
Martin Weinelt
bf1914e12a Merge remote-tracking branch 'origin/staging-next' into staging 2022-04-05 04:05:02 +02:00
Vladimír Čunát
6fd0d1bb4b
stdenv bootstrap tools: adjust with glibc-2.34 (2nd try)
This time without rebuilding stdenvs.
2022-04-04 20:40:42 +02:00
Vladimír Čunát
e66ed266f8
Revert "stdenv bootstrap tools: adjust with glibc-2.34"
This reverts commit c1ef87e8d0.
I didn't realize that it would rebuild all linux stdenvs;
I certainly don't want to incur such a rebuild at this point.
2022-04-04 20:40:41 +02:00
Vladimír Čunát
c1ef87e8d0
stdenv bootstrap tools: adjust with glibc-2.34
https://hydra.nixos.org/build/171494041
2022-04-04 19:54:28 +02:00
Linus Heckemann
dbdacbca8e Merge remote-tracking branch 'upstream/nixos-unstable' into better-error-for-bad-buildinputs 2022-04-02 23:12:56 +02:00
github-actions[bot]
ffa96ab0c0
Merge staging-next into staging 2022-04-02 18:01:37 +00:00
github-actions[bot]
fc17fe6417
Merge master into staging-next 2022-04-02 18:01:07 +00:00
Linus Heckemann
93681a52a5 stdenv: check that all inputs are of an appropriate type
Fixes #24462
2022-04-02 19:09:10 +02:00
7c6f434c
8188f10752
Merge pull request #166430 from alyssais/openssl-static-retry
treewide: use lib.getLib for OpenSSL libraries
2022-04-02 12:59:55 +00:00
Artturin
1e2a288f0e stdenv: print the time the phase took if it was longer than 30s
will be useful for finding why a build runs for a long time on hydra
because of tests or the build etc etc
2022-04-02 00:42:02 +03:00
Alyssa Ross
fd78240ac8
treewide: use lib.getLib for OpenSSL libraries
At some point, I'd like to make another attempt at
71f1f4884b ("openssl: stop static binaries referencing libs"), which
was reverted in 195c7da07d.  One problem with my previous attempt is
that I moved OpenSSL's libraries to a lib output, but many dependent
packages were hardcoding the out output as the location of the
libraries.  This patch fixes every such case I could find in the tree.
It won't have any effect immediately, but will mean these packages
will automatically use an OpenSSL lib output if it is reintroduced in
future.

This patch should cause very few rebuilds, because it shouldn't make
any change at all to most packages I'm touching.  The few rebuilds
that are introduced come from when I've changed a package builder not
to use variable names like openssl.out in scripts / substitution
patterns, which would be confusing since they don't hardcode the
output any more.

I started by making the following global replacements:

    ${pkgs.openssl.out}/lib -> ${lib.getLib pkgs.openssl}/lib
    ${openssl.out}/lib -> ${lib.getLib openssl}/lib

Then I removed the ".out" suffix when part of the argument to
lib.makeLibraryPath, since that function uses lib.getLib internally.

Then I fixed up cases where openssl was part of the -L flag to the
compiler/linker, since that unambigously is referring to libraries.

Then I manually investigated and fixed the following packages:

 - pycurl
 - citrix-workspace
 - ppp
 - wraith
 - unbound
 - gambit
 - acl2

I'm reasonably confindent in my fixes for all of them.

For acl2, since the openssl library paths are manually provided above
anyway, I don't think openssl is required separately as a build input
at all.  Removing it doesn't make a difference to the output size, the
file list, or the closure.

I've tested evaluation with the OfBorg meta checks, to protect against
introducing evaluation failures.
2022-03-30 15:10:00 +00:00
Robert Hensing
7e0ef81015 stdenv.mkDerivation: Always sanitize derivation name 2022-03-30 10:29:19 +02:00
Bernardo Meurer
e2d150f6b1
Merge pull request #134917 from NixOS/binutils-2.37
binutils: 2.35.1 -> 2.38
2022-03-29 10:16:36 -07:00
Bernardo Meurer
8e5d2ead61
binutils: rename gold to enableGold 2022-03-28 18:50:11 -07:00
Bernardo Meurer
7bac80fef8
stdenv: fix binutils' bootstrap
Co-authored-by: TredwellGit <tredwell@tutanota.com>
2022-03-28 18:50:04 -07:00
github-actions[bot]
b2fc63a844
Merge staging-next into staging 2022-03-26 12:02:50 +00:00
github-actions[bot]
811b90f085
Merge master into staging-next 2022-03-26 12:01:14 +00:00
Adam Joseph
13e63f67dd make-bootstrap-tools-cross: fix typo which prevents hydra eval
When implementing this reviewer request:

  https://github.com/NixOS/nixpkgs/pull/161158#discussion_r822256070

to move mips64el-unknown-linux-* from platforms.nix to examples.nix, I
neglected to update the reference in make-bootstrap-tools-cross.nix.
As a result, the hydra jobs to generate a bootstrap tarball for
mips64el are not running:

  https://hydra.nixos.org/eval/1750828?filter=mips&compare=1750620&full=#tabs-errors

This commit fixes the problem, so the hydra job can run.  Once it
does, I will submit a PR adding the trusted bootstrap tarball hash to
pkgs/stdenv/linux/bootstrap-files/.

Co-authored-by: sterni <sternenseemann@systemli.org>
2022-03-26 12:03:00 +01:00
github-actions[bot]
8357b101a5
Merge staging-next into staging 2022-03-25 12:02:05 +00:00
Vladimír Čunát
0a8b4eddd2
Merge branch 'master' into staging-next 2022-03-25 10:16:56 +01:00
Artturin
33cce15e42 treewide: remove meta.repositories
there's no documentation for meta.repositories and its not widely used
2022-03-24 23:56:14 +02:00
sternenseemann
b5cad4d4a4 stdenv/setup.sh: make sure $sourceRoot has +x before cd-ing
This change is prompted by the following, admittedly cursed tarball:

```
> curl https://registry.npmjs.org/char-regex/-/char-regex-1.0.2.tgz 2>/dev/null \
  | tar -ztv
drw-rw-rw- 0/0               0 2020-02-18 10:50 package
-rw-rw-rw- 0/0             297 2020-02-18 10:50 package/index.d.ts
-rw-rw-rw- 0/0            1920 2020-02-18 10:50 package/index.js
-rw-rw-rw- 0/0            1092 2020-01-31 11:31 package/LICENSE
-rw-rw-rw- 0/0             937 2020-02-18 10:51 package/package.json
-rw-rw-rw- 0/0             713 2020-02-18 10:50 package/README.md
```

The minimal reproducer for the issue is the following derivation trying
to work around the uid 0 issue with `dontMakeSourcesWritable = true`:

```nix
{ stdenv, fetchurl }:

stdenv.mkDerivation {
  name = "test";

  src = fetchurl {
    sha1 = "d744358226217f981ed58f479b1d6bcc29545dcf";
    url = "https://registry.npmjs.org/char-regex/-/char-regex-1.0.2.tgz";
  };

  dontMakeSourcesWritable = true;

  installPhase = ''
    cp -R . $out
  '';
}
```

This currently fails in the following way:

```
these derivations will be built:
  /nix/store/pc3jbydl0xcc8nrndf5xkf7hdhpgpb41-test.drv
building '/nix/store/pc3jbydl0xcc8nrndf5xkf7hdhpgpb41-test.drv'...
unpacking sources
unpacking source archive /nix/store/v9p98kqplf4kflmy91p0687xlvr6klb1-char-regex-1.0.2.tgz
source root is package
find: 'package/index.d.ts': Permission denied
find: 'package/index.js': Permission denied
find: 'package/LICENSE': Permission denied
find: 'package/package.json': Permission denied
find: 'package/README.md': Permission denied
/nix/store/6c47azxacncswc1pllzj28zfzqw40d7c-stdenv-linux/setup: line 1311: cd: package: Permission denied
builder for '/nix/store/pc3jbydl0xcc8nrndf5xkf7hdhpgpb41-test.drv' failed with exit code 1
error: build of '/nix/store/pc3jbydl0xcc8nrndf5xkf7hdhpgpb41-test.drv' failed
```

As you can see, the issue is that `$sourceRoot` isn't executable,
prohibiting the call to `cd`. This can be fixed by running
`chmod +x "${sourceRoot}"` before `cd` regardless of
`dontMakeSourcesWritable` in `unpackPhase` since if `chmod` fails, `cd`
would fail as well and we are out of options.

Verified that the workaround works locally.

Another thing to investigate is investigating if we should use
`--no-same-owner` for `tar` and if it helps in this case as well.
See also <https://github.com/Profpatsch/yarn2nix/issues/56>.
2022-03-24 11:13:38 +01:00
github-actions[bot]
691557fc47
Merge staging-next into staging 2022-03-18 00:03:09 +00:00
Adam Joseph
12371a51e6 lib/systems: add mips64el definitions
MIPS has a large space of {architecture,abi,endianness}; this commit
adds all of them to lib/systems/platforms.nix so we can be done with
it.

Currently lib/systems/inspect.nix has a single "isMips" predicate,
which is a bit ambiguous now that we will have both mips32 and mips64
support, with the latter having two ABIs.  Let's add four new
predicates (isMips32, isMips64, isMips64n32, and isMips64n64) and
treat the now-ambiguous isMips as deprecated in favor of the
more-specific predicates.  These predicates are used mainly for
enabling/disabling target-specific workarounds, and it is extremely
rare that a platform-specific workaround is needed, and both mips32
and mips64 need exactly the same workaround.

The separate predicates (isMips64n32 and isMips64n64) for ABI
distinctions are, unfortunately, useful.  Boost's user-scheduled
threading (used by nix) does does not currently supports mips64n32,
which is a very desirable ABI on routers since they rarely have
more than 2**32 bytes of DRAM.
2022-03-10 20:30:16 -08:00
github-actions[bot]
c15f51235d
Merge staging-next into staging 2022-03-07 12:02:15 +00:00
Maximilian Bosch
856376bb20
Merge pull request #162197 from trofi/fix-gcc-wrapper-in-stdenv
pkgs/stdenv/linux/default.nix: restore dropped gcc-wrapper
2022-03-01 18:35:49 +01:00
Sandro
800e9e9994
Merge pull request #161079 from SuperSandro2000/exit-handler 2022-03-01 14:41:56 +01:00
Felix Buehler
63a37b844c darwin: deprecate phases 2022-02-28 10:58:08 +01:00
Sergei Trofimovich
32c30ae43c pkgs/stdenv/linux/default.nix: restore dropped gcc-wrapper
7459a40 `stdenv-bootstrap: force using new libc from stage2` overrode
`gcc-wrapper`. 91fa7657 `stdenv: revert gcc hack` dod not restore the initial
value.

As a result attempt to build glibc-2.35 fails early on perl as:

   ld: cannot find -lnsl: No such file or directory
   collect2: error: ld returned 1 exit status
   I can't compile the test program.
   (The supplied flags or libraries might be incorrect.)

The change restores `gcc-wrapper` as it was before 7459a40.
2022-02-28 09:42:20 +00:00
Maximilian Bosch
91fa7657d1
stdenv: revert gcc hack
See https://github.com/NixOS/nixpkgs/pull/133431#discussion_r805339569
2022-02-27 15:10:58 +01:00
Maximilian Bosch
7459a4021c
stdenv-bootstrap: force using new libc from stage2
This fix is needed to work around linker-errors such as

    undefined reference to `__libc_csu_fini'

which I got in almost every derivation which is part of stage2. The
reasoning behind this is that the startup-code was simplified[1] and
thus `__libc_csu_fini` doesn't exist anymore.

A workable solution is to use a newer libc which properly links in
stage3. And actually this seems expected given the rationale for stage3:

    # Construct a third stdenv identical to the 2nd, except that this
    # one uses the rebuilt Glibc from stage2.  It still uses the recent
    # binutils and rest of the bootstrap tools, including GCC.

So this patch basically overrides the libraries inside `gcc-unwrapped` -
which is basically the bootstrap tools and thus also contains the libc
used in stage3 - with the shared objects from the freshly built libc
from stage2.

[1] https://sourceware.org/pipermail/libc-alpha/2021-March/123079.html
2022-02-27 10:25:33 +01:00
Sandro Jäckel
b7da6c7da7
stdenv, dep-licenses.sh: do not skip handling of other exit traps
See https://github.com/akinomyoga/ble.sh/issues/179
2022-02-20 16:27:24 +01:00
Naïm Favier
1ffdf02435
stdenv/check-meta: remove onlyLicenses check 2022-02-17 13:09:24 +01:00
github-actions[bot]
335510eb84
Merge master into staging-next 2022-02-08 00:01:49 +00:00
zowoq
f5227f0643 bootstrap-tools: disable iconv for the hello test 2022-02-08 08:57:11 +10:00
Dmitry Kalinkin
3087088c41
Merge branch 'staging-next' into staging
Conflicts:
	pkgs/top-level/aliases.nix
	pkgs/top-level/python-aliases.nix
2022-02-01 21:37:39 -05:00
matthewcroughan
7bea56b425 stdenv/check-meta: add note for Flake usage
Flake users that use a command like `nix build nixpkgs#hello` on a
broken/insecure package will not be able to use an environment variable
to override that behavior, unless they pass `--impure` to the command.

Co-authored-by: pkharvey <kayharvey@protonmail.com>
2022-02-01 13:23:46 -05:00
github-actions[bot]
ce60c22080
Merge staging-next into staging 2022-01-30 06:01:52 +00:00
matthewcroughan
b0c0e0d7eb stdenv: introduce withCFlags
Adds an easy method of appending compiler flags to your stdenv via a
list.

Co-authored-by: tomberek <tomberek@users.noreply.github.com>
Co-authored-by: Gytis Ivaskevicius <gytis02.21@gmail.com>
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
2022-01-29 21:23:12 -05:00
Jan Tojnar
2da5ce4ab3 Merge branch 'staging-next' into staging
; Conflicts:
;	pkgs/development/python-modules/fakeredis/default.nix
2022-01-28 14:05:11 +01:00
Sandro
27cccd4e49
Merge pull request #151363 from Stunkymonkey/doc-updateWalker 2022-01-27 14:06:36 +01:00
Felix Buehler
59c55f4558 update-walker: remove because unused 2022-01-26 21:46:59 +01:00
Sandro Jäckel
24880b690b stdenv: fix shellcheck complaining about things not being posix compliant 2022-01-23 03:50:23 +01:00
Sandro Jäckel
37fdba0b4f stdenv: restore bash options -e/-u to the values they where before
Source https://github.com/akinomyoga/ble.sh/issues/169#issuecomment-1019049032

Author: akinomyoga
2022-01-23 03:50:09 +01:00
Dmitry Kalinkin
7673650020
stdenv/darwin: fix for curl with zstd and idn2
The `curlMinimal` is to be used throughout the early bootstrap
stages. The final stage will allow the new references of the `curl`.

Fixes: 29526bc2 ('curl: IDN support requires libidn2 package')
2022-01-13 19:45:30 -05:00
John Ericson
baf290023c
Merge pull request #153790 from sternenseemann/cross-bootstrap-logic
Cross bootstrapping logic fixes benefitting llvmPackages
2022-01-08 12:39:30 -08:00
sternenseemann
766f5ffb76 llvmPackages_*: respect cc for target when choosing C++ flavour
llvmPackages_*.clang should check the default compiler for the package
set it is targeting (targetPackages.stdenv.cc) instead of the compiler
that has been used to build it (stdenv.cc) in order to get some sense of
whether to use libc++ or libstdc++.

Since we are now inspecting targetPackages in the llvmPackages.clang
attribute, we need to avoid using it in the cross stdenv — which just
forces us to explicitly request libcxxClang for darwin instead of
relying on the clang attribute to pick it for us.

We also need to do something similar for targetPackages.stdenv.cc: Here
the llvmPackages.clang logic would work as we want (inspect
targetPackages.stdenv.cc and if it doesn't exist, make the choice based
on stdenv.cc), but it gets locked in a cycle with the previous package.
We can easily break this, however: We know that the previous set had
clang and the next one doesn't exist, so we'd choose libcxxClang any day
of the week.
2022-01-07 14:42:41 +01:00
Vladimír Čunát
0b7de2ab41
Merge #151399: stdenv: update aarch64 bootstrap-files
...into staging
2022-01-04 20:08:39 +01:00
Naïm Favier
f2065d81ad stdenv/generic: introduce shellDryRun
Add `shellDryRun` to the generic stdenv and substitute it for uses of
`${stdenv.shell} -n`. The point of this layer of abstraction is to add
the flag `-O extglob`, which resolves #126344 in a more direct way.
2021-12-27 20:30:01 -05:00
github-actions[bot]
e38de71da8
Merge master into staging-next 2021-12-25 00:01:28 +00:00
rembo10
eefebccc12 cmake make-derivation: fix hostPlatform uname release when cross compiling 2021-12-24 13:18:42 +05:30
Dmitry Kalinkin
1a9297102c
Merge branch 'master' into staging-next 2021-12-23 16:58:33 -05:00
Bernardo Meurer
ba3cc463ca
Merge pull request #147942 from oxalica/test/riscv-bootstrap-tools
stdenv: bootstrap riscv64 (Step 2)
2021-12-23 21:40:31 +00:00
David Guibert
8e7acfc587 aarch64: update bootstrap-files
fixes #112086

Fetch bootstrap files built by Hydra and re-add them as FODs to skip
currently dangling URLs. They are already available on
https://cache.nixos.org

https://hydra.nixos.org/job/nixpkgs/trunk/stdenvBootstrapTools.aarch64-linux.dist ->
https://hydra.nixos.org/build/160330465#tabs-buildinputs

These commands have been run to get the output hashes as FODs:

nix store add-file --name bootstrap-tools.tar.xz  $(nix-store -r /nix/store/9y4qnwflnxkjrqnfbzhsjv6zq2r0m48z-stdenv-bootstrap-tools --option binary-caches https://cache.nixos.org)/on-server/bootstrap-tools.tar.xz
nix store add-path --name busybox                 $(nix-store -r /nix/store/9y4qnwflnxkjrqnfbzhsjv6zq2r0m48z-stdenv-bootstrap-tools --option binary-caches https://cache.nixos.org)/on-server/busybox
2021-12-20 10:53:19 +01:00
github-actions[bot]
209e8d0932
Merge master into staging-next 2021-12-11 00:01:48 +00:00
Guillaume Girol
4a8f9970e2
Merge pull request #147544 from cab404/fix-stdenv-override
stdenv: move overriden stdenv in closure
2021-12-10 20:29:30 +00:00
github-actions[bot]
f11307e4b2
Merge staging-next into staging 2021-11-30 00:02:29 +00:00
oxalica
a5851f18c5
stdenv: add bootstrap files for riscv64
Bootstrap files are from
https://hydra.nixos.org/build/159891432
and
https://hydra.nixos.org/build/159891436
2021-11-30 05:03:30 +08:00
Vladimir Serov
523c701c0b
stdenv: move overriden stdenv in closure
Before that, base stdenv passed non-makeOverridable version of itself
inside. This cause it to be lost on package-name.stdenv.
2021-11-27 01:23:32 +03:00
oxalica
8cf96b9426
stdenv: patch autoconf script for riscv in stage 2 2021-11-26 00:32:33 +08:00
oxalica
fc3ea54fb3
make-bootstrap-tools: produce libatomic on riscv platform 2021-11-26 00:32:33 +08:00
toonn
5d23e617ec stdenv: Pass standalone argument for libcxxabi 2021-11-24 15:17:13 -08:00
toonn
63a3ead084 stdenv: Darwin no longer needs 10.11 workaround
Co-authored-by: happysalada <raphael@megzari.com>
2021-11-24 15:17:13 -08:00
toonn
77fc5f8c14 bootstrap-tools: Introduce getLib for lib outputs
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
Co-authored-by: Jörg Thalheim <joerg@thalheim.io>
2021-11-24 15:17:13 -08:00
toonn
cc767e1d05 darwin.stdenv: Update to LLVM 11 and clang 11.1.0 2021-11-24 15:17:13 -08:00
toonn
84454c3233 darwin.Libsystem: Patch TargetConditionals.h
`TargetConditionals.h` was missing several definitions, like
`TARGET_OS_TV` that are part of SDK 10.12 at least. And one that doesn't
seem to occur in any SDK afaict, `TARGET_OS_EMBEDDED_OTHER`.

I added the definitions from SDK 10.12 verbatim and defined
`TARGET_OS_EMBEDDED_OTHER` to be equal to `0`.

I think none of this works if `darwin.Libsystem` is used to build for
linux or iOS though so maybe this needs a more thorough fix?

This reverts 336d82617f because it's no
longer necessary.
2021-11-24 15:17:13 -08:00
toonn
e07eef85e8 xnu: Fix python3 patch
This reverts 7da313e10a because it's no
longer necessary.
2021-11-24 15:17:13 -08:00
Daiderd Jordan
7e4880c21b stdenv: update darwin bootstrap tools for llvm11
Build from bdffd0b276.

https://hydra.nixos.org/build/124753463
(cherry picked from commit f4a08349ed43a36ad5489a4ab548fcf0c129215d)
2021-11-24 15:17:13 -08:00
Alyssa Ross
2ebeb02a99 stdenv/setup: tell libtool about library paths
Packages that use libtool run it as a wrapper around the linker.
Before calling the linker, libtool will determine what libraries would
be linked, and check if there's a corresponding libtool
archive (libfoo.la) file in the same directory .  This file
contains extra information about the library.  This is especially
important for static linking, because static archives don't contain
dependency information, so we need libtool to use the .la files to
figure out which libraries actually need to be linked against.

But in Nixpkgs, this has never worked.  libtool isn't able to find any
libraries, because only the compiler wrapper knows how to find them,
and the compiler wrapper is opaque to libtool.  This is why
pkgsStatic.util-linuxMinimal doesn't build prior to this patch — it
depends on libpam, which depends on libaudit, and if libtool can't
find the .la file, nothing will tell the linker to also link against
libaudit when linking libpam.  (It was previously possible to build a
static util-linux, because linux-pam only recently had the audit
dependency added.)

There are a couple of ways we could fix this, so that libtool knows
where to look for .la files.

 * Set LD_LIBRARY_PATH/DYLD_LIBRARY_PATH/whatever, which libtool will
   examine.  This would have major side effects though, because the
   dynamic linker looks at it too.

 * Inject libtool scripts with the appropriate information.  That's
   what I've done here.  It was the obvious choice because we're
   already finding and modifying the libtool scripts, to remove paths
   outside the Nix store that libtool might check in unsandboxed
   builds.  Instead of emptying out the system paths, we can
   repopulate it with our own library paths.

(We can't use a wrapper like we do for other tools in Nixpkgs, because
libtool scripts are often distributed in source tarballs, so we can't
just add a wrapped version of libtool as a dependency.  That's why
there's already the fixLibtool function in stdenv.)

With this change, libtool is able to discover .la files, and
pkgsStatic.util-linuxMinimal can build again, linking correctly
against libpam and libaudit.
2021-11-23 21:33:16 +00:00
Artturi
0809a3a44b
Merge pull request #137209 from milahu/patch-10 2021-10-25 20:32:06 +03:00
Rick van Schijndel
6a848de95d stdenv: add llvm-config to meson cross file
This is e.g. required to get mesa cross-compiling, but is useful for everyone depending on libllvm
2021-10-13 20:44:10 +02:00
sterni
5d0972c63b
Merge pull request #138289 from sternenseemann/fix-non-gnu-strip
stdenv: move --enable-deterministic-archives flag into GNU wrapper
2021-10-07 12:29:59 +02:00
happysalada
a634fbe065 stdenv: add shopt inherit_errexit 2021-10-06 00:42:58 +09:00
happysalada
84e4715a14 stdenv: use named ref to clarify intent 2021-10-06 00:42:29 +09:00
Emily
20a6d8a241 libxml2: use libiconv on Darwin
This fixes e.g. python3Packages.beautifulsoup4, which has tests relying
on the wider encoding support enabled by libiconv.

Fixes #137678.
2021-10-02 10:05:17 +09:00
Ryan Burns
40299257cc Merge branch 'master' into staging-next 2021-09-23 18:43:00 -07:00
Michael Stone
470466fdbd
sigtool: 4a3719b4 -> 2a13539d (#138453)
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2021-09-23 17:14:07 +02:00
sternenseemann
11fe2fc3cf stdenv: move --enable-deterministic-archives flag into GNU wrapper
`--enable-deterministic-archives` is a GNU specific strip flag and
causes other strip implementations (for example LLVM's, see #138013)
to fail. Since strip failures are ignored, this means that stripping
doesn't work at all in certain situation (causing unnecessary
dependencies etc.).

To fix this, no longer pass `--enable-deterministic-archives`
unconditionally, but instead add it in a GNU binutils specific strip
wrapper only.

`commonStripFlags` was only used for this flag, so we can remove
it altogether.

Future work could be to make a generic strip wrapper, with support for
nix-support/strip-flags-{before,after} and NIX_STRIP_FLAGS_{BEFORE,AFTER}.
This possibly overkill and unnecessary though -- also with the
additional challenge of incorporating the darwin strip wrapper somehow.
2021-09-18 15:11:17 +02:00
Winter
8cff7796d7
stdenv: re-add isMachO helper function (#138334) 2021-09-17 23:09:06 -04:00
Vladimír Čunát
183cd6b09f
Merge #138186: stdenv: remove isMachO helper function
...into staging-next
2021-09-17 18:31:26 +02:00
Winter
f8edf7720d stdenv: remove isMachO helper function
This reverts commit 488395c0f8.

Currently, `nix print-dev-env` fails to execute if this function is present, because of its use of hex literals.
Until this issue (https://github.com/NixOS/nix/issues/5262) is solved, we should revert this to prevent breakage.
2021-09-16 16:56:12 -04:00
happysalada
33518fcb45 stdenv/setup.sh: fix read -N 0 for bash 5
somehow `read -N 0` behavior changed in bash 5. `read -d ''` has identical behavior
the purpose of the function is to read stdin and exit 1 on a null byte (i.e. if stdin is the content of a binary)

(cherry picked from commit 5d0acf20f8)
2021-09-12 09:48:54 +09:00
milahu
5e2f703e83
unpackFile: ignore timestamp warnings 2021-09-09 20:13:21 +02:00
happysalada
4c92bb8bdf stdenv: fix nix_build_cores guess
- use builtin arithmetic instead of external expr
- simplify logic with bash builtins
2021-09-07 00:36:55 +09:00
happysalada
02c142a2dd stdenv: fix showBuildStats
- remove going through another file
- use builtin instead of external cat
- improve echo formatting
2021-09-06 22:51:33 +09:00
Martin Weinelt
7f732aca66 Revert "Merge remote-tracking branch 'origin/python-unstable' into staging-next"
This reverts commit b041b2e1b2, reversing
changes made to 5b6c2380ad.
2021-09-05 15:02:25 +02:00
happysalada
e32bf6f4f2 stdenv setup.sh: remove combined [ in favor of [[
[ ... ] && [ ...] -> [[ ... && ... ]]
2021-08-30 10:27:45 +09:00
happysalada
fd89fb6248 stdenv: remove bash version compatibility hack 2021-08-30 10:26:56 +09:00
happysalada
13049cd33e stdenv: remove combined command conditional 2021-08-30 10:26:14 +09:00
happysalada
2fa9facc49 stdenv: arithmetic fixes 2021-08-30 10:25:40 +09:00
happysalada
1200c8175c stdenv: reduce io 2021-08-30 10:24:30 +09:00
happysalada
54475daa27 stdenv: declare missing variables 2021-08-30 10:24:00 +09:00
Sebastián Mancilla
488395c0f8
stdenv: add isMachO helper function (#133808)
Detect if a binary is a Mach-O file.
2021-08-21 15:33:03 -04:00
John Ericson
3edba5edfc pkgsStatic: Finally obviate overlay! 2021-08-20 06:09:48 +00:00
John Ericson
904625852d pkgsStatic: Inline more of static overlay 2021-08-19 21:55:06 +00:00
John Ericson
221ca6f2ff
Merge pull request #134463 from Ericson2314/stdenv-adapter-latebind
stdenv: Fix overriding + `overrideAttrs`
2021-08-18 19:01:54 -04:00
John Ericson
b4cc2a2479 pkgs/stdenv/make-derivation: Reindent
We previously make it just be the function, not a single-item attrset,
without deindenting to make a readable diff. No we deindent.
2021-08-18 17:22:52 +00:00
John Ericson
f110a182a6 stdenv: Fix overriding + overrideAttrs
The old stdenv adapters were subtly wrong in two ways:

 - `overrideAttrs` leaked the original, unoverridden `mkDerivation`.

 - `stdenv.override` would throw away any manually-set `mkDerivation`
   from a stdenv reverting to the original.

Now, `mkDerivation` is controlled (nearly directly) via an argument, and
always correctly closes over the final ("self") stdenv. This means the
adapters can work entirely via `.override` without any manual `stdenv //
...`, and both those issues are fixed.

Note hashes are changed, because stdenvs no previously overridden like
`stdenvNoCC` and `crossLibcStdenv` now are. I had to add some
`dontDisableStatic = true` accordingly. The flip side however is that
since the overrides compose, we no longer need to override anything but
the default `stdenv` from which all the others are created.
2021-08-18 17:22:50 +00:00
github-actions[bot]
1b4a28fed4
Merge master into staging-next 2021-08-07 12:01:25 +00:00
Artturin
40944bbab7 stdenv/check-meta: add maxSilent
Hydra supports it
https://github.com/NixOS/hydra/blob/master/src/hydra-eval-jobs/hydra-eval-jobs.cc#L172
2021-08-07 02:18:59 +03:00
Alyssa Ross
0901dfb214 stdenv/setup: force libtool to skip dep checks
When we "fix" libtool, we empty out its system library path to avoid
it discovering libraries in e.g. /usr when the sandbox is disabled.
But this also means that the checks libtool does to make sure it can
find the libraries its supposed to be linking to won't work.  On Linux
and Darwin, this isn't a problem, because libtool doesn't actually
perform any checks, but it is on at least NetBSD and Cygwin[1].

So, we force libtool not to do these checks on any platform, bringing
the more exotic platforms into line with the existing behaviour on
Linux and Darwin.

Without this change, lots of library packages produce warnings like
this in their build output on the platforms with checks by default:

    *** Warning: linker path does not have real file for library -lz.
    *** I have the capability to make that library automatically link in when
    *** you link to this library.  But I can only do this if you have a
    *** shared version of the library, which you do not appear to have
    *** because I did check the linker path looking for a file starting
    *** with libz but no candidates were found. (...for regex pattern test)
    *** The inter-library dependencies that have been dropped here will be
    *** automatically added whenever a program is linked with this library
    *** or is declared to -dlopen it.

And dependent packages break because libtool doesn't link their
transitive dependencies.  So making this change fixes _lots_ of
packages on those platforms.

[1]: https://git.savannah.gnu.org/cgit/libtool.git/tree/m4/libtool.m4?id=544fc0e2c2a03129a540aebef41ad32bfb5c06b8#n3445
2021-07-27 20:42:31 +00:00
Jörg Thalheim
7bf8b97d68
Merge pull request #130601 from happysalada/stdenv_cosmetic
Stdenv cosmetic
2021-07-23 09:45:43 +01:00
happysalada
4fc50527f8 stdenv: typo 2021-07-19 14:49:47 +09:00
Vladimír Čunát
4bd38c330f
Revert #127736: stdenv changes towards an alternative shell
At least for now.  Such changes are risky (we have very many packages),
and apparently it needs more testing/review without blocking other
changes.

This reverts the whole range 4d0e3984918^..8752c327377,
except for one commit that got reverted in 6f239d7309 already.
(that MR didn't even get its merge commit)
2021-07-17 20:39:47 +02:00
Vladimír Čunát
88a6ee04bf
Merge #129854: stdenv/darwin: finalize URLs (into staging-next) 2021-07-16 17:41:02 +02:00
Vladimír Čunát
2a44031d56
stdenv/make-bootstrap-tools: nuke yet another header
This is needed after glibc bump from PR #111616.
2021-07-16 16:15:32 +02:00
Vladimír Čunát
f47ee76a6a
stdenv/darwin: switch to the official bootstrap URLs 2021-07-16 10:44:04 +02:00
Vladimír Čunát
3b437b95df
Merge #129854: stdenv/darwin: update bootstrap tools
... for x86_64-darwin (into staging-next)
It wouldn't bootstrap otherwise.

Unfortunately we still haven't managed to get the tarballs
on the proper URLs, but GitHub should be reliable enough
and surely almost noone will bootstrap themselves anyway.
2021-07-16 10:26:10 +02:00
Vladimír Čunát
6f239d7309
Revert "setup.sh: fatal: This word should yield a string, but it contains an array"
This reverts commit bf99a819a1.
It caused regressions in some packages; see:
https://github.com/NixOS/nixpkgs/commit/bf99a819a160
2021-07-16 10:00:29 +02:00
Vladimír Čunát
d6a51653d7
stdenv/darwin: update bootstrap tools for x86_64-darwin
Updated python3 will block bootstrapping otherwise.

Verification instructions:
$ xdg-open https://hydra.nixos.org/build/144118577#tabs-details
$ nix build /nix/store/sh9cd0rarr8a2db1zdr4fzl4qsmvz5jp-stdenv-bootstrap-tools
$ cat /nix/store/sh9cd0rarr8a2db1zdr4fzl4qsmvz5jp-stdenv-bootstrap-tools/nix-support/hydra-build-products
$ nix hash-path /nix/store/9h4d7s313wv3gkfwi493yr1wvdsz9lf2-stdenv-bootstrap-tools/on-server/{sh,bzip2,mkdir,cpio}
sha256-igMAVEfumFv/LUNTGfNi2nSehgTNIP4Sg+f3L7u6SMA=
sha256-K3rhkJZipudT1Jgh+l41Y/fNsMkrPtiAsNRDha/lpZI=
sha256-VddFELwLDJGNADKB1fWwWPBtIAlEUgJv2hXRmC4NEeM=
sha256-SWkwvLaFyV44kLKL2nx720SvcL4ej/p2V/bX3uqAGO0=
$ nix hash-file /nix/store/9h4d7s313wv3gkfwi493yr1wvdsz9lf2-stdenv-bootstrap-tools/on-server/bootstrap-tools.cpio.bz2
sha256-b65dXbIm6o6s6U8tAiGpR6SMfvfn/VFcZgTHBetJZis=
2021-07-10 17:03:40 +02:00
Vladimír Čunát
7c25c1fd88
Merge branch 'glibc-2.33' into staging 2021-07-06 16:17:13 +02:00
Vladimír Čunát
5a71fa8447
stdenv bootstrap: hack around glibc version mismatch
With this we shouldn't need other workarounds for the LTO problems.
2021-07-06 16:02:11 +02:00
Raphael Megzari
cf5882f27c Update pkgs/stdenv/generic/setup.sh:use [[ instead of [
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2021-07-06 09:27:18 +09:00
Raphael Megzari
2c3b847d29 Update pkgs/stdenv/generic/setup.sh: use [[ instead of [
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2021-07-06 09:27:18 +09:00
Raphael Megzari
80c9096d9c Update pkgs/stdenv/generic/setup.sh: group var declaration
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2021-07-06 09:27:18 +09:00
Raphael Megzari
bcdfbfa19f Update pkgs/stdenv/generic/setup.sh: group var declaration
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2021-07-06 09:27:18 +09:00
happysalada
06688b73ca setup.sh: separate command from combined conditional 2021-07-06 09:27:18 +09:00
Raphael Megzari
caeb3c915f Update pkgs/stdenv/generic/setup.sh
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
2021-07-06 09:27:18 +09:00
happysalada
5f8b5465e1 setup.sh: remove bash empty array old hack 2021-07-06 09:27:18 +09:00
happysalada
fc5438bfcf setup.sh: use [[ for combined conditionals 2021-07-06 09:27:18 +09:00
happysalada
5617feb330 setup.sh: remove extraneous cat cmd 2021-07-06 09:27:18 +09:00
happysalada
1f255cbe5c setup.sh: arithmetic conditional style
used "recommended" arithmetic conditional style
2021-07-06 09:27:18 +09:00
happysalada
9f156d0adc setup.sh: fatal: Undefined variable
define hook arrays before assigning to them
2021-07-06 09:27:18 +09:00
happysalada
bf99a819a1 setup.sh: fatal: This word should yield a string, but it contains an array
remove implicit array comparison in case
2021-07-06 09:27:18 +09:00
happysalada
1a71a5fb08 setup.sh: shellcheck disable=SC1091
https://github.com/koalaman/shellcheck/wiki/SC1091
2021-07-06 09:27:18 +09:00
happysalada
b1c5a9f008 setup.sh: shellcheck disable=SC2034
https://github.com/koalaman/shellcheck/wiki/SC2034
2021-07-06 09:27:18 +09:00
happysalada
740582739c setup.sh: missing white space 2021-07-06 09:27:18 +09:00
happysalada
faca9440fb setup.sh: add quotes where semantically valid 2021-07-06 09:27:18 +09:00
happysalada
5f01543070 setup.sh: fatal: Invalid integer constant 'relHostOffset'
dereference variables passed to mapOffset
2021-07-06 09:27:18 +09:00
happysalada
958c180d1f setup.sh: shellcheck disable=SC2123
https://github.com/koalaman/shellcheck/wiki/SC2123
2021-07-06 09:27:18 +09:00
happysalada
8dbbd8fbe3 setup.sh: shellcheck disable=SC1090
https://github.com/koalaman/shellcheck/wiki/SC1090
2021-07-06 09:27:18 +09:00
happysalada
796f1cdbea shellcheck disable=SC2148
https://github.com/koalaman/shellcheck/wiki/SC2148
2021-07-06 09:27:18 +09:00
happysalada
422579f718 shellcheck disable=SC2154
https://github.com/koalaman/shellcheck/wiki/SC2154
2021-07-06 09:27:18 +09:00
happysalada
b49f22fd3e setup.sh: replace type -p with test -e
type -p will exit 1 on failure.
Test makes the intent clearer here.
2021-07-06 09:27:18 +09:00
happysalada
ac27528177 setup.sh: arithmetic fixes
this one is a little more controversial
see https://github.com/oilshell/oil/issues/864
for more information
2021-07-06 09:27:18 +09:00
happysalada
c335a18ea5 setup.sh: shellcheck 2206 + 2207
https://github.com/koalaman/shellcheck/wiki/SC2206
https://github.com/koalaman/shellcheck/wiki/SC2207

admittedly this one is a lot less pretty
2021-07-06 09:27:18 +09:00
happysalada
430fdb7e0d setup.sh: shellcheck 2086
https://github.com/koalaman/shellcheck/wiki/SC2086
2021-07-06 09:27:18 +09:00
happysalada
40ad05b4cb builder.sh: check if variable is non null
This is not the "correct" way to check if a variable is non null in
bash. There is already an instance of the "right" way to do it in
setup.sh. Bash is "generous" enough to accept the original input though.
I couldn't find the relevant shellcheck.
2021-07-06 09:27:18 +09:00
happysalada
e39ee04d97 builder.sh: shellcheck 2129
https://github.com/koalaman/shellcheck/wiki/SC2129
2021-07-06 09:27:18 +09:00
happysalada
4d0e398491 builder.sh: shellcheck 2086
https://github.com/koalaman/shellcheck/wiki/SC2086
2021-07-06 09:27:18 +09:00
Niklas Hambüchen
3ac484ef83
Merge pull request #129328 from nh2/manual-fix-hardening-flags-sections
manual: hardening: Fix disabled flags prose being in previous section
2021-07-05 23:11:22 +02:00
Niklas Hambüchen
ac36a0f9eb manual: hardening: Fix disabled flags prose being in previous section
This confused the hell out of me, as I didn't spot the

> The following flags are disabled by default ...

when reading about `pie`, because that sentence was hidden in the
previous hardening flag's section.

Also explain that `pie` hardening is on by default on musl.
2021-07-05 16:32:55 +02:00
Sandro
d871186cdf
Merge pull request #108102 from matthewbauer/bash-version-check
stdenv/setup.sh: Add version check to setup script
2021-06-24 13:06:46 +02:00
Matthew Bauer
8fa084dc0f stdenv/setup.sh: Add version check to setup script
Only bash 4+ works in setup.sh. To make sure this is obvious, we can
check BASH_VERSINFO to get the major version number of Bash.

While Bash 3 is pretty rare, it still comes stock in macOS.

We *could* provide a warning here for non-Bash shells, but it’s not
always clear whether they will work or not. Zsh should have no trouble
while busybox sh, fish, or any others. There’s no great way to detect
what feature set the shell supports.

Fixes #71625
2021-06-23 01:32:58 -05:00
Jan Tojnar
e3dfa79441
Merge branch 'staging-next' into staging
Regenerated pkgs/servers/x11/xorg/default.nix to resolve the conflict.
2021-06-16 19:59:05 +02:00
Daiderd Jordan
40ffedfb38
darwin: expose bootstrapLlvmVersion as argument in the stdenv
This enables the bootstrap stdenv test to specify the actual llvm
of the newly generated build instread of assuming it's the same version
as the current stdenv.
2021-06-13 17:32:06 +02:00
Alyssa Ross
b0b5ef7286 stdenv: introduce dontAddStaticConfigureFlags
With removeUnknownConfigureFlags, it's impossible to express a package
that needs --enable-static, but will not accept --disable-shared,
without overriding the result of removeUnknownConfigureFlags _again_
in pkgs/top-level/static.nix.

It would be much better (and more in line with the rest of Nixpkgs) if
we encoded changes needed for static builds in package definitions
themselves, rather than in an ever-expanding list in static.nix.  This
is especially true when doing it in static.nix is going to require
multiple overrides to express what could be expressed with stdenv
options.

So as a step in that direction, and to fix the problem described
above, here I replace removeUnknownConfigureFlags with a new stdenv
option, dontAddStaticConfigureFlags.  With this mechanism, a package
that needs one but not both of the flags just needs to set
dontAddStaticConfigureFlags and then set up configureFlags manually
based on stdenv.hostPlatform.isStatic.
2021-06-11 14:16:05 -07:00
Daiderd Jordan
bdffd0b276
Merge pull request #121055 from toonn/bootstrap-tools-bump
darwin bootstrap-tools bump LLVM to 11
2021-06-09 08:23:20 +02:00
Jonathan Ringer
04bdefc4b8 stdenv/native: fix bintools import 2021-06-07 11:26:15 +00:00
toonn
c7b433acf9
bootstrap-tools: libclang-cpp already included
Changes to llvmPackages have caused the `libclang-cpp*.dylib` files to
be included in the `clang-unwrapped.lib` output. So we no longer need to
copy them from libclang.
2021-06-06 20:17:32 +02:00
toonn
336d82617f
bootstrap-tools: Patch TargetConditionals.h
`TargetConditionals.h` was missing several definitions, like
`TARGET_OS_TV` that are part of SDK 10.12 at least. And one that doesn't
seem to occur in any SDK afaict, `TARGET_OS_EMBEDDED_OTHER`.

I added the definitions from SDK 10.12 verbatim and defined
`TARGET_OS_EMBEDDED_OTHER` to be equal to `0`.

This is a modified version of a patch to avoid a stdenv rebuild.
2021-06-06 20:17:13 +02:00
toonn
fa49e70c1b
bootstrap-tools: Include libclang-cpp.11.1.dylib too
We need to include the library with the version suffix because
`libclang-cpp.dylib` is just a symlink.
2021-06-05 19:56:53 +02:00
toonn
d259fcc3e2
bootstrap-tools: Include libclang-cpp.dylib
Credits to @LnL7, this is based on commit
ae68f1c447 from PR #85151.
2021-06-05 19:56:53 +02:00
toonn
4a2698221b
bootstrap-tools: Update to LLVM 11 2021-06-05 19:56:53 +02:00
toonn
3b8782a806
bootstrap-tools: Add bootstrapFiles argument
I was having a hard time testing new bootstrapFiles because
`make-bootstrap-tools.nix` imports `pkgspath` but does not pass anything
but the current system.

This is merely for convenience and I'm not entirely certain it's a
sensible thing to do, maybe generating new bootstrapFiles while
overriding the current bootstrapFiles isn't something you're supposed to
do?
2021-06-05 19:56:53 +02:00
toonn
7da313e10a
bootstrap-tools: Fix xnu python3 patch
This patch only affects the xnu included in the bootstrap-tools to avoid
a stdenv rebuild.
2021-06-05 19:55:38 +02:00
Sandro
c3f5d24b8d
Merge pull request #125494 from siraben/remove-stdenv-lib 2021-06-05 16:16:28 +02:00
Ben Siraphob
d2c9f816e3 stdenv: remove lib 2021-06-03 19:50:15 +07:00
happysalada
1c1c7685d9 stdenv.darwin: nixpkgs-fmt 2021-06-02 19:03:48 +09:00
Andrew Childs
38207735f4 darwin/make-bootstrap-tools: move "lib" from install name to rpath
The rpath structure for the bootstrap tools was reworked to minimize
the amount of rewriting required on unpack, but the test was not
updated to match the different structure.

Additionally [1] builds that use the bootstrap version of libc++
cannot find libc++abi if the reference includes the "lib"
component (ie, libc++ refers to libc++abi with
@rpath/lib/libc++abi.dylib).

[1] https://logs.nix.samueldr.com/nix-darwin/2021-05-18#4993282

Test failure observed on Hydra: https://hydra.nixos.org/build/143130126
2021-05-24 15:04:39 +09:00
Andrew Childs
5d22db3c5e
stdenv/darwin: add bootstrap tools for aarch64-darwin
Taken from PR #12355; other parts are still being discussed.
2021-05-18 21:34:57 +02:00
Andrew Childs
768aae66ef stdenv/darwin: Apple Silicon support 2021-05-17 00:27:02 +09:00
Andrew Childs
3eacdfe24a stdenv/darwin: bootstrap tools for Apple Silicon 2021-05-17 00:27:02 +09:00
Andrew Childs
a7bcb6b936 darwin cross: include CoreFoundation in stdenv 2021-05-17 00:27:01 +09:00
John Ericson
18c38f8aee treewide: All the linker to be chosen independently
This will begin the process of breaking up the `useLLVM` monolith. That
is good in general, but I hope will be good for NetBSD and Darwin in
particular.

Co-authored-by: sterni <sternenseemann@systemli.org>
2021-05-14 21:29:51 +00:00
John Ericson
7bba32a069 darwin packages: Get ready for cross
If things build fine with `stdenvNoCC`, let them use that. If tools
might be prefixed, prepare for that, either by directly splicing or just
using the env vars provided by the wrapper setup-hooks.

Co-authored-by: Dmitry Kalinkin <dmitry.kalinkin@gmail.com>
2021-05-11 16:07:01 -04:00
John Ericson
77e97ac4ca glibc: Use pname and version 2021-05-10 20:56:33 +00:00
John Ericson
a3e54cb582 Merge remote-tracking branch 'upstream/staging-next' into staging 2021-05-06 15:48:25 -04:00
John Ericson
470640e7fe treewide: Do a number of no-op cleanups for cross and darwin
I am taking the non-invasive parts of #110914 to hopefully help out with #111988.

In particular:

 - Use `lib.makeScopeWithSplicing` to make the `darwin` package set have
   a proper `callPackage`.

 - Adjust Darwin `stdenv`'s overlays keeping things from the previous
   stage to not stick around too much.

 - Expose `binutilsNoLibc` / `darwin.binutilsNoLibc` to hopefully get us
   closer to a unified LLVM and GCC bootstrap.
2021-05-06 11:17:26 -04:00
Andrew Childs
7869d16545 llvmPackages: Multuple outputs for everythting
Also begin to start work on cross compilation, though that will have to
be finished later.

The patches are based on the first version of
https://reviews.llvm.org/D99484. It's very annoying to do the
back-porting but the review has uncovered nothing super major so I'm
fine sticking with what I've got.

Beyond making the outputs work, I also strove to re-sync the packages,
as they have been drifting pointlessly apart for some time.

----

Other misc notes, highly incomplete

- lvm-config-native and llvm-config are put in `dev` because they are
  tools just for build time.

- Clang no longer has an lld dep. That was introduced in
  db29857eb3, but if clang needs help
  finding lld when it is used we should just pass it flags / put in the
  resource dir. Providing it at build time increases critical path
  length for no good reason.

----

A note on `nativeCC`:

`stdenv` takes tools from the previous stage, so:

1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.stdenv.cc`: `(?0, ?1, x)`

while:

1. `pkgsBuildBuild`: `(?1, x, x)`
2. `pkgsBuildBuild.targetPackages`: `(x, x, ?2)`
3. `pkgsBuildBuild.targetPackages.stdenv.cc`: `(?1, x, x)`
2021-04-30 05:41:00 +00:00
github-actions[bot]
9cf0393c37
Merge staging-next into staging 2021-04-28 18:14:28 +00:00
John Ericson
50a11f4f43
Merge pull request #120993 from regnat/easy-ca
Make the bootsrap respect the contentAddressedByDefault setting
2021-04-28 11:27:59 -04:00
regnat
4105c06bf3 Also make the bootstrap tools generation CA
(And fix an ofborg eval error btw)
2021-04-28 12:46:43 +02:00