Commit Graph

6397 Commits

Author SHA1 Message Date
github-actions[bot]
a59e8336d7
Merge staging-next into staging 2023-04-03 18:01:56 +00:00
Bernardo Meurer
f1f6ca8bcd
Merge pull request #209870 from amjoseph-nixpkgs/pr/stdenv/external-gcc-bootstrap 2023-04-03 08:19:03 -07:00
Andrew Hamon
e929a99848 writeShellApplication: allow substitutions and remote building.
Unlike most other trivial builders, writeShellApplication has a
relatively heavy checkphase that invokes shellcheck. On my system,
a small writeShellApplication takes over 1 second to build.

This PR allows substitutions and stops preferring local builds for
writeShellApplication.
2023-04-02 14:36:32 -07:00
Adam Joseph
6c209e862e emacs: path fixes resulting from libgccjit changes
The Nix-driven bootstrap of gcc resulted in some changes to the
structure of the `libgccjit` outpaths, and also added an additional
output (`libgcc`) to `gcc`.

This commit makes the corresponding changes in the `emacs`
derivation in order to not break emacs.

Emacs is the only user of `libgccjit` in nixpkgs at the moment.
2023-04-02 13:49:54 -07:00
Adam Joseph
7553d0fe29 stdenv: Nix-driven bootstrap of gcc
#### Summary

By default, when you type `make`, GCC will compile itself three
times.  This PR inhibits that behavior by configuring GCC with
`--disable-bootstrap`, and reimplements the triple-rebuild using
Nix rather than `make`/`sh`.

 #### Immediate Benefits

- Allow `gcc11` and `gcc12` on `aarch64` (without needing new
  `bootstrapFiles`)
- Faster stdenv rebuilds: the third compilation of gcc
  (i.e. stageCompare) is no longer a `drvInput` of the final stdenv.
  This allows Nix to build stageCompare in parallel with the rest of
  nixpkgs instead of in series.
- No more copying `libgcc_s` out of the bootstrap-files or other
  derivations
- No more Frankenstein compiler: the final gcc and the libraries it
  links against (mpfr, mpc, isl, glibc) are all built by the same
  compiler (xgcc) instead of a mixture of the bootstrapFiles'
  compiler and xgcc.
- No more [static lib{mpfr,mpc,gmp,isl}.a hack]
- Many other small `stdenv` hacks eliminated
- `gcc` and `clang` share the same codepath for more of `cc-wrapper`.

 #### Future Benefits

- This should allow using a [foreign] `bootstrap-files` so long as
  `hostPlatform.canExecute bootstrapFiles`.
- This should allow each of the libraries that ship with `gcc`
  (lib{backtrace, atomic, cc1, decnumber, ffi, gomp, iberty,
  offloadatomic, quadmath, sanitizer, ssp, stdc++-v3, vtv}) to be
  built in separate (one-liner) derivations which `inherit src;`
  from `gcc`, much like https://github.com/NixOS/nixpkgs/pull/132343

 #### Incorporates

- https://github.com/NixOS/nixpkgs/pull/210004
- https://github.com/NixOS/nixpkgs/pull/36948 (unreverted)
- https://github.com/NixOS/nixpkgs/pull/210325
- https://github.com/NixOS/nixpkgs/pull/210118
- https://github.com/NixOS/nixpkgs/pull/210132
- https://github.com/NixOS/nixpkgs/pull/210109
- https://github.com/NixOS/nixpkgs/pull/213909
- https://github.com/NixOS/nixpkgs/pull/216136
- https://github.com/NixOS/nixpkgs/pull/216237
- https://github.com/NixOS/nixpkgs/pull/210019
- https://github.com/NixOS/nixpkgs/pull/216232
- https://github.com/NixOS/nixpkgs/pull/216016
- https://github.com/NixOS/nixpkgs/pull/217977
- https://github.com/NixOS/nixpkgs/pull/217995

 #### Closes

- Closes #108305
- Closes #108111
- Closes #201254
- Closes #208412

 #### Credits

This project was made possible by three important insights, none of
which were mine:

1. @ericson2314 was the first to advocate for this change, and
   probably the first to appreciate its advantages.  Nix-driven
   (external) bootstrap is "cross by default".

2. @trofi has figured out a lot about how to get gcc to not mix up
   the copy of `libstdc++` that it depends on with the copy that it
   builds, by moving the `bootstrapFiles`' `libstdc++` into a
   [versioned directory].  This allows a Nix-driven bootstrap of gcc
   without the final gcc would still having references to the
   `bootstrapFiles`.

3. Using the undocumented variable [`user-defined-trusted-dirs`]
   when building glibc.  When glibc `dlopen()`s `libgcc_s.so`, it
   uses a completely different and totally special set of rules for
   finding `libgcc_s.so`.  This trick is the only way we can put
   `libgcc_s.so` in its own separate outpath without creating
   circular dependencies or dependencies on the bootstrapFiles.  I
   would never have guessed to use this (or that it existed!) if it
   were not for a [comment in guix] which @Mic92 [mentioned].

My own role in this PR was basically: being available to go on a
coding binge at an opportune moment, so we wouldn't waste a
[crisis].

[aarch64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662822938
[amd64-compare-ofborg]: https://github.com/NixOS/nixpkgs/pull/209870/checks?check_run_id=10662825857
[nonexistent sysroot]: https://github.com/NixOS/nixpkgs/pull/210004
[versioned directory]: https://github.com/NixOS/nixpkgs/pull/209054
[`user-defined-trusted-dirs`]: https://sourceware.org/legacy-ml/libc-help/2013-11/msg00026.html
[comment in guix]: 5e4ec82181/gnu/packages/gcc.scm (L253)
[mentioned]: https://github.com/NixOS/nixpkgs/pull/210112#issuecomment-1379608483
[crisis]: https://github.com/NixOS/nixpkgs/issues/108305
[foreign]: https://github.com/NixOS/nixpkgs/pull/170857#issuecomment-1170558348
[static lib{mpfr,mpc,gmp,isl}.a hack]: 2f1948af9c/pkgs/stdenv/linux/default.nix (L380)
2023-04-02 13:49:41 -07:00
Adam Joseph
39c7885cd9 cc-wrapper: if isClang, add -L${gccForLibs.libgcc}/lib
When wrapping `clang` and using a `gccForLibs` whose `libgcc` is in
its own output (rather than the `lib` output), this commit will adds
`-L${gccForLibs.libgcc}/lib` to `cc-ldflags`.

If that flag is not added, `firefox` will fail to compile because it
invokes `clang-wrapper` with `-fuse-ld=lld` and passes `-lgcc_s` to
`lld`, but does not tell `lld` where to find `libgcc_s.so`.  In that
situation, firefox will fail to link.
2023-04-02 03:04:20 -07:00
Adam Joseph
3b49fb2ab6 trivial-builders/test/references.nix: fix eval
The command

```
nix-build -A tests.trivial-builders.references --show-trace
```

fails eval with

```
in job ‘nixpkgs.tests.trivial-builders.references’:
error: The option `meta.description' does not exist. Definition values:
       - In `makeTest parameters': "Run the Nixpkgs trivial builders tests"
```

because `meta.description` and `meta.license` are not valid for
`nixosTest`s (they are valid for `mkDerivation` of course).

This has been causing Hydra eval failures:

  https://hydra.nixos.org/jobset/nixos/pr-209870-gcc-external-bootstrap#tabs-errors

Let's fix eval by removing these attributes.
2023-04-02 03:04:20 -07:00
github-actions[bot]
47ddbcb1f0
Merge staging-next into staging 2023-03-31 00:03:05 +00:00
Sandro
06c40b7498
Merge pull request #218991 from Trundle/builddotnetmodule-space-wrapper-args 2023-03-31 01:19:55 +02:00
github-actions[bot]
04fb768a55
Merge staging-next into staging 2023-03-30 18:01:42 +00:00
Naïm Favier
62a3b6448f Revert "symlinkjoin: print warning when keeping existing file" 2023-03-30 13:21:42 -03:00
github-actions[bot]
0a1c5be4ef
Merge staging-next into staging 2023-03-30 06:01:44 +00:00
sternenseemann
192d03bee4 writers: fix aarch64-darwin workaround if content is a string
When authoring #220966, I missed that the `//` would take effect solely
in the else branch, fixing hledger-check-fancyassertions, but not
tests.writers.bin.haskell.
2023-03-29 03:29:13 +02:00
github-actions[bot]
496dc37e4b
Merge staging-next into staging 2023-03-29 00:03:13 +00:00
Antonio Nuno Monteiro
91137380e6
buildDunePackage: use --mandir instead of --man 2023-03-28 11:07:28 -07:00
github-actions[bot]
202cb8119f
Merge staging-next into staging 2023-03-28 18:01:48 +00:00
Atemu
91f5aa3446
Merge pull request #213378 from Atemu/steam-env-vars
steam: add extraEnv argument
2023-03-28 17:25:14 +02:00
figsoda
63b9aeed13
Merge pull request #223320 from figsoda/auditable 2023-03-27 19:32:02 -04:00
Vladimír Čunát
1acb9fd044
Merge branch 'master' into staging 2023-03-27 09:36:27 +02:00
Atemu
bdf0dd28b9
Merge pull request #219091 from cdepillabout/buildfhsuserenv-version
buildFHSUserEnv: add version arg
2023-03-27 08:46:24 +02:00
figsoda
201d4b7c5c rustPlatform.buildRustPackage: make auditable the default 2023-03-26 22:47:06 -04:00
zowoq
9861cf4a7f rustPlatform.fetchCargoTarball: use sparse protocol for crates.io 2023-03-27 08:08:40 +10:00
Atemu
192c3ecd4b buildFHSEnvBubblewrap: allow deeper introspection via passthru 2023-03-26 17:19:29 +02:00
github-actions[bot]
72fc224baa
Merge staging-next into staging 2023-03-26 06:01:39 +00:00
github-actions[bot]
ff3e86d2b4
Merge master into staging-next 2023-03-26 06:01:06 +00:00
Yureka
4329de1264 rustPlatform.fetchCargoTarball: fail on git dependencies
The reason is that we can not expect the extended logic run on git
dependencies starting from Cargo 1.68 to be reproducible in future
 versions, and thus the output hash would not be sufficiently stable.
https://github.com/rust-lang/cargo/pull/11414
2023-03-26 01:52:03 +01:00
github-actions[bot]
333618087c
Merge staging-next into staging 2023-03-26 00:03:19 +00:00
github-actions[bot]
07fb9cae4e
Merge master into staging-next 2023-03-26 00:02:45 +00:00
Yureka
5ce282e57e rustPlatform.importCargoLock: follow symlinks when copying tree
fixes build when git dependencies contain symlinks into parent directory

needed for libdeltachat
2023-03-26 00:02:16 +01:00
Yureka
37b31d4a2f rustPlatform.importCargoLock: always fetch submodules when builtins.fetchGit is used 2023-03-26 00:02:16 +01:00
Artturin
780669daf5 treewide: don't hardcode /nix/store (no rebuilds changes)
improve experience for other store locations
2023-03-24 20:11:33 +02:00
github-actions[bot]
9e6337ddc0
Merge staging-next into staging 2023-03-24 12:02:22 +00:00
Vladimír Čunát
12dd95fbb1
Merge branch 'master' into staging-next 2023-03-24 09:07:41 +01:00
Vladimír Čunát
cb10bd6cb3
lapack: force a rebuild on x86_64-darwin
I have no idea why, but Hydra cached a corrupted binary
/nix/store/h0412qmi29ivgrc42lfhi1x290d6l429-lapack-3/lib/liblapack.dylib
so rebuilding it fixes issues in various dependants, e.g.
https://hydra.nixos.org/log/nqfq62pqbbmsq08kb2pljdkz9ldyr57b-python3.10-numpy-1.24.2.drv
2023-03-24 08:57:05 +01:00
Ivan Trubach
dcc576176c buildGo{Module,Package}: fix precedence for or operator 2023-03-24 15:45:25 +10:00
D Anzorge
c86b1a0bca rustPlatform.importCargoLock: handle workspace Cargo.toml false positives
Since we grep for 'workspace', it's possible the script ends up running
on a Cargo.toml that has the word 'workspace' in a comment, but does not
actually use workspaces
2023-03-24 15:45:16 +10:00
zowoq
6085b3b7f7
Merge pull request #215408 from astro/rust-sysroot 2023-03-24 15:44:59 +10:00
Vladimír Čunát
0a88dec4f3
Merge branch 'staging-next' into staging 2023-03-21 12:57:16 +01:00
Louis Blin
e2825fb1b5 rustPlatform.importCargoLock: improved semantics for 'extraRegistries' option
Clearer API following the recommendation on the pull request that
introduced this option a few days ago:
https://github.com/NixOS/nixpkgs/pull/221381#discussion_r1141226293
2023-03-20 14:40:04 -04:00
Martin Weinelt
9959ec97f8
Merge remote-tracking branch 'origin/master' into staging-next
Conflicts:
- pkgs/development/python-modules/wxPython/4.0.nix
- pkgs/development/python-modules/wxPython/4.1.nix
2023-03-20 15:07:36 +01:00
mdarocha
d093086a2b buildDotnetModule: add support for using combinePackages as dotnet-sdk
This allows packages that require several dotnet versions to build (like
BeatSaberModManager) to properly depend on the dotnet-sdk specific deps.
This in turns avoids having to regenerate the deps of those packages
after each dotnet-sdk update.

This also changes nuget-to-nix to accept a file with a list of
exclusions instead of a folder.
2023-03-19 20:53:39 +01:00
github-actions[bot]
c70342c89c
Merge master into staging-next 2023-03-19 00:02:27 +00:00
Winter
5686f0064d rustPlatform.importCargoLock: add support for git dependencies that use workspace inheritance
Rust 1.64.0 added support for workspace inheritance, which allows
for crates to inherit values such as dependency version constraints or
package metadata information from their workspaces [0].

This works by having workspace members specify a value as a table, with
`workspace` set to true. Thus, supporting this in importCargoLock is as
simple as walking the crate's Cargo.toml, replacing inherited values
with their workspace counterpart.

This is also what a forthcoming Cargo release will do for `cargo vendor` [1],
but we can get ahead of it ;)

[0]: https://blog.rust-lang.org/2022/09/22/Rust-1.64.0.html#cargo-improvements-workspace-inheritance-and-multi-target-builds
[1]: https://github.com/rust-lang/cargo/pull/11414
2023-03-18 20:34:59 +01:00
github-actions[bot]
4742d1b3d6
Merge staging-next into staging 2023-03-18 18:01:35 +00:00
github-actions[bot]
403b148aa5
Merge master into staging-next 2023-03-18 18:01:02 +00:00
Robert Hensing
1dd6d307f7
Merge pull request #221766 from amjoseph-nixpkgs/pr/trivial-builders/test/fix-eval
trivial-builders/test/references.nix: fix eval
2023-03-18 15:20:53 +01:00
github-actions[bot]
00623e191a
Merge staging-next into staging 2023-03-18 14:01:22 +00:00
github-actions[bot]
7f0b38bc9a
Merge master into staging-next 2023-03-18 14:00:45 +00:00
Linus Heckemann
1ba1b35d7f
Merge pull request #183874 from zhaofengli/bwrap-fhs-preserve-etc-symlink
build-fhs-userenv-bubblewrap: Preserve symlinks in /etc
2023-03-18 13:15:44 +01:00
github-actions[bot]
ad9c132f13
Merge staging-next into staging 2023-03-18 06:01:47 +00:00