Commit Graph

2874 Commits

Author SHA1 Message Date
Randy Eckenrode
d5e2c40823
darwin.stdenv: remove darwin.ICU from the final overlay
Nothing in the stdenv needs it. Keeping it there pulls bootstrap tools into its build environment.
2024-07-22 20:54:22 -04:00
Artturin
60e9cffe2c Merge branch 'master' into staging-next 2024-07-22 22:53:31 +03:00
Randy Eckenrode
312e20a312
darwin.stdenv: fix scons Python override
This reduces the number of Python builds in the bootstrap to two: a minimal build and a normal build. Both have LTO disabled, which is required due to missing LLVM LTO libraries. This is necessary to correctly enable LTO builds in Python because it needs `llvm-ar` from `stdenv.cc.cc.libllvm`, which does not exist in the bootstrap.
2024-07-20 21:32:38 -04:00
Audrey Dutcher
c3c71a9809 [squashme] finish rename 2024-07-20 16:40:02 -07:00
Audrey Dutcher
8d34730121 [squashme] elaborate comment on unpack-bootstrap-files.sh 2024-07-20 09:23:29 -07:00
Audrey Dutcher
3cf427ebe3 [squashme] rename linkBS to linkBootstrap 2024-07-20 09:21:23 -07:00
Audrey Dutcher
c494d17409 [squashme] remove unused/bad code 2024-07-20 09:20:33 -07:00
Audrey Dutcher
37affb5b76 [squashme] Remove some hacks 2024-07-19 21:32:08 -07:00
Audrey Dutcher
9dbd5a9a9d [squashme] do the requested changes 2024-07-19 21:32:07 -07:00
Audrey Dutcher
1e2071847d stdenv/freebsd: reinit
The old stdenv didn't work, and was also impure. The new one works, and
is pure. Presently, the bootstrap tools are cross compiled into one small
nar and one large tar, which is then unpacked, patched, and split into
smaller derivations. Efforts were made to make the boot process as short
as possible - there are only two clangs built, and as many packages are
propagated between stages as possible while leaving the bootstrap tools
out of the final stdenv's closure.
2024-07-19 21:32:05 -07:00
toonn
c3c5870e96
Merge pull request #307880 from reckenrode/ld64
cctools: 973.0.1 -> 1010.6
2024-07-17 08:22:14 +02:00
bolives-hax
59dabc0d10
add s390x to make-bootstrap-tools-cross.nix 2024-07-16 20:33:49 +02:00
Robert Scott
b207b6ef74 cc-wrapper: add support for shadowstack hardening flag 2024-07-14 21:25:47 +01:00
Robert Scott
c68739f4f0 bootstrap-tools-musl: match bootstrap-tools hardeningUnsupportedFlags
seems we missed adding stackclashprotection
2024-07-14 21:25:47 +01:00
Yueh-Shun Li
c65cf1b9c5
pkgs/stdenv/generic/setup.sh: fix unbound variables and pass ShellCheck checks (#298831)
* pkgs/stdenv/generic/setup.sh: supress ShellCheck error about Bash array indexing syntax

* pkgs/stdenv/generic/setup.sh: loop by per-line read

* pkgs/stdenv/generic/setup.sh: fix variable quoting

* pkgs/stdenv/generic/setup.sh: separate declaration and assignment
2024-07-14 19:17:55 +03:00
Randy Eckenrode
a49dbb2544
darwin.stdenv: switch to top-level cctools and ld64 2024-07-13 17:54:36 -04:00
Randy Eckenrode
10c87ee2c7
stdenv: set NIX_DONT_SET_RPATH_FOR_TARGET on Darwin 2024-07-13 17:54:35 -04:00
Randy Eckenrode
9403fdc4a6
darwin.stdenv: consolidate stage 2 into one stage
Separate stages are no longer necessary because CF is not built from source and will not be built in the future.
2024-07-13 17:54:35 -04:00
Randy Eckenrode
05e5d7f73e
darwin.stdenv: clean up GNU binutils ban
- Only propagate binutils-unwrapped; and
- Clarify in the `throw` that it cannot be used in the Darwin bootstrap.
2024-07-13 17:54:35 -04:00
Randy Eckenrode
78da51cdb0
darwin.stdenv: adjust flags for llvm-strip in bootstrap tools 2024-07-13 17:54:35 -04:00
Randy Eckenrode
8559d6466b
darwin.stdenv: always use a response file
The bootstrap tools have been updated, so it is no longer necessary to suppress using a response file in the Darwin stdenv bootstrap.
2024-07-13 17:54:35 -04:00
Randy Eckenrode
65a37e7b14
darwin.stdenv: make sure curl cannot be used
Fetchers can use the `curl` binary from the bootstrap tools. Allowing packages in the Darwin bootstrap to link curl makes any curl update cause a full rebuild on Darwin, which is undesirable.
2024-07-13 17:54:35 -04:00
Martin Weinelt
176a56c40f
Merge remote-tracking branch 'origin/staging-next' into staging
Conflicts:
- pkgs/applications/misc/blender/default.nix
- pkgs/development/web/nodejs/nodejs.nix
2024-07-11 01:06:28 +02:00
John Ericson
000b58721f Merge remote-tracking branch 'upstream/master' into openbsd-static 2024-07-09 18:18:47 -04:00
John Ericson
676df1cf2d openbsd: Add static linking support
I've had better luck creating statically-linked binaries that work than
dynamically-linked ones, so this is needed quite practically.
2024-07-09 18:17:06 -04:00
Tristan Ross
cfe063d174
overrideSDK: fix missing host platform inside of override 2024-07-08 17:21:55 -07:00
Tristan Ross
7a95a8948b
Merge pull request #324155 from paparodeo/x64-sdk11-no-rebuilds
treewide: change various flags to allow x64 darwin to default to sdk 11.0 when ready
2024-07-08 17:19:17 -07:00
Martin Weinelt
e7b1ee45cf
Merge remote-tracking branch 'origin/staging-next' into staging
Conflicts:
- pkgs/tools/misc/watchlog/default.nix
2024-07-06 01:52:35 +02:00
Randy Eckenrode
75504615ad
stdenv/darwin/make-bootstrap-tools.nix: use darwin.binutils-unwrapped
After https://github.com/NixOS/nixpkgs/pull/322388, darwin.cctools is just Apple’s cctools again. The replacement for what the bootstrap tools wants is darwin.binutils-unwrapped.
2024-07-04 10:45:08 -04:00
Ryan Hendrickson
40428630bc
Merge pull request #317106 from trofi/dashed-source
stdenv: handle $sourceRoot that starts with dash
2024-07-02 23:48:25 -04:00
Reno Dakota
3cb23cec23
treewide: fixes to allow x64 darwin to default to sdk 11
update code to not assume that x64 darwin must use sdk 10.12. After this
change it's possible to build a sdk 11 stdenv on darwin x64
2024-07-02 23:30:01 +00:00
Randy Eckenrode
c6e9b98725
darwin.stdenv: make sure GNU binutils cannot be used
GNU binutils is not preferred on Darwin, and newer versions have issues building. Make it an evaluation error to use it in the Darwin stdenv bootstrap.
2024-06-27 08:10:34 -04:00
Randy Eckenrode
2b21ece48b
darwin.stdenv: update for darwin.binutils cleanup
- Only link binaries that exist for stage 0 cctools and LLVM bintools;
- Drop cctools-llvm in favor of the updated darwin.binutils; and
- Update llvm-manages Python overrides (needed for newer versions of LLVM).
2024-06-27 08:10:34 -04:00
Randy Eckenrode
e1ab7a10d8
darwin.stdenv: run LLVM tests only once 2024-06-27 07:33:47 -04:00
Randy Eckenrode
bb96a168d7
darwin.stdenv: reduce number of times Python is built 2024-06-27 07:33:47 -04:00
K900
e2665b3080 stdenv: fix typo 2024-06-24 10:17:10 +03:00
Martin Weinelt
2096642430
Merge remote-tracking branch 'origin/staging-next' into staging
Conflicts:
- pkgs/os-specific/windows/mingw-w64/default.nix
2024-06-23 19:09:00 +02:00
K900
1374c58013
Merge pull request #320370 from eclairevoyant/fix-checkmeta-predicates
check-meta: fix instructions
2024-06-22 14:28:12 +03:00
éclairevoyant
dbef1af49e
check-meta: rename local binding 2024-06-22 05:16:10 -04:00
John Ericson
744d7e7477 *bsdCross: Remove these package sets
I realized what rhelmot did in 61202561d9
(specify what packages just need `stdenvNoLibc`) is definitely the right
approach for this, and adjusted NetBSD and OpenBSD to likewise use it.
With that change, we don't need these confusing and ugly `*bsdCross`
package sets at all!

We can get rid of a lot more libc-related `*Cross`, and I will do so
soon, but this is the first step.

(adapted from commit 51f1ecaa59)
2024-06-21 11:47:36 -04:00
Vladimír Čunát
879821772c
Revert #320852: Clean up cross bootstrapping
It rebuilt stdenv on *-darwin; we can't do that in nixpkgs master.
This reverts commit 2f20501c5f, reversing
changes made to fd469c24af.
2024-06-21 07:48:24 +02:00
Rick van Schijndel
43ce0f9ee0
Merge pull request #318256 from risicle/ris-stack-clash-protection
cc-wrapper: add stack clash protection hardening flag
2024-06-19 19:54:30 +02:00
Ryan Lahfa
193b2c7637
Merge pull request #310387 from Qyriad/diag/log-hooks2
stdenv: log hooks as they run (take II)
2024-06-19 14:22:10 +02:00
John Ericson
51f1ecaa59 Clean up cross bootstrapping
For a long time, we've had `crossLibcStdenv`, `*Cross` libc attributes,
and `*bsdCross` pre-libc package sets. This was always bad because
having "cross" things is "not declarative": the naming doesn't reflect
what packages *need* but rather how we *provide* something. This is
ugly, and creates needless friction between cross and native building.

Now, almost all of these `*Cross` attributes are gone: just these are
kept:

- Glibc's and Musl's are kept, because those packages are widely used
  and I didn't want to risk changing the native builds of those at this
  time.

- generic `libcCross`, `theadsCross`, and friends, because these relate
  to the convolulted GCC bootstrap which still needs to be redone.

The BSD and obscure Linux or freestnanding libcs have conversely all
been made to use a new `stdenvNoLibc`, which is like the old
`crossLibcStdenv` except:

1. It usable for native and cross alike

2. It named according to what it *is* ("a standard environment without
   libc but with a C compiler"), rather than some non-compositional
   jargon ("the stdenv used for building libc when cross compiling",
   yuck).

I should have done this change long ago, but I was stymied because of
"infinite recursions". The problem was that in too many cases we are
overriding `stdenv` to *remove* things we don't need, and this risks
cyles since those more minimal stdenvs are used to build things in the
more maximal stdenvs.

The solution is to pass `stage.nix` `stdenvNoCC`, so we can override to
*build up* rather than *tear down*. For now, the full `stdenv` is also
passed, so I don't need to change the native bootstraps, but I can see
this changing as we make things more uniform and clean those up.

Finally, the BSDs also had to be cleaned up, since they have a few
pre-libc dependencies, demanding a systematic approach. I realized what
rhelmot did in 61202561d9 (specify what
packages just need `stdenvNoLibc`) is definitely the right approach for
this, and adjusted NetBSD and OpenBSD to likewise use it.
2024-06-18 17:04:16 -04:00
éclairevoyant
396629c7b6
check-meta: fix instructions
Corrections to 17718ac255
2024-06-16 18:18:42 -04:00
Sergei Trofimovich
193fd8a237 generic/setup.sh: handle $sourceRoot that starts with dash
Without the change `runPhase` fails on tarballs like
`diffoscope-269` that contain single top-level `-269` root as:

    diffoscope> unpacking source archive /nix/store/p620nidkm73vrp0z6kk5krmrm4vg7bxd-diffoscope-269.tar.bz2
    diffoscope> source root is -269
    diffoscope> setting SOURCE_DATE_EPOCH to timestamp 1717143039 of file ./-269/tests/utils/versions.py
    diffoscope> chmod: invalid mode: ‘-269’
    diffoscope> Try 'chmod --help' for more information.

Currently `diffoscope-269` has a `sourceRoot = "./-269";` workaround to
bypass the failure.
2024-06-10 23:56:46 +01:00
Franz Pletz
3db93c351d cc-wrapper: add stack clash protection hardening flag
Most Linux distributions are enabling this these days and it does
protect against real world vulnerabilities as demonstrated by
CVE-2018-16864 and CVE-2018-16865.

Fix #53753.

Information on llvm version support gleaned from
6609892a2d
68e07da3e5
092507a730

Information on gcc version support a lot harder to gather,
but both 32bit and 64bit arm do appear to be supported
based on the test suite.
2024-06-07 20:23:46 +01:00
Silvan Mosberger
a3fad6299c
Merge pull request #314553 from gefla/annotate-substitutestream
Annotate substituteStream deprecation warning
2024-06-03 15:50:33 +02:00
Arnout Engelen
d57327020e
Merge pull request #297987 from raboof/check-meta-fix-allow-predicate-instructions
check-meta: fix 'predicate' instructions
2024-05-30 18:43:35 +02:00
Gerd Flaig
1949b0d16b Annotate substituteStream deprecation warning
This adds the derivation name to the warning to make debugging easier.
2024-05-25 18:32:10 +02:00
Randy Eckenrode
4392e5a53c
overrideSDK: handle propagated lists of inputs
Improves `overrideSDK` compatibility with https://github.com/oxalica/rust-overlay.
2024-05-23 23:21:33 -04:00
Qyriad
7acc356606 stdenv: log hooks as they run (take II)
A second take at eb28e5e72e, which was reverted for the extra logging
during the internals of `nix-shell -p`. This commit does the same
logging, but to $NIX_LOG_FD instead, which is echoed during any normal
build, but not during the internals of `nix-shell -p`.

[1]: eb28e5e72e
2024-05-22 16:28:31 -06:00
github-actions[bot]
0471a8be05
Merge staging-next into staging 2024-05-03 00:03:09 +00:00
github-actions[bot]
3588aea450
Merge master into staging-next 2024-05-03 00:02:39 +00:00
Artturin
124c828155
Merge pull request #269461 from Artturin/addcrossreplace
config.replaceCrossStdenv: add
2024-05-03 01:25:05 +03:00
Randy Eckenrode
c88b49062a
libiconv: use libiconv-darwin 2024-04-27 18:09:53 -04:00
Randy Eckenrode
89c9b73ce7
darwin.stdenv: propagate atf and kyua
Avoid building these packages more than once. Even though they require
linking to dylibs, they’re only used for running tests.
2024-04-27 18:09:53 -04:00
Randy Eckenrode
ad38102a67
darwin.stdenv: avoid an infinite recursion
libiconv-darwin depends on Meson, which (indirectly) depends on
libiconv. When libiconv-darwin is set as libiconv, it will cause an
infinite recursion. Avoid the infinite recursion by using libiconvReal
in stage 1. Every stage after that can use libiconv-darwin.
2024-04-27 18:09:52 -04:00
David McFarland
c642665a04 stdenv: fix missing dependencies in __sandboxProfile and __impureHostDeps
Fixes: 7f3ca3e21a (stdenv: Fix handling of dependencies and hooks)
Fixes: #237458
2024-04-19 15:02:27 -03:00
Vladimír Čunát
8c0c955a09
Merge branch 'master' into staging-next 2024-04-19 07:52:28 +02:00
sternenseemann
7be562d046 wrapCC, wrapBintools: move expand-response-params bootstrapping out
The cc and bintools wrapper contained ad hoc bootstrapping logic for
expand-response-params (which was callPackage-ed in a let binding). This
lead to the strange situation that the bootstrapping logic related to
expand-response-params is split between the wrapper derivations (where
it is duplicated) and the actual stdenv bootstrapping.

To clean this up, the wrappers simply should take expand-response-params
as an ordinary input: They need an adjacent expand-response-params (i.e.
one that runs on their host platform), but don't care about the how.
Providing this is only problematic during stdenv bootstrapping where we
have to pull it from the previous stage at times.
2024-04-18 20:49:13 +02:00
sternenseemann
b2a568906a wrapCC, wrapBintools: use runtimeShell instead of stdenv shell
We don't need to artificially make sure that we can execute the wrapper
scripts on the build platform by using stdenv's shell (which comes from
buildPackages) since our cross infrastructure will get us the wrapper
from buildPackages. The upside of this change is that cross-compiled
wrappers (e.g. pkgsCross.aarch64-multiplatform.gcc) will actually work
when executed!

For bootstrapping this is also not a problem, since we have a long
build->build platform chain so runtimeShell is just as good as
stdenvNoCC.shell. We do fall back to old ways, though, by explicitly
using the bootstrap-tools shell in stage2, so the adjacent bash is only
used from stage4 onwards. This is unnecessary in principle (I'll try
removing this hack in the future), but ensures this change causes zero
rebuilds.
2024-04-18 20:49:13 +02:00
github-actions[bot]
ae22f8403a
Merge master into staging-next 2024-04-18 18:00:58 +00:00
John Ericson
0820a71359
Merge pull request #304033 from sternenseemann/wrapper-cleanups
wrapCC, wrapBintools, stdenv: make Nix code a bit more consistent
2024-04-18 10:57:18 -04:00
github-actions[bot]
4908c48bf5
Merge master into staging-next 2024-04-15 12:01:00 +00:00
sternenseemann
0198f43efb stdenv: consistently use self: super: … for bootstrapping overlays
As of late, `final: prev: …` for overlays has become more prevalent in
newer code. This is also exhibited in some code (presumably added
recently) in stdenv. This change is not about any merits or demerits of
any naming convention, but rather aims to make the nomenclature in
stdenv bootstrapping consistent to lessen confusion.

I've chosen to stick to `self: super: …` convention because:

1. It is more common in the code as it stands.

2. Using `final: prev: …` makes the code more confusing, als it causes
   `prev` to be in scope alongside `prevStage`. `prevStage` actually
   bears no relation to `prev` even though their naming suggests it,
   making it easy to confuse them (the former is the (final) package set
   of the previous stage while the latter is just the `prev`/`super` of
   the overlay “chaining” on a completely fresh package set, i.e. `prev`
   doesn't even relate to the previous stage's `overrides` argument).

This change also corrects a naming error in stdenv/native which had no
effect, as the variables were unused.
2024-04-15 12:53:13 +02:00
Jade Lovelace
78945a827c stdenv: make inputDerivation never fixed-output
This fixes using inputDerivation on derivations that are fixed-output.

Previously:

```
nix-repl> drv = runCommand "huh" { outputHash = "sha256-47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU="; outputHashAlgo = "sha256"; outputHashType = "flat"; } "touch $out"

nix-repl> drv.inputDerivation
«derivation /nix/store/d8mjs6cmmvsr1fv7psm6imis5pmh9bcs-huh.drv»

nix-repl> :b drv.inputDerivation
error: fixed output derivation 'huh' is not allowed to refer to other store paths.
       You may need to use the 'unsafeDiscardReferences' derivation attribute, see the manual for more details.
```

Fixes: https://github.com/NixOS/nixpkgs/issues/304209
2024-04-15 00:52:28 -07:00
github-actions[bot]
b50b9be6e4
Merge master into staging-next 2024-04-13 06:00:55 +00:00
Dan Callahan
ec6c594944 stdenv/check-meta: Fix error message for disallowed unfree packages
Nixpkgs tries to print a helpful message when it blocks unfree packages,
but the suggestion is subtly broken. The predicate only matches on the
package's name, but the suggestion includes the full name-version pair.

Fixed by formatting the message with the same function as the predicate.

This issue arises because check-meta defines its own local getName with
semantics divergent from lib.getName. The former includes the version,
the latter does not.

Example Before:

    Alternatively you can configure a predicate to allow specific packages:
      { nixpkgs.config.allowUnfreePredicate = pkg: builtins.elem (lib.getName pkg) [
          "obsidian-1.5.12"
        ];
      }

Example After:

    Alternatively you can configure a predicate to allow specific packages:
      { nixpkgs.config.allowUnfreePredicate = pkg: builtins.elem (lib.getName pkg) [
          "obsidian"
        ];
      }

Fixes #303116
2024-04-10 16:15:24 +01:00
Vladimír Čunát
3299c31f44
stdenv: avoid setuid issues
See #300635.  Maybe in time we'll have a better solution.
2024-04-10 07:52:44 +02:00
Weijia Wang
3f59355d84 Merge branch 'staging-next' into staging 2024-04-09 08:26:23 +02:00
annalee
8e7f7b4fa0 stdenv.darwin: bootstrap darwin using updated tools
- update the hashes and tools needed to extract the bootstrap-tools archive
- unify the x64 and aarch64 unpack process
2024-04-07 18:33:42 +02:00
adisbladis
509577b789
Merge pull request #297353 from adisbladis/check-meta-validity-only
stdenv/check-meta: Make `checkValidity` only check validity
2024-04-07 22:49:50 +12:00
github-actions[bot]
882e0f27d3
Merge staging-next into staging 2024-04-05 18:01:27 +00:00
Ryan Lahfa
36cd37b5dc
Merge pull request #297074 from lolbinarycat/tarfix
stdenv: pass --mode=+w to tar, ensuring extraction is possible
2024-04-04 13:51:39 -07:00
Sergei Trofimovich
178f1e1f5d pkgs/top-level/release.nix: drop unused dist targets
The PR https://github.com/NixOS/nixpkgs/pull/284090 exposed `build`
target directly on hydra. We are using it now in
https://github.com/NixOS/nixpkgs/tree/master/maintainers/scripts/bootstrap-files
to generate bootstrap tarballs on `linux` and `darwin`.

`dist` was not very useful as it was a bit hard to link back to hydra
jobs that build it. Let's just drop it.
2024-04-04 21:40:08 +01:00
adisbladis
000f61a610 stdenv/check-meta: Make checkValidity only check validity
checkValidity has the responsibility to check if a derivation's attributes are valid.
Previously it also had the overloaded task of creating a subset of meta attributes:
- unfree
- broken
- unsupported
- insecure

Not only is this overloading strange, these attributes were only ever consumed by `commonMeta`.

This change makes checkValidity _only_ check for validity, and removes the creation of any meta attributes from `checkValidity` and moves them to `commonMeta`.

This is technically a breaking change but I don't expect any external nixpkgs consumers to rely on these implementation details.
2024-04-03 21:38:43 +13:00
github-actions[bot]
fef3e5b690
Merge staging-next into staging 2024-04-03 00:02:45 +00:00
annalee
39b3d15ad8
freshBootstrapTools.{build,test}: group tools together 2024-04-01 22:19:04 +00:00
annalee
5748bfdd76
freshBootstrapTools.{build,test}: allow exes with libs unpack 2024-04-01 22:19:04 +00:00
annalee
72870b3293
freshBootstrapTools.{build,test}: fix as script. test all executables
as is a wrapper around clang and the path to bash and clang need to be
updated when unpacked

run all executables in bin get run in the test
2024-04-01 22:17:02 +00:00
github-actions[bot]
d7ece48399
Merge staging-next into staging 2024-03-30 18:01:46 +00:00
github-actions[bot]
50812f5204
Merge master into staging-next 2024-03-30 18:01:17 +00:00
K900
59bccda153 Revert "stdenv: add meta.repository field"
This reverts commit 7e1443abbb.
2024-03-30 17:19:21 +03:00
K900
0e8bcaa661 Revert "stdenv/check-meta: Don't create new environments when computing meta.repository"
This reverts commit f8b091d53f.
2024-03-30 17:18:47 +03:00
github-actions[bot]
49b966576c
Merge staging-next into staging 2024-03-30 12:01:34 +00:00
github-actions[bot]
48d06167c6
Merge master into staging-next 2024-03-30 12:01:06 +00:00
K900
ba79149c66 stdenv/check-meta: don't infrec on unsupported platforms 2024-03-30 14:30:20 +03:00
adisbladis
bff4c55f3a
Merge pull request #300177 from adisbladis/meta-repository-perf-fixups
stdenv/check-meta: Fix performance regressions introduced in #294347
2024-03-30 20:22:33 +13:00
adisbladis
dc49349c84 stdenv/check-meta: Inherit remaining lib access into scope
Hopefully this will result in people not adding new `lib.xxx` to check-meta.nix.
2024-03-30 19:20:59 +13:00
adisbladis
f8b091d53f stdenv/check-meta: Don't create new environments when computing meta.repository 2024-03-30 19:19:05 +13:00
adisbladis
c605bfd501
Merge pull request #297350 from adisbladis/commonmeta-output-list
stdenv/check-meta: Remove outputsToInstall list concat from common meta
2024-03-30 19:07:32 +13:00
github-actions[bot]
14d4fad30c
Merge staging-next into staging 2024-03-30 00:02:47 +00:00
github-actions[bot]
04a895835e
Merge master into staging-next 2024-03-30 00:02:16 +00:00
a-n-n-a-l-e-e
2809c84cfb
Merge pull request #294347 from lolbinarycat/meta.repository
stdenv: add meta.repository field
2024-03-29 13:03:04 -07:00
binarycat
7e1443abbb stdenv: add meta.repository field 2024-03-29 14:34:09 -04:00
a-n-n-a-l-e-e
7ad496eb80
Merge pull request #300032 from NixOS/revert-295558-darwin-bootstrap-update
Revert "stdenv.darwin: bootstrap darwin using updated tools"
2024-03-29 11:05:07 -07:00
a-n-n-a-l-e-e
6b44d9d62d
Revert "stdenv.darwin: bootstrap darwin using updated tools" 2024-03-29 09:33:31 -07:00
Randy Eckenrode
71c6ee9295
darwin.stdenv: add sdkRoot to extraNativeBuildInputs
Setting the SDK root by default allows `overrideSDK` to correctly set
the SDK version when using a different SDK. It also allows the correct
SDK version to be set when using an older deployment target. Not setting
the correct SDK version can result in unexpected behavior at runtime.

Examples:

* Automatic dark mode switching requires linking against an SDK version
  of 10.14 or newer. With the current behavior, the only way to do this
  is by using a 10.14+ deployment target even when the application
  supports older platforms when build with a newer SDK.
* MetalD3D checks that the system version is at least 14.0. The API it
  uses returns a compatibility version when the the SDK is older than
  11.0, which causes it to display an error and terminate the
  application even when even when its requirements are all met.
2024-03-29 10:07:50 -04:00
Randy Eckenrode
f61e189ad3
overrideSDK: support all deps attributes
This is effectively a rewrite of `overrideSDK`. It was required because
`wrapGAppsHook` propagates `depsTargetTarget` with the expectation that
it will effectively be `buildInputs` when the hook is itself used as a
`nativeBuildInput`. This propagates Gtk, which itself propagates the
default Dariwn SDK, making it effectively impossible to override the SDK
when a package depends on Gtk and uses `wrapGAppsHook`.

This rewrite implements the following improvements:

* Cross-compilation should be supported correctly (untested);
* Supports public and private frameworks;
* Supports SDK `libs`;
* Remaps instead of replacing extra (native) build inputs in the stdenv;
* Updates any Darwin framework references in `nix-support`; and
* It updates `xcodebuild` regardless of which input its in.

The implementation avoids recursion for performance reasons. Instead, it
enumerates transitive dependencies and walks the list from the leaf
packages backwards to the parent packages.
2024-03-29 10:03:08 -04:00
Randy Eckenrode
1f231173a5
darwin.stdenv: drop curl from assertions 2024-03-29 10:00:30 -04:00
Randy Eckenrode
017eda24ae
darwin.stdenv: use wrapBintoolsWith instead of a direct import 2024-03-29 10:00:28 -04:00
annalee
f8ad6442d5
stdenv.darwin: bootstrap darwin using updated tools
- update the hashes and tools needed to extract the bootstrap-tools archive
- unify the x64 and aarch64 unpack process
- unpin libxml for python-minimal
2024-03-22 13:27:27 +00:00
Arnout Engelen
17718ac255
check-meta: fix 'predicate' instructions
In the code examples for using `allowUnfreePredicate` and
`allowNonSourcePredicate`, we use `lib.getName` (which removes
the version) and not the local `getName` (which keeps the version).
2024-03-22 09:42:53 +01:00
Randy Eckenrode
b7d77e743c
Merge pull request #296750 from reckenrode/darwin-nocurl
darwin.stdenv: drop curl from the bootstrap
2024-03-21 23:42:15 -04:00
Robert Hensing
8f6b396c49
Merge pull request #297711 from hercules-ci/rename-adapters-finalAttrs
pkgs/stdenv/adapters.nix: Rename internal name
2024-03-21 17:15:28 +01:00
Robert Hensing
8da01484cd pkgs/stdenv/adapters.nix: Rename internal name
`finalAttrs` is never the first argument. This should have been
called `prevAttrs` all along.
It confused me for a bit, because the callback _must not_ be strict
in `finalAttrs` (the first of _two_ parameters), or it will
recurse infinitely while trying to figure out what the attrNames are.
2024-03-21 13:51:50 +01:00
github-actions[bot]
36232fff71
Merge master into staging-next 2024-03-20 18:01:02 +00:00
Connor Baker
e871fcf859 stdenvAdapters.useLibsFrom: use targetStdenv.cc.override
As @SomeoneSerge pointed out in https://github.com/NixOS/nixpkgs/pull/281371#discussion_r1531174155,
by avoiding `wrapCCWith` and using `targetStdenv.cc.override`, we avoid roundtrip wrapping and
are able to use `coreutils` from `targetStdenv`.
2024-03-20 16:02:08 +00:00
adisbladis
1e2f8f2a84 stdenv/check-meta: Remove outputsToInstall list concat from common meta
Normally either of "bin" or "out" will hit first so we can avoid dynamic looping altogether.
2024-03-20 16:29:17 +13:00
annalee
3cdd8d05f0
stdenv.darwin.make-boostrap-tools: include unpack script in the archive 2024-03-19 17:28:32 +00:00
annalee
b310830359
stdenv.darwin.make-boostrap-tools: update tools and unpin LLVM11
- unpin LLVM11. fix discrepancy with freshBootstrapTools and the tools
  built on hydra. pinning the stdenv for the hydra build doesn't pin the
  tools as the included packages are able to change.

- remove unused LLVM tools & libs which reduces the uncompressed and
  compressed file sizes by more than 1/2. compressed tarball is now 40M
  and uncompressed is around 200M

- add @loader_path/. to dylibs that reference other libs in the archive.
  this is needed for libraries with re-exports.

- validate shared objects with @rpath references contain the reference
  in lib

- add a test to verify that the @loader_path/ works for libc++ as it
  re-exports libc++abi
2024-03-19 17:28:32 +00:00
annalee
88f192b247
stdenv.darwin.make-boostrap-tools: refactor to use getBin, getDev and getLib 2024-03-19 17:28:32 +00:00
annalee
7c4b2a599c
freshBootstrapTools: LLVM11 unpin 2024-03-19 14:54:23 +00:00
binarycat
f9094faf70 stdenv: pass --mode=+w to tar, ensuring extraction is possible
fixes #191137
2024-03-18 20:54:31 -04:00
github-actions[bot]
ac91cf83c1
Merge master into staging-next 2024-03-18 00:02:20 +00:00
Randy Eckenrode
07451425e7
darwin.stdenv: drop curl from the bootstrap
The Darwin bootstrap currently requires curl, but it is not strictly
required. The bootstrap requires it for two things:

* Fetchers; and
* As a transitive dependency of llvm-manpages (via Sphinx).

For the fetchers, the bootstrap curl can be used. For hatch-vcs, the
dependency, its tests can be disabled. Doing this allows curl to be
dropped from the Darwin stdenv bootstrap.
2024-03-17 16:48:35 -04:00
Robert Hensing
05f4b7b46d make-derivation.nix: Fix checkMetaRecursively
Oddly, I can't reproduce the error, but this change will make it
more robust.
See https://github.com/NixOS/nixpkgs/pull/295378#issuecomment-2002094487
2024-03-17 19:28:47 +01:00
github-actions[bot]
c160165e60
Merge master into staging-next 2024-03-14 18:01:26 +00:00
Robert Hensing
ba463e70e3 make-derivation.nix: Float out unsafeDerivationToUntrackedOutpath 2024-03-12 20:15:47 +01:00
Robert Hensing
0ab7b23637 make-derivation.nix: Update inline docs 2024-03-12 19:06:07 +01:00
Robert Hensing
797ad5ae87 make-derivation.nix: Float out new constant removedOrReplacedAttrNames 2024-03-12 18:43:11 +01:00
Robert Hensing
cac23248b1 make-derivation.nix: Float out knownHardeningFlags 2024-03-12 18:41:57 +01:00
Robert Hensing
464d8c4a6c make-derivation.nix: Float inward darwin-specific derivation attributes 2024-03-12 18:41:18 +01:00
Robert Hensing
2fb7255cc0 make-derivation.nix: Evaluate flip 2024-03-12 18:39:19 +01:00
Robert Hensing
90197b6833 make-derivation.nix: Apply map composition law
... after inlining chooseDevOutputs.
2024-03-12 18:37:53 +01:00
Robert Hensing
954d9ce64b make-derivation.nix: Drop duplicate functionality from previous commit
These two commits make for a cleaner commit history and git blame than
https://github.com/NixOS/nixpkgs/pull/295105, where this refactor
was developed.

See its commit messages for details and design choices, esp. up to
and including 37f76fd4c3.
2024-03-12 18:32:03 +01:00
Robert Hensing
8dad51a2e2 make-derivation.nix: Split makeDerivationArgument, mkDerivation with duplicate functionality 2024-03-12 18:16:56 +01:00
github-actions[bot]
06e5eb63d3
Merge staging-next into staging 2024-03-11 18:01:53 +00:00
Robert Hensing
de516f6f13 make-derivation.nix: Return mkDerivation as an attribute 2024-03-11 17:06:02 +01:00
Robert Hensing
ab56d3ac71 make-derivation.nix: Move into let binding 2024-03-11 17:02:59 +01:00
a-n-n-a-l-e-e
ce789e7e35
llvmPackages_{12,13,14,15,16,17,git}.{libcxx,libcxxabi}: merge libcxxabi into libcxx (#292043)
- merge libcxxabi into libcxx for LLVM 12, 13, 14, 15, 16, 17, and git.
- remove the link time workaround `-lc++ -lc++abi` from 58 packages as it is no longer required.
- fixes https://github.com/NixOS/nixpkgs/issues/166205
- provides alternative fixes for. https://github.com/NixOS/nixpkgs/issues/269548 https://github.com/NixOS/nix/issues/9640
- pkgsCross.x86_64-freebsd builds work again

This change can be represented in 3 stages
1. merge libcxxabi into libcxx -- files: pkgs/development/compilers/llvm/[12, git]/{libcxx, libcxxabi}
2. update stdenv to account for merge -- files: stdenv.{adapters, cc.wrapper, darwin}
3. remove all references to libcxxabi outside of llvm (about 58 packages modified)

### merging libcxxabi into libcxx
- take the union of the libcxxabi and libcxx cmake flags
- eliminate the libcxx-headers-only package - it was only needed to break libcxx <-> libcxxabi circular dependency
- libcxx.cxxabi is removed. external cxxabi (freebsd) will symlink headers / libs into libcxx.
- darwin will re-export the libcxxabi symbols into libcxx so linking `-lc++` is sufficient.
- linux/freebsd `libc++.so` is a linker script `LINK(libc++.so.1, -lc++abi)` making `-lc++` sufficient.
- libcxx/default.nix [12, 17] are identical except for patches and `LIBCXX_ADDITIONAL_LIBRARIES` (only used in 16+)
- git/libcxx/defaul.nix  does not link with -nostdlib when useLLVM is true so flag is removed. this is not much different than before as libcxxabi used -nostdlib where libcxx did not, so libc was linked in anyway.

### stdenv changes
- darwin bootstrap, remove references to libcxxabi and cxxabi
- cc-wrapper: remove c++ link workaround when libcxx.cxxabi doesn't exist (still exists for LLVM pre 12)
- adapter: update overrideLibcxx to account for a pkgs.stdenv that only has libcxx

### 58 package updates
- remove `NIX_LDFLAGS = "-l${stdenv.cc.libcxx.cxxabi.libName}` as no longer needed
- swift, nodejs_v8 remove libcxxabi references in the clang override

https://github.com/NixOS/nixpkgs/pull/292043
2024-03-11 03:53:37 -07:00
OPNA2608
5ec7dcd7c5 pkgs/stdenv/linux: init powerpc64-unknown-linux-gnuabielfv2 bootstrap-files
sha256sum of files to be uploaded:

$ sha256sum /nix/store/8frm8kk8gzpv31r289ai5jgkwfikmpm4-stdenv-bootstrap-tools-powerpc64-unknown-linux-gnuabielfv2/on-server/*
0af311476b54b399f3024b92e9c518363acd2b15f713e83d0bb4fb3f8f26d98b  /nix/store/8frm8kk8gzpv31r289ai5jgkwfikmpm4-stdenv-bootstrap-tools-powerpc64-unknown-linux-gnuabielfv2/on-server/bootstrap-tools.tar.xz
529a053a8022e89357aaa608e57aaddaa7c3ded93d633916ddca92bb81e22125  /nix/store/8frm8kk8gzpv31r289ai5jgkwfikmpm4-stdenv-bootstrap-tools-powerpc64-unknown-linux-gnuabielfv2/on-server/busybox

Suggested commands to upload files to 'tarballs.nixos.org':

    $ nix-store --realize /nix/store/8frm8kk8gzpv31r289ai5jgkwfikmpm4-stdenv-bootstrap-tools-powerpc64-unknown-linux-gnuabielfv2
    $ aws s3 cp --recursive --acl public-read /nix/store/8frm8kk8gzpv31r289ai5jgkwfikmpm4-stdenv-bootstrap-tools-powerpc64-unknown-linux-gnuabielfv2/on-server/ s3://nixpkgs-tarballs/stdenv/powerpc64-unknown-linux-gnuabielfv2/57cf2e0b24fb52344cc718913eaed78f389b1319
    $ aws s3 cp --recursive s3://nixpkgs-tarballs/stdenv/powerpc64-unknown-linux-gnuabielfv2/57cf2e0b24fb52344cc718913eaed78f389b1319 ./
    $ sha256sum bootstrap-tools.tar.xz busybox
    $ sha256sum /nix/store/8frm8kk8gzpv31r289ai5jgkwfikmpm4-stdenv-bootstrap-tools-powerpc64-unknown-linux-gnuabielfv2/on-server/*
2024-03-10 12:36:34 +01:00
a-n-n-a-l-e-e
cf1a332500
Revert "stdenv: log build hooks as they run" 2024-03-09 17:32:20 -08:00
github-actions[bot]
c3cf93646a
Merge staging-next into staging 2024-03-09 00:02:40 +00:00
Weijia Wang
172fde7e27
Merge pull request #293257 from OPNA2608/add/make-bootstrap-tools-cross-powerpc64
pkgs/stdenv/linux/make-bootstrap-tools-cross: Add powerpc64 entry
2024-03-08 21:00:53 +01:00
github-actions[bot]
a51832837a
Merge staging-next into staging 2024-03-04 18:01:24 +00:00
github-actions[bot]
fe2479b166
Merge master into staging-next 2024-03-04 18:00:55 +00:00
OPNA2608
5f8522eeab pkgs/stdenv/linux/make-bootstrap-tools-cross: Add powerpc64 entry
So Hydra can cross-compile bootstrapFiles for the platform.
2024-03-04 15:01:29 +01:00
a-n-n-a-l-e-e
d7425f0f21
Merge pull request #288250 from trofi/make-bootstrap-tools-cross-new-target
pkgs/stdenv/linux/make-bootstrap-tools-cross.nix: allow entries for t…
2024-03-04 05:18:39 -08:00
Ryan Lahfa
241ca50541
Merge pull request #290081 from Qyriad/diag/log-hooks
stdenv: log build hooks as they run
2024-03-04 00:17:07 +01:00
Qyriad
ff372db9b4
prev: per-review: reword comment regarding logging of implicit hooks
Addresses the following review comment: https://github.com/NixOS/nixpkgs/pull/290081#discussion_r1503503065

This will be squashed into the previous commit pending review.

Co-authored-by: Philip Taron <philip.taron@gmail.com>
2024-02-26 19:59:01 -07:00
Qyriad
9aef09586e prev: apply fixes from code review
Addresses the following review comments:
- eb28e5e72e (r1501466125)
- eb28e5e72e (r1501466232)

This will be squashed into the previous commit pending review.
2024-02-25 22:19:37 -07:00
Robert Scott
4a91b3e798 cc-wrapper: add trivialautovarinit hardening flag support
this equates to -ftrivial-auto-var-init=pattern

clang has removed support for -ftrivial-auto-var-init=zero and
are unlikely to re-add it, so use -ftrivial-auto-var-init=pattern
on both compilers if only to make behaviour more consistent
between the two.

add to pkgsExtraHardening's defaultHardeningFlags.
2024-02-24 12:00:09 +00:00
Qyriad
eb28e5e72e stdenv: log build hooks as they run 2024-02-17 23:15:09 -07:00
Vladimír Čunát
e81960770a
Merge #288866: pkgs/stdenv/linux: update x86_64-unknown-linux-gnu bootstrap-files
...into staging
2024-02-17 10:38:31 +01:00
Vladimír Čunát
eff11adc57
Merge #286721: stdenv: fix rare tar.xz decompression issues
...into staging
2024-02-17 08:45:01 +01:00
Sergei Trofimovich
a0f80e965c pkgs/stdenv/linux/make-bootstrap-tools-cross.nix: allow entries for to-be uplaoded targets
Newly added targets should be fine to be added as hudra jobs before the
first uplaod happens.

Co-authored-by: Cosima Neidahl <opna2608@protonmail.com>
2024-02-15 09:08:58 +00:00
Isidor Zeuner
11a19109b6 stdenv: disregard xz exit status in order to fix subtle decompression issues
There is a subtle bug with unpacking `tar.xz` archives which seems to happen only on some setups, and sometimes not in a reproducible manner (https://github.com/NixOS/nixpkgs/issues/278130, https://github.com/NixOS/nixpkgs/issues/20950). On the last occurrence, it could be tracked down to `xz` failing from a `SIGPIPE`, which can happen when it's connected to `tar` through a pipe and `tar` exits earlier (see e.g. https://www.linuxquestions.org/questions/slackware-14/%5Bpatch%5D-tar-issuing-a-sigpipe-in-installpkg-4175637923/ or https://bugs.gentoo.org/573642#c5).

Since `tar` should be able by itself to detect whether the archive is complete, I suggest to disregard the exit code from the `xz` invocation, done in this PR.

Fixes  https://github.com/NixOS/nixpkgs/issues/278130 (script tested here: https://github.com/NixOS/nixpkgs/pull/286579)
Probably also fixes https://github.com/NixOS/nixpkgs/issues/20950 (issue not reproduced here, feedback therefore welcome)
2024-02-13 17:00:48 +01:00
Sergei Trofimovich
74c4e93541 pkgs/stdenv/linux: update x86_64-unknown-linux-gnu bootstrap-files
sha256sum of files to be uploaded:

$ sha256sum /nix/store/dw6vr6m5w7ysrdrbs0s5wdgbjmbnr7gx-stdenv-bootstrap-tools/on-server/*
61096bd3cf073e8556054da3a4f86920cc8eca81036580f0d72eb448619b50cd  /nix/store/dw6vr6m5w7ysrdrbs0s5wdgbjmbnr7gx-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz
87627883b1517087d1c34fadc340bc8deafb3782eb0c60035271822c64dd3401  /nix/store/dw6vr6m5w7ysrdrbs0s5wdgbjmbnr7gx-stdenv-bootstrap-tools/on-server/busybox

Suggested commands to upload files to 'tarballs.nixos.org':

    $ nix-store --realize /nix/store/dw6vr6m5w7ysrdrbs0s5wdgbjmbnr7gx-stdenv-bootstrap-tools
    $ aws s3 cp --recursive --acl public-read /nix/store/dw6vr6m5w7ysrdrbs0s5wdgbjmbnr7gx-stdenv-bootstrap-tools/on-server/ s3://nixpkgs-tarballs/stdenv/x86_64-unknown-linux-gnu/82b583ba2ba2e5706b35dbe23f31362e62be2a9d
    $ aws s3 cp --recursive s3://nixpkgs-tarballs/stdenv/x86_64-unknown-linux-gnu/82b583ba2ba2e5706b35dbe23f31362e62be2a9d ./
    $ sha256sum bootstrap-tools.tar.xz busybox
    $ sha256sum /nix/store/dw6vr6m5w7ysrdrbs0s5wdgbjmbnr7gx-stdenv-bootstrap-tools/on-server/*
2024-02-13 09:10:00 +00:00
Kait Lam
355ab764b4 stdenv: refactor of --replace-{quiet,warn,fail} logic
This is a small simplification of the control flow surrounding these cases. It should make it more obvious when each case happens, and also explicitly defines the current behaviour of --replace.
2024-02-11 10:48:37 +10:00
Rick van Schijndel
2c98eaf597
Merge pull request #286526 from pbsds/fix-replace-quiet-1707148527
stdenv: fix `substituteStream --replace-quiet` deprecation warning
2024-02-07 21:35:19 +01:00
Bernardo Meurer
d73b17633c
Merge pull request #285906 from trofi/update-x86_64-unknown-linux-musl-bootstrap-files
pkgs/stdenv/linux: update x86_64-unknown-linux-musl bootstrap-files
2024-02-05 14:18:03 -05:00
Peder Bergebakken Sundt
cb4c41f93a stdenv: fix substituteStream --replace-quiet deprecation warning 2024-02-05 17:28:02 +01:00
Sergei Trofimovich
5de19bf709 pkgs/stdenv/linux: update x86_64-unknown-linux-musl bootstrap-files
sha256sum of files to be uploaded:

$ sha256sum /nix/store/b0x0qcbf1gsp50jzw52sbbgdp3jlwcjf-stdenv-bootstrap-tools/on-server/*
b745b6311ed4c2d3f260419c468f543ae5df68fe2e5192995c499819c04eb8e0  /nix/store/b0x0qcbf1gsp50jzw52sbbgdp3jlwcjf-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz
f4ae6a814d5053b396af788cd47629385cbfa17df5197ff3c33de62dc1dd9116  /nix/store/b0x0qcbf1gsp50jzw52sbbgdp3jlwcjf-stdenv-bootstrap-tools/on-server/busybox

Suggested commands to upload files to 'tarballs.nixos.org':

    $ nix-store --realize /nix/store/b0x0qcbf1gsp50jzw52sbbgdp3jlwcjf-stdenv-bootstrap-tools
    $ aws s3 cp --recursive --acl public-read /nix/store/b0x0qcbf1gsp50jzw52sbbgdp3jlwcjf-stdenv-bootstrap-tools/on-server/ s3://nixpkgs-tarballs/stdenv/x86_64-unknown-linux-musl/125cefd4cf8f857e5ff1aceaef9230ba578a033d
    $ aws s3 cp --recursive s3://nixpkgs-tarballs/stdenv/x86_64-unknown-linux-musl/125cefd4cf8f857e5ff1aceaef9230ba578a033d ./
    $ sha256sum bootstrap-tools.tar.xz busybox
    $ sha256sum /nix/store/b0x0qcbf1gsp50jzw52sbbgdp3jlwcjf-stdenv-bootstrap-tools/on-server/*
2024-02-02 20:32:44 +00:00
github-actions[bot]
2f68b418f1
Merge staging-next into staging 2024-01-30 06:01:22 +00:00
Bernardo Meurer
f228f378e3
Merge pull request #284542 from trofi/i686-linux-bootstrap-update 2024-01-29 21:21:40 -05:00
Sergei Trofimovich
43695b8db4 pkgs/stdenv/darwin: move bootstrap files definitions to bootstrap-files/ directory
The change moves definition of bootstrap files slightly closer to
`linux` structure to eventually allow those to update in bulk:
https://github.com/NixOS/nixpkgs/issues/253713
2024-01-28 22:41:09 +00:00
Sergei Trofimovich
46dae33dd9 pkgs/stdenv/linux: update i686-unknown-linux-gnu bootstrap-files
sha256sum of files to be uploaded:

$ sha256sum /nix/store/5x6dldhza7if5s6wsicaxa8fbndyixps-stdenv-bootstrap-tools/on-server/*
293021dedf7568932230effb3453a3533e9f5c8f48bbe1fb72e383ade5b3f4df  /nix/store/5x6dldhza7if5s6wsicaxa8fbndyixps-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz
d5bffc3231c9f95699ee34118e26bc788e09c608735e5edb2b433057011fddd1  /nix/store/5x6dldhza7if5s6wsicaxa8fbndyixps-stdenv-bootstrap-tools/on-server/busybox

Suggested commands to upload files to 'tarballs.nixos.org':

    $ nix-store --realize /nix/store/5x6dldhza7if5s6wsicaxa8fbndyixps-stdenv-bootstrap-tools
    $ aws s3 cp --recursive --acl public-read /nix/store/5x6dldhza7if5s6wsicaxa8fbndyixps-stdenv-bootstrap-tools/on-server/ s3://nixpkgs-tarballs/stdenv/i686-unknown-linux-gnu/125cefd4cf8f857e5ff1aceaef9230ba578a033d
    $ aws s3 cp --recursive s3://nixpkgs-tarballs/stdenv/i686-unknown-linux-gnu/125cefd4cf8f857e5ff1aceaef9230ba578a033d ./
    $ sha256sum bootstrap-tools.tar.xz busybox
    $ sha256sum /nix/store/5x6dldhza7if5s6wsicaxa8fbndyixps-stdenv-bootstrap-tools/on-server/*
2024-01-28 14:49:11 +00:00
Artturin
b9e5637ade config.replaceCrossStdenv: add
Example with `clangUseLLVM` which is the default when using `useLLVM`

```nix
config.replaceCrossStdenv = { buildPackages, baseStdenv }:
  if baseStdenv.targetPlatform.useLLVM or false
  then (buildPackages.stdenvAdapters.overrideCC baseStdenv buildPackages.llvmPackages_16.clangUseLLVM)
  else baseStdenv;
```

The conditional necessary, otherwise the other sets(such as `pkgsCross.aarch64-multiplatform.llvmPackages`)
without `useLLVM` will use the stdenv without the necessary conditions to avoid infinite
recursion because of [targetLlvmLibraries](644b234e1c/pkgs/development/compilers/llvm/16/default.nix (L208))
usage.

[`replaceStdenv` is not used when cross-compiling](d77bda728d/pkgs/stdenv/cross/default.nix (L12-L13))

`replaceStdenv` uses an additional stage to replace the stdenv to avoid
infinite recursion and other issues but that should not be necessary for cross.
2024-01-27 19:54:11 +02:00
Weijia Wang
4354538139 Merge branch 'staging-next' into staging 2024-01-27 04:17:36 +01:00
Alyssa Ross
ea67e455d3 stdenvBootstrapTools: update aarch64 musl
The aarch64 musl bootstrap tools are woefully outdated.  Just getting
them to the point of being able to build new versions of themselves
required a number of hacks[1] that can be reverted once we have new
bootstrap tools, and before that it stdenv didn't even build for the
preceding three years.

[1]: https://github.com/NixOS/nixpkgs/pull/169764

So, following the script established by previous bootstrap tools
updates:

Files came from this Hydra build:

	https://hydra.nixos.org/build/246470544

…which used nixpkgs revision dd5621df6d
to instantiate:

	/nix/store/g480ass2vjmakaq03z7k2j95xnxh206a-stdenv-bootstrap-tools.drv

…and then built:

	/nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools

I downloaded these files from Hydra and prefetched them into the Nix
store with the following commands:

	STOREPATH=95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools
	OPTIONS="--option binary-caches https://cache.nixos.org --option trusted-public-keys cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
	nix --extra-experimental-features nix-command store prefetch-file \
	  file://$(nix --extra-experimental-features nix-command store add-file --name bootstrap-tools.tar.xz  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz)
	nix --extra-experimental-features nix-command store prefetch-file --executable \
	  file://$(nix --extra-experimental-features nix-command store add-path --name busybox $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox)

These commands produced the following output:

	warning: you did not specify '--add-root'; the result might be removed by the garbage collector
	Downloaded 'file:///nix/store/fm8ys5bb737j82xgyiciyzpcwmzyi9b2-bootstrap-tools.tar.xz' to '/nix/store/jml0gh0q2rnc9sgr87irz0jfbl0fq545-fm8ys5bb737j82xgyiciyzpcwmzyi9b2-bootstrap-tools.tar.xz' (hash 'sha256-ZY9IMOmx1VOn6uoFDpdJbTnPX59TEkrVCzWNtjQ8/QE=').
	warning: you did not specify '--add-root'; the result might be removed by the garbage collector
	Downloaded 'file:///nix/store/rn9fx55mw2s0qg90zdjxa9xn11nlmjpg-busybox' to '/nix/store/9qylz8gqll63pprwkwlyfs9g4zilak2m-rn9fx55mw2s0qg90zdjxa9xn11nlmjpg-busybox' (hash 'sha256-WuOaun7U5enbOy8SuuCo6G1fbGwsO16jhy/oM8K0lAs=').

I used the hashes from the output above to create the fetchurl
invocation which is part of this commit.

I then started the bootstrap with the following command:

	nix --extra-experimental-features nix-command build -L -f . --arg localSystem '(import ./lib).systems.examples.aarch64-multiplatform-musl' hello

As @lovesegfault requested, here are the the sha256sums of all the
on-server components for extra verification:

	sha256sum /nix/store/${STOREPATH}/on-server/*

…which produced the following output:

	658f4830e9b1d553a7eaea050e97496d39cf5f9f53124ad50b358db6343cfd01  /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools/on-server/bootstrap-tools.tar.xz
	20cdfecb084ddb6b6b958f2b78fd2cc1d9641632f81ec7d5a48fae0a963ad0fa  /nix/store/95lm0y33dayag4542s8bi83s31bw68dr-stdenv-bootstrap-tools/on-server/busybox
2024-01-25 18:13:54 +01:00
github-actions[bot]
7af142cf2c
Merge staging-next into staging 2024-01-22 00:02:41 +00:00
github-actions[bot]
dceddd03df
Merge master into staging-next 2024-01-22 00:02:14 +00:00
Francesco Gazzetta
0d7c62f5c9 pkgs/stdenv/linux: update riscv64 bootstrap-files
This PR updates the bootstrap tarballs for riscv64-linux with new Hydra-generated ones.

Fixes #275848 (bootstrap assembler too old to build gcc 13).

I'll be following the script used in #151399, #168199, #183487, and #188334.

Files came from [this](https://hydra.nixos.org/build/246376732#tabs-summary) Hydra build, which used nixpkgs revision 160cedc144 to instantiate:

```
/nix/store/cpiajh4l83b08pynwiwkpxj53d78pcxr-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu.drv
```

and then built:

```
/nix/store/8a92pj40awdw585mcb9dvm4nyb03k3q3-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu
```

I downloaded these files from Hydra and prefetched them into the nix store with the following commands:

```
STOREPATH=8a92pj40awdw585mcb9dvm4nyb03k3q3-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu
OPTIONS="--option binary-caches https://cache.nixos.org --option trusted-public-keys cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY="
nix store prefetch-file \
  file://$(nix store add-file --name bootstrap-tools.tar.xz  $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/bootstrap-tools.tar.xz)
nix store prefetch-file --executable \
  file://$(nix store add-path --name busybox $(nix-store ${OPTIONS} -r /nix/store/${STOREPATH})/on-server/busybox)
```

These commands produced the following output:

```
Downloaded 'file:///nix/store/xs74wcxq3qc12plfc70fds8inbndhcfm-bootstrap-tools.tar.xz' to '/nix/store/3fal4gikp92013kac6rdmfbrch2s859b-xs74wcxq3qc12plfc70fds8inbndhcfm-bootstrap-tools.tar.xz' (hash 'sha256-0LxRd7fdafQezNJ+N2tuOfm0KEwgfRSts5fhP0e0r0s=').
Downloaded 'file:///nix/store/9ndpna6jrlac4y9fappdjm0sxx0g2bja-busybox' to '/nix/store/kb7wyy30y1gxcmdajljr26kxxac606qa-9ndpna6jrlac4y9fappdjm0sxx0g2bja-busybox' (hash 'sha256-OGO96QUzs2n5pGipn/V87AxzUY9OWKZl417nE8HdZIE=').
```

I used the hashes from the output above to create the `fetchurl` invocation which is part of this commit.

I then started the bootstrap with the following command:

```
nix build -L -f . --arg localSystem '(import ./lib).systems.examples.riscv64' hello
```

As @lovesegfault requested, here are the the `sha256sum`s of all the `on-server` components for extra verification:

```
sha256sum /nix/store/${STOREPATH}/on-server/*
```

which produced the following output:

```
d0bc5177b7dd69f41eccd27e376b6e39f9b4284c207d14adb397e13f47b4af4b  /nix/store/8a92pj40awdw585mcb9dvm4nyb03k3q3-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu/on-server/bootstrap-tools.tar.xz
65f9433abb598f63c932d33351b14f686551512b1cece1e64c2d0e76aa0ec52e  /nix/store/8a92pj40awdw585mcb9dvm4nyb03k3q3-stdenv-bootstrap-tools-riscv64-unknown-linux-gnu/on-server/busybox
```
2024-01-20 16:27:22 +01:00
Robert Scott
40868719b0 cc-wrapper: add zerocallusedregs hardening flag
this uses the value `used-gpr` which seems to be a commonly
chosen value for general use
2024-01-20 13:48:33 +00:00
Peder Bergebakken Sundt
4c1b74bac7 stdenv: substituteStream: escape echoed pattern in --replace mismatch warning 2024-01-14 22:08:03 +01:00
Peder Bergebakken Sundt
e07a2fab7f stdenv: substituteStream: deprecate --replace in favor of --replace-{fail,warn,quiet} 2024-01-14 22:07:58 +01:00
sternenseemann
f734056102 stdenv: fix evaluation if !stdenv.cc.hasCC
stdenv.cc may throw, e.g. in the case of pkgsCross.ghcjs where we must
not force it for the purpose of attribute accessing (`or` doesn't
implicitly tryEval…).

Regression introduced in 1a5bd697ad.
2024-01-13 12:45:36 +01:00
Someone Serge
e6e279913c
stdenvAdapters.useLibsFrom: init 2024-01-12 17:38:00 +00:00
Weijia Wang
2f9e98ccf3 Merge branch 'master' into staging-next 2024-01-10 23:12:44 +01:00
Weijia Wang
df865c667f
Merge pull request #279463 from trofi/stdenv.adapters-fix-overrideLibcxx-cross
stdenv.adapters: fix `overrideLibcxx` to refer to existing `compiler…
2024-01-10 19:49:00 +01:00
K900
bff9ee8862 Merge remote-tracking branch 'origin/master' into staging-next 2024-01-09 13:03:34 +03:00
Ryan Burns
c324705cc3 treewide: simplify exec format conditionals 2024-01-07 17:43:33 -08:00
Sergei Trofimovich
55249c817f stdenv.adapters: fixx overrideLibcxx to refer to existing compiler-rt
Before the change `pkgsLLVM` attributes were failing to pull in
`compiler-rt` on `x86_64-linux`:

    $ nix build --no-link -f. pkgsLLVM.asciidoc-full
    error:
       error: attribute 'llvmPackages_13' missing

       at pkgs/stdenv/adapters.nix:86:32:

           85|       inherit libcxx;
           86|       extraPackages = [ cxxabi pkgs.pkgsTargetTarget."llvmPackages_${lib.versions.major llvmLibcxxVersion}".compiler-rt ];
             |                                ^
           87|     });

It happens because `pkgs.pkgsTargetTarget` are always empty for
cross-packages like `pkgsLLVM.`, `pkgsCross.*.` or
`--arg crossSystem '...'`.
2024-01-07 22:11:49 +00:00
Weijia Wang
eb5334c27d Merge branch 'master' into staging-next 2024-01-03 03:22:40 +01:00
Robert Hensing
af62e8e2fb
Merge pull request #273935 from adisbladis/lib-meta-typechecks-go-brrr
stdenv/check-meta: Use bespoke type checking
2024-01-02 14:40:50 +01:00
Vladimír Čunát
de84adb292
Merge branch 'master' into staging-next 2023-12-29 14:00:30 +01:00
Sergei Trofimovich
3f3e74bc93 stdenv: fix eval of pkgsMusl packages with platform constraints
Before the change `pkgsMusl.adobe-reader` was failing the interpreter:

    $ nix-instantiate --eval --strict --expr 'with import ./. {}; builtins.tryEval pkgsMusl.adobe-reader'
error:
       error: evaluation aborted with the following error message: 'unsupported platform for the pure Linux stdenv'

After the change `pkgsMusl.adobe-reader` returns catchable excation:

    $ nix-instantiate --eval --strict --expr 'with import ./. {}; builtins.tryEval pkgsMusl.adobe-reader'
    { success = false; value = false; }

Noticed when was exploring `nixpkgs` for uncatchable evaluation errors.
Ideally those should only happen when there is a code bug in the use
site. In this case it's just a package with incompatible constraints for
`musl`.

Changed uncatchable `abort` to `throw`.
2023-12-28 15:29:29 +00:00
adisbladis
4a5f2bd6e8 stdenv/check-meta: Use bespoke type checking
Aka `checkMeta` goes brrr.

Using the module system type checking works OK & generates good error messages.
The performance of using it however is terrible because of the value merging it does being very allocation heavy.

By implementing a very minimal type checker we can drastically improve the performance when nixpkgs is evaluated with `checkMeta = true`.
2023-12-26 15:14:42 +13:00
github-actions[bot]
bae9ba4ab0
Merge staging-next into staging 2023-12-14 00:02:44 +00:00
Sergei Trofimovich
fccb48062f darwin/make-bootstrap-tools.nix: set -headerpad_max_install_names for coreutils
Without the change bootstrapTools build fails as:
https://cache.nixos.org/log/g5wyq9xqshan6m3kl21bjn1z88hx48rh-stdenv-bootstrap-tools.drv

    error: install_name_tool: changing install names or rpaths can't be redone for: /nix/store/0hxg356h7lnl2hck23wrdpbib3cckx41-stdenv-bootstrap-tools/bin/tac (for architecture x86_64) because larger updated load commands do not fit (the program must be relinked, and you may need to use -headerpad or -headerpad_max_install_names)
2023-12-13 17:57:25 +00:00
Martin Weinelt
e8dd1cf066 Merge remote-tracking branch 'origin/staging-next' into staging 2023-12-12 14:13:07 +01:00
adisbladis
1e66093cca
Merge pull request #269546 from adisbladis/stdenv-meta-no-intermediate-alloc
stdenv: Avoid allocating intermediate attrset when checking meta validity
2023-12-12 10:57:44 +13:00
adisbladis
6aa414bcfd
Merge pull request #269782 from adisbladis/stdenv-meta-tolist-license-allocations
stdenv: Avoid some list allocations in check-meta when checking licenses
2023-12-12 10:55:58 +13:00
adisbladis
9a0a097a94 stdenv: Avoid allocating intermediate attrset when checking meta validity
This is a small performance optimization. It should be impercetible to most.

Benchmarks:

- Before
``` json
{
  "cpuTime": 0.2777960002422333,
  "envs": {
    "bytes": 3832648,
    "elements": 189513,
    "number": 144784
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 50229344
  },
  "list": {
    "bytes": 655304,
    "concats": 3249,
    "elements": 81913
  },
  "nrAvoided": 218962,
  "nrFunctionCalls": 127718,
  "nrLookups": 40946,
  "nrOpUpdateValuesCopied": 1563978,
  "nrOpUpdates": 8542,
  "nrPrimOpCalls": 113032,
  "nrThunks": 329605,
  "sets": {
    "bytes": 29774864,
    "elements": 1824537,
    "number": 36392
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 235909,
    "number": 24432
  },
  "values": {
    "bytes": 9691392,
    "number": 403808
  }
}
```

- After
```
{
  "cpuTime": 0.2615779936313629,
  "envs": {
    "bytes": 3833832,
    "elements": 189661,
    "number": 144784
  },
  "gc": {
    "heapSize": 402915328,
    "totalBytes": 50212960
  },
  "list": {
    "bytes": 655304,
    "concats": 3249,
    "elements": 81913
  },
  "nrAvoided": 218814,
  "nrFunctionCalls": 127718,
  "nrLookups": 40798,
  "nrOpUpdateValuesCopied": 1563978,
  "nrOpUpdates": 8542,
  "nrPrimOpCalls": 113032,
  "nrThunks": 329457,
  "sets": {
    "bytes": 29765392,
    "elements": 1824093,
    "number": 36244
  },
  "sizes": {
    "Attr": 16,
    "Bindings": 16,
    "Env": 16,
    "Value": 24
  },
  "symbols": {
    "bytes": 235909,
    "number": 24432
  },
  "values": {
    "bytes": 9687840,
    "number": 403660
  }
}
```
2023-12-12 00:08:34 +13:00
adisbladis
3b13bd5c84 stdenv: Avoid some list allocations in check-meta when checking licenses 2023-12-12 00:03:20 +13:00
Robert Scott
dc2247a3b5 stdenvAdapters: add withDefaultHardeningFlags 2023-12-09 16:32:15 +00:00
Robert Scott
1a5bd697ad mkDerivation, bintools-wrapper: move defaultHardeningFlags determination to bintools-wrapper
this makes it a lot easier to create a modified stdenv with a
different set of defaultHardeningFlags and as a bonus allows us
to inject the correct defaultHardeningFlags into toolchain wrapper
scripts, reducing repetition.

while most hardening flags are arguably more of a compiler thing,
it works better to put them in bintools-wrapper because cc-wrapper
can easily refer to bintools but not vice-versa.

mkDerivation can still easily refer to either when it is constructed.

this also switches fortran-hook.sh to use the same defaults for
NIX_HARDENING_ENABLE as for C. previously NIX_HARDENING_ENABLE
defaults were apparently used to avoid passing problematic flags
to a fortran compiler, but this falls apart as soon as mkDerivation
sets its own NIX_HARDENING_ENABLE - cc.hardeningUnsupportedFlags
is a more appropriate mechanism for this as it actively filters
out flags from being used by the wrapper, so switch to using that
instead.

this is still an imperfect mechanism because it doesn't handle a
compiler which has both langFortran *and* langC very well - applying
the superset of the two's hardeningUnsupportedFlags to either
compiler's invocation. however this is nothing new - cc-wrapper
already poorly handles a langFortran+langC compiler, applying two
setup hooks that have contradictory options.
2023-12-09 16:30:45 +00:00
Dmitry Kalinkin
025a278148 Merge remote-tracking branch 'origin/master' into staging
Conflicts:
	pkgs/tools/networking/ofono/default.nix
2023-12-08 12:27:28 -05:00
Martin Weinelt
570af7d909
Merge pull request #263446 from emilylange/stdenv-xz-multithread
stdenv: enable multithreading for `xz` decompression
2023-12-06 15:41:01 +01:00
github-actions[bot]
035a649a67
Merge master into staging-next 2023-12-06 06:01:30 +00:00
Rick van Schijndel
ba75066821
Merge pull request #271362 from pbsds/rm-dead-patches
treewide: remove unreferenced patch files
2023-12-06 03:28:15 +01:00
Vladimír Čunát
305cd929fd
Merge branch 'staging' into staging-next 2023-12-01 07:33:49 +01:00
Peder Bergebakken Sundt
f41aba3739 treewide: remove unreferenced patch files
Found with `fd \\.patch$ pkgs/ -x bash -c 'rg -F "{/}" pkgs/ -q || echo {}'`
2023-12-01 06:11:20 +01:00
Alyssa Ross
0d44b5b773 pkgsStatic.stdenv: fix custom CMake LINKER_LANGUAGE
If a CMake target has a non-default LINKER_LANGUAGE set, CMake will
manually add the libraries it has detected that language's compiler as
linking implicitly.  When it does this, it'll pass -Bstatic and
-Bdynamic options based on the vibes it gets from each such detected
library.  This in itself isn't a problem, because the compiler
toolchain, or our wrapper, or something, seems to be smart enough to
ignore -Bdynamic for those libraries.  But it does create a problem if
the compiler adds extra libraries to the linker command line after
that final -Bdynamic, because those will be linked dynamically.  Since
our compiler is static by default, CMake should reset to -Bstatic
after it's done manually specifying libraries, but CMake didn't
actually know that our compiler is static by default.  The fix for
that is to tell it, like so.

Until recently, this problem was difficult to notice, because it would
result binaries that worked, but that were dynamically linked.  Since
e08ce498f0 ("cc-wrapper: Account for NIX_LDFLAGS and NIX_CFLAGS_LINK
in linkType"), though, -Wl,-dynamic-linker is no longer mistakenly
passed for executables that are supposed to be static, so they end up
created with a /lib interpreter path, and so don't run at all on
NixOS.

This fixes pkgsStatic.graphite2.
2023-11-30 10:46:44 +00:00
github-actions[bot]
e019e7cdb1
Merge staging-next into staging 2023-11-29 12:01:23 +00:00
piegames
cb47f7e108
Merge pull request #267309: check-meta: Improve performance 2023-11-29 12:46:12 +01:00
Artturi
4b27e14f8c
Merge pull request #265738 from Artturin/patchshebangcon
stdenv: run patchShebangs on the configure script when it's a file
2023-11-26 01:28:41 +02:00
Artturin
e68001550d stdenv: run patchShebangs on the configure script when it's a file
if the configure script has a `/usr/bin/env` or some other shebang which
is not in the sandbox then there will be errors such as

`...-stdenv-linux/setup: line 1299: ./configure: cannot execute: required file not found`

There are 250 files which `patchShebangs` `./configure`

https://github.com/search?q=NOT+is%3Afork+lang%3Anix+%2FpatchShebangs+.%5C%2Fconfigure%2F&type=code
2023-11-25 18:16:12 +02:00