Commit Graph

2448 Commits

Author SHA1 Message Date
Vladimír Čunát
39919b8f21
stdenv: use improved strip.sh for aarch64-linux
Adapted from PR #246164
TODO: clean up / use it everywhere on the next rebuild.
2023-07-30 21:18:30 +02:00
Artturin
9816ca55c2 stdenv: Make condition clearer
-z checks for a empty string
-n checks for a not empty string

It makes more sense to run the chmod if the string is not empty
2023-07-28 04:20:26 +03:00
github-actions[bot]
c05c2c2f5c
Merge master into staging-next 2023-07-22 00:02:13 +00:00
Artturin
29b4a92458 stdenvAdapters: Remove 6 year old comment
Comment was added in 3c36dcc and hasn't been needed because makeStaticBinaries already adds glibc.static
2023-07-21 00:49:44 +03:00
Artturin
cb29bc6ace stdenvAdapters: Fix condition
Fixes issue 244232
2023-07-21 00:49:40 +03:00
Uri Baghin
0275cf0e6d
Merge pull request #241025 from uri-canva/fix-native
stdenv: fix stagesNative
2023-07-17 13:18:40 +10:00
Weijia Wang
2b692b1d9d freshBootstrapTools: fix build on darwin 2023-07-15 19:54:50 +02:00
Theodore Ni
bb219736e4
stdenvBootstrapTools: in darwin, only run install_name_tool on Mach-O files 2023-07-14 10:22:55 -07:00
Randy Eckenrode
856ebe6fec
darwin.stdenv: allow patchShebangs during the bootstrap
This fixes pyicu (and any other package that uses `icu-config` instead
of the CMake or some other module to get the build flags).

What happened here is the bootstrap disables `patchShebangs` to avoid
propagating the bootstrap tools to the final stdenv (due to `sh` and
`bash` being on the `PATH` from the bootstrap tools). Because of that,
the `#!/bin/sh` line in `icu-config` was not updated, causing it to
invoke the system bash on Darwin. While that is undesirable in its own
right, when the system bash is invoked as `sh`, `echo -n` will print
`-n`, resulting in the breakage see in https://github.com/NixOS/nixpkgs/pull/241951#issuecomment-1627604354.

The fix is to build bash earlier in the bootstrap while making sure it
is picked up over the one in the bootstrap tools. That allows
`patchShebangs` to be enabled during the bootstrap. Any package with
scripts that is included in the final stdenv should now have its
scripts’ shebang lines properly patched.
2023-07-09 16:53:55 -06:00
github-actions[bot]
c60d48234a
Merge master into staging-next 2023-07-09 12:01:18 +00:00
Randy Eckenrode
6f6687ee24
darwin.stdenv: only run install_name_tool on files 2023-07-08 20:49:50 -06:00
Randy Eckenrode
feb89eed00
darwin.stdenv: fix portable libsystem hook with sandboxing
When sandboxing is enabled, the hook tries to run `install_name_tool`
and fails because the system one is inaccessible. Having it use
`targetPrefix` allows it to find and use the cross-install_name_tool.
2023-07-08 20:49:49 -06:00
Vladimír Čunát
58392652f0
Merge #224822: hardening flags: enable fortify3 by default
...into staging
2023-07-06 10:33:24 +02:00
Weijia Wang
f0a11a54be
Merge pull request #240433 from reckenrode/darwin-stdenv-rework
darwin.stdenv: Darwin stdenv rework
2023-07-05 13:03:24 +03:00
figsoda
34181be024 stdenvAdapters: don't use lib.optional with a list 2023-07-03 21:20:17 -04:00
Artturin
42c94d06fd stdenv: fix overriding with attrset when finalAttrs isn't used
```
nix-repl> (pkgs.htop.overrideAttrs { pname = "hello-overriden"; }).pname
error:
       … while evaluating a branch condition

         at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/lib/customisation.nix:86:7:

           85|     in
           86|       if builtins.isAttrs result then
             |       ^
           87|         result // {

       … while calling the 'isAttrs' builtin

         at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/lib/customisation.nix:86:10:

           85|     in
           86|       if builtins.isAttrs result then
             |          ^
           87|         result // {

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: attempt to call something which is not a function but a set

       at /nix/store/phn5cahwacv9wjgalygw62x8l4xbl6x3-source/pkgs/stdenv/generic/make-derivation.nix:58:21:

           57|           f = self: super:
           58|             let x = f0 super;
             |                     ^
           59|             in
```
2023-07-03 21:34:50 +03:00
Randy Eckenrode
ebc1bcf409
swift-corelibs: don’t link against libcurl
swift-corelibs uses libcurl to implement `NSURLSession` in Foundation
via the symbols exported by CF. Foundation is not build on Darwin, and
these symbols are not exported by the system CoreFoundation.

By not linking against libcurl, this breaks a cycle between CF and
libcurl. That should allow libcurl to drop the patch disabling
linking against the SystemConfiguration and restore NAT64 support.

Unfortunately, the Darwin stdenv bootstrap still needs to build
dependencies that use `fetchFromGitHub`. While it can drop curl from the
final stdenv, it still needs to use it during the stdenv bootstrap.
2023-07-02 17:56:25 -04:00
Randy Eckenrode
a845397040
darwin.stdenv: refactor stdenv definition
In preparation for bumping the LLVM used by Darwin, this change
refactors and reworks the stdenv build process. When it made sense,
existing behaviors were kept to avoid causing any unwanted breakage.
However, there are some differences. The reasoning and differences are
discussed below.

- Improved cycle times - Working on the Darwin stdenv was a tedious
  process because `allowedRequisites` determined what was allowed
  between stages. If you made a mistake, you might have to wait a
  considerable amount of time for the build to fail. Using assertions
  makes many errors fail at evaluation time and makes moving things
  around safer and easier to do.
- Decoupling from bootstrap tools - The stdenv build process builds as
  much as it can in the early stages to remove the requirement that the
  bootstrap tools need bumped in order to bump the stdenv itself. This
  should lower the barrier to updates and make it easier to bump in the
  future. It also allows changes to be made without requiring additional
  tools be added to the bootstrap tools.
- Patterned after the Linux stdenv - I tried to follow the patterns
  established in the Linux stdenv with adaptations made to Darwin’s
  needs. My hope is this makes the Darwin stdenv more approable for
  non-Darwin developers who made need to interact with it. It also
  allowed some of the hacks to be removed.
- Documentation - Comments were added explaining what was happening and
  why things were being done. This is particular important for some
  stages that might not be obvious (such as the sysctl stage).
- Cleanup - Converting the intermediate `allowedRequisites` to
  assertions revealed that many packages were being referenced that no
  longer exist or have been renamed. Removing them reduces clutter and
  should help make the stdenv bootstrap process be more understandable.
2023-07-02 17:56:24 -04:00
uri
f829b24045 stdenv: fix stagesNative 2023-07-02 19:07:23 +10:00
Robert Hensing
1f5a0b0135
Merge pull request #239005 from Artturin/setorfunoverr
stdenv: let overrideAttrs accept attrset OR function
2023-07-01 15:27:23 +02:00
Artturin
0fdae31531 stdenv: let overrideAttrs accept attrset OR function
Makes overrideAttrs usable in the same way that `override` can be used.
It allows the first argument of `overrideAttrs` to be either a function
or an attrset, instead of only a function:

hello.overrideAttrs (old: { postBuild = "echo hello"; })
hello.overrideAttrs { postBuild = "echo hello"; }

Previously only the first example was possible.

Co-authored-by: adisbladis <adisbladis@gmail.com>
Co-authored-by: matthewcroughan <matt@croughan.sh>
2023-06-30 23:28:00 +03:00
github-actions[bot]
ffc0f1526c
Merge master into staging-next 2023-06-26 06:01:14 +00:00
Uri Baghin
6e51b1a92e
Merge pull request #235990 from uri-canva/fix-static
nixStatic: Fix darwin.
2023-06-26 15:22:01 +10:00
Robert Scott
db3e94c3b2 hardening flags: enable fortify3 by default 2023-06-25 11:50:46 +01:00
github-actions[bot]
ea69de970d
Merge master into staging-next 2023-06-23 18:01:07 +00:00
Artturin
d88b362808 stdenv: finish removal of deprecated string/null Flags
forgot to do it before 23.05 but it's ok
2023-06-22 20:37:06 +03:00
github-actions[bot]
fc5ef80184
Merge staging-next into staging 2023-06-22 12:02:08 +00:00
Artturin
ac031478cb stdenv: use lib.isX instead of typeOf Y == X
more readable
2023-06-22 12:31:52 +03:00
Uri Baghin
fd8e2ca0df stdenv: fix makeStaticDarwin not composing with stdenvNoCC 2023-06-22 15:33:54 +10:00
Adam Joseph
b9c1ae2a5f stdenv: eliminate duplicate gnu-config in extraNativeBuildInputs
Accidentally omitted from #237968.
2023-06-16 02:33:57 -07:00
Adam Joseph
3b8e3c1f3d stdenv: updateAutotoolsGnuConfigScriptsHook unconditionally
Unlike autoreconfHook, updateAutotoolsGnuConfigScriptsHook adds
almost no compilations.  Therefore, in the interest of building the
same source code on every platform wherever possible, let's
eliminate the conditional guards around
updateAutotoolsGnuConfigScriptsHook in stdenv.
2023-06-15 19:47:38 -07:00
toonn
b7d4899ad5
Merge pull request #234861 from reckenrode/corefoundation-rpath
swift-corelibs: set NIX_COREFOUNDATION_RPATH in a hook
2023-06-11 17:13:54 +02:00
toonn
782dbaf30f
Merge pull request #234859 from reckenrode/cctools-llvm
cctools-llvm: init at 11.1.0-973.0.1
2023-06-08 19:23:56 +02:00
Randy Eckenrode
df7487ff17
cctools-llvm: init at 11.1.0-973.0.1
cctools-llvm is a replacement for cctools that replaces as much of cctools with equivalents from LLVM that it can reasonably do. This was motivated by wanting to reduce dependencies on cctools, which are updated infrequently by upstream.

To provide a motivating example, the version of `strip` included in cctools cannot properly strip the archives in compiler-rt in LLVM 15. Paths are left to bootstrap tools, resulting in failed requisites checks in the final stdenv build. Since `strip` needs replaced, the opportunity was taken to replace other provided they are functional replacements.

Note: This has to be done in cctools (or some equivalent) because some derivations (noteably LLVM) use the bintools of the stdenv directly instead of going through the wrapper.

The following tools from LLVM are not used in this derivation:

* LLD - not fully compatible with ld64 yet and potentially too big of a change;
* libtool - not a drop-in replacement yet because it does not support linker passthrough, which is needed by xcbuild;
* lipo - crashes when running the LLVM test suite;
* install_name_tool - fails when trying to build swift-corefoundation; and.
* randlib - not completely a drop-in replacement, so leaving it out for now.

If other incompatabilities are found, the tools can be reverted or made conditional. For example, cctools `strip` is preferred on older versions of LLVM (which lack the compiler-rt issue) or when cctools itself is a new enough version because `llvm-strip` on LLVM 11 produces files that older verions of `codesign_allocate` cannot process correctly.

One final caveat/note: Some tools are not duplicated or linked from cctools-port. The names of the tools and which ones were linked was determined based on what is provided upstream in Xcode and is installed on macOS system.
2023-06-06 19:05:22 -04:00
Robert Hensing
dae6b98a90
Merge pull request #234883 from sternenseemann/nix-structured-attrs
stdenv/setup.sh: deal with Nix < 2.4 structured attrs
2023-05-31 19:41:58 +02:00
Randy Eckenrode
7c043234ec
darwin.stdenv: drop NIX_COREFOUNDATION_RPATH from preHook
Rely on swift-corefoundation to set this in its hook. That way, it will
be dropped from the `extraBuildInputs` during a cross build.
2023-05-30 13:28:34 -04:00
sternenseemann
1a29857b8a stdenv/setup.sh: deal with Nix < 2.4 structured attrs
Nix does not (as far it is documented) guarantee that NIX_ATTRS_*_FILE
is set, the only [documented] guarantee seems to be:

> […] made available to the builder via the file .attrs.json in the
> builder’s temporary directory.

This guarantee is of course affected by https://github.com/NixOS/nix/issues/6736,
so it seems to be prudent to fall back to the Nix 2.3 style ATTRS_*_FILE
env vars before defaulting to the expected location in case neither is
available.

See also:

- https://github.com/NixOS/nixpkgs/pull/214937#discussion_r1178101895
- https://github.com/nixos/nixpkgs/commit/afef6588e250

[documented]: https://nixos.org/manual/nix/stable/language/advanced-attributes.html#adv-attr-structuredAttrs
2023-05-29 23:12:24 +02:00
Weijia Wang
91b69f8555
Merge pull request #228714 from helsinki-systems/upd/gnugrep
[staging] gnugrep: 3.7 -> 3.11
2023-05-24 12:47:48 +03:00
github-actions[bot]
2881fdf82f
Merge staging-next into staging 2023-05-20 12:01:51 +00:00
figsoda
98b9e41f61 pkgs: fix typos 2023-05-19 22:31:04 -04:00
Sergei Trofimovich
c3bdae6ce5
Merge pull request #223861 from eliasnaur/no-nix-lib64-in-self-rpath
stdenv: don't set NIX_LIB*_IN_SELF_RPATH by default
2023-05-17 07:08:57 +01:00
Janne Heß
11c8f46850
gnugrep/stdenv: Fix PCRE support by replacing PCRE lib 2023-05-15 10:21:34 +02:00
Robert Hensing
9524f57dd5
Merge pull request #230666 from alyssais/inputDerivation-passAsFile
stdenv: fix inputDerivation with passAsFile
2023-05-09 20:36:29 +02:00
Alyssa Ross
a295c4566e
stdenv: fix inputDerivation with passAsFile
passAsFile passes the values of Nix bindings to the builder as
files, so if those values contained references, they wouldn't end up
in the inputDerivation output.  To fix that, append the contents of
every such passed file to the output.

We only have shell builtins in this derivation, so we can't use cat.
The only way I know of appending the contents of one file to another
using only shell builtins is as I've done here, but it requires
putting the contents of the file on echo's argv.  This might end up
causing problems with large files.  Regardless, I think we should try
this, as a failure is better than silently producing an incorrect
result like the previous behavior.
2023-05-09 14:00:57 +00:00
Alyssa Ross
cd0bccbbe0
stdenv: fix inputDerivation with allowed refs
Same motivation as for the disallowed references.
2023-05-08 12:02:59 +00:00
github-actions[bot]
bf19e21376
Merge master into staging-next 2023-05-04 12:01:18 +00:00
Weijia Wang
e2adc3a3a2 bootstrap-tools-cross: add loongarch64-linux 2023-05-04 09:51:11 +03:00
github-actions[bot]
489a4937eb
Merge master into staging-next 2023-05-03 10:25:04 +00:00
Sergei Trofimovich
ba63ea9268
Merge pull request #229547 from trofi/stdenv-ca-false
stdenv: do not pass `__contentAddressed = false` as environment variable
2023-05-03 11:23:56 +01:00
github-actions[bot]
e7d9c8250f
Merge master into staging-next 2023-05-03 00:02:12 +00:00