cp on macOS doesn't support the -T flag, which causes the fetch-deps
script to fail. Use Nix's coreutils to ensure the script works
consistently across all platforms.
cp on macOS doesn't support the -T flag, which causes the fetch-deps
script to fail. Appending `/.` to the source argument replicates the
same functionality.
Fixes#186752. This adds buildVMMemorySize (defaults to 512 MiB) to
buildImage, which is passed to vm.runInLinuxVM. This is needed for
larger base images, which may otherwise cause container build failures
due to OOM in the VM.
Tell rust if we want our binaries linked statically or dynamically.
Otherwise the compiler will always produce statically linked binaries for musl
targets, as this is the default.
One significant use case is adding `passthru.tests` to setup-hooks,
and help increase test coverage for mission-critical setup-hooks.
As `meta`, `passthru` doesn't go into the build script directly.
However, passing an empty set to `passthru` breaks nixpkgs-review
and OfBorg tests, so pass it only when specified.
Some packages are defined by the build proccess, and change every time
the dotnet-sdk package changes. To avoid having to regenerate every
dependant packages dependencies every dotnet update, this moves these
packages into the `dotnet-sdk` `passthru` attribute, and includes them
every time `buildDotnetModule` is used.
Before the change separate-debug-info.sh did the stripping itself.
This scheme has a few problems:
1. Stripping happens only on ELF files. *.a and *.o files are skipped.
Derivations have to do it manually. Usually incorrectly
as they don't run $RANLIB (true for `glibc` and `musl`).
2. Stripping happens on all paths. Ideally only `stripDebugList` paths
should be considered.
3. Host strip is called on Target files.
This change offloads stripping logic to strip hook. This strips more
files for `glibc` and `musl`. Now we can remove most $STRIP calls
from individual derivations.
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
The initial intent was to strip .a and .o files, not .a.o files.
While at it expanded stripping for $lib output as well.
Without the change `libgcc.a` was not stripped and `.debug*` sections
made into final binaries. It's not a problem on it's own, but it's an
unintended side-effect. Noticed on `crystal_1_0` test failure where
`crystal` was not able to handle `dwarf-5`.
While at it allowed absolute file names to be passed to stripDebugList
and friends.
The --self-contained and --no-self-contained switches were
added to the dotnet build command starting with .NET 6.
The switch is equivalent to the setting the SelfContained
property, so we use the property for backwards compatibility.
Now the tool will only strip binaries if a strip executable is passed
via the STRIP environment variable. This is exposed via the strip
option for makeInitrdNG and the NixOS option boot.initrd.systemd.strip.
We are replicating one mechanism behind `-Z build-std`.
There isn't yet crate2nix support for this, but one can (and I do) add
the missing stdlib deps (for this feature to pick up) with overrides.
POSIX sh (and `bash`) impose a restriction on environment variable name
format and disallow hypheps in the names. Normally it's not a problem
as nothing usually tries to refer nyphenated names.
One exception is `nix develop` (https://github.com/NixOS/nix/issues/6848):
$ nix develop -f. gcc -L
gcc-wrapper> ...-get-env.sh: line 70: expand-response-params: bad substitution
Note that bash usually uses explicitly created `expandResponseParams`
variant of the same variable.
To work the problem around let's avoid environment variable export and move
it to `passthru` for `cc` (used ina few places) and remove it completely for
`binutils` (does not seem to be used at all).
A full check would be more complicated to write -
and more importantly - probably also more expensive.
Motivation: eval-time catch for errors like in commit 8198636be0.
'strip' does not normally preserve archive index in .a files.
This usually causes linking failures against static libs like:
$ nix build --no-link -f. pkgsCross.mingw32.re2c
> ...-i686-w64-mingw32-binutils-2.38/bin/i686-w64-mingw32-ld:
/nix/store/...-i686-w64-mingw32-stage-final-gcc-13.0.0-lib/i686-w64-mingw32/lib/libstdc++.dll.a:
error adding symbols: archive has no index; run ranlib to add one
We restore the index by running ranlib explicitly.
This change mimics existing strip{All,Debug}List variables to
allow special stripping directories just for Target.
The primary use case in mind is gcc where package has to install
both host and target ELFs. They have to be stripped by their own
strip tools accordingly.
Co-authored-by: Rick van Schijndel <Mindavi@users.noreply.github.com>
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
In some cases `$pkgs_src` can be a path. For example with `FSharp.Core` when it comes with dotnet SDK.
In these cases we need to fallback on default URL otherwise curl fails.
Without this change cross-built gcc fails to detect stack protector style:
$ nix log -f pkgs/stdenv/linux/make-bootstrap-tools-cross.nix powerpc64le.bootGCC | fgrep __stack_chk_fail
checking __stack_chk_fail in target C library... no
checking __stack_chk_fail in target C library... no
It happens because gcc treats search paths differently:
https://gcc.gnu.org/git/?p=gcc.git;a=blob;f=gcc/configure.ac;h=446747311a6aec3c810ad6aa4190f7bd383b94f7;hb=HEAD#l2458
if test x$host != x$target || test "x$TARGET_SYSTEM_ROOT" != x ||
test x$build != x$host || test "x$with_build_sysroot" != x; then
...
if test "x$with_build_sysroot" != "x"; then
target_header_dir="${with_build_sysroot}${native_system_header_dir}"
elif test "x$with_sysroot" = x; then
target_header_dir="${test_exec_prefix}/${target_noncanonical}/sys-include"
elif test "x$with_sysroot" = xyes; then
target_header_dir="${test_exec_prefix}/${target_noncanonical}/sys-root${native_system_header_dir}"
else
target_header_dir="${with_sysroot}${native_system_header_dir}"
fi
else
target_header_dir=${native_system_header_dir}
fi
By passing --with-build-sysroot=/ we trick cross-case to use
`target_header_dir="${with_sysroot}${native_system_header_dir}"`
which makes it equivalent to non-cross
`target_header_dir="${with_build_sysroot}${native_system_header_dir}"`
Tested the following setups:
- cross-compiler without libc headers (powerpc64le-static)
- cross-compiler with libc headers (powerpc64le-debug)
- cross-build compiler with libc headers (powerpc64le bootstrapTools)
Before the change only 2 of 3 compilers detected libc headers.
After the change all 3 compilers detected libc headers.
For darwin we silently ignore '-syslibroot //' argument as it does not
introduce impurities.
While at it dropped mingw special case for no-libc build. Before the change
we passed both '--without-headers --with-native-system-headers-dir' for
no-libc gcc-static builds. This tricked darwin builds to find sys/sdt.h
and fail inhibid_libc builds. Now all targets avoid passing native headers
for gcc-static builds.
While at it fixed correct headers passing to
--with-native-system-headers-dir= in host != target case: we were passing
host's headers where intention was to pass target's headers.
Noticed the mismatch as a build failure on pkgsCross.powernv.stdenv.cc
on darwin where `sys/sdt.h` is present in host's headers (libSystem)
but not target's headers (`glibc`).
Co-authored-by: Adam Joseph <54836058+amjoseph-nixpkgs@users.noreply.github.com>
Since 1ac53985 "*-wrapper; Switch from `infixSalt` to `suffixSalt`"
(2020) 'TARGET_' prefix (and infix) is no more. '_FOR_TARGET' suffix
is the only used suffix for target-specific tools and flags.
Use that in stip instead of always-empty variable.
this shouldn't change any binary available in the default build environment
because bintools-unwrapped is already in path ( idk where it comes from but i know because objcopy is in path but not in the wrapper )
this just makes all the binaries available under 'bintools' instead of
having to use 'bintools-unwrapped'
reduces confusion because now 'objcopy' and others will be in 'bintools'
A function to generate pkg-config files for Nix packages that need to create them ad hoc,
like blas and lapack.
Inspiration taken from `makeDesktopItem`.
Currently when cross compiling the `buildPackages.libredirect` has the wrong dynamic library extension.
To reproduce the issue run something like:
```
file $(nix-build -A pkgsCross.mingwW64.buildPackages.libredirect)/lib/libredirect.dll
/nix/store/80llmqa9lkabg3qnmglngzz22fwf739q-libredirect-0/lib/libredirect.dll: Mach-O 64-bit dynamically linked shared library x86_64
```
or
```
nix-diff $(nix-instantiate -A libredirect) $(nix-instantiate -A pkgsCross.mingwW64.buildPackages.libredirect)
```
By default, Cargo will only enable line tables. -g enables full debug
info. The RUSTFLAGS environment variable is examined by Cargo,
similar to how the NIX_*FLAGS* variables are examined by our compiler
wrappers.
Before this change `srcOnly git` gives:
duplicate derivation output 'debug', at pkgs/stdenv/generic/make-derivation.nix:270:7
This was because separateDebugInfo = true was passed on to the srcOnly
mkDerivation as well as the outputs list _including_ the debug output.
Luckily we don't need to untangle this mess since srcOnly is only
supposed to have a single output anyways.
Transform exit handlers of the form
trap cleanup EXIT [INT] [TERM] [QUIT] [HUP] [ERR]
(where cleanup is idempotent)
to
trap cleanup EXIT
This fixes a common bash antipattern.
Each of the above signals causes the script to exit. For each signal,
bash first handles the signal by running `cleanup` and then runs
`cleanup` again when handling EXIT.
(Exception: `vscode/*` prevents the second run of `cleanup` by removing
the trap in cleanup`).
Simplify the cleanup logic by just trapping exit, which is always run
when the script exits due to any of the above signals.
Note: In case of borgbackup, the exit handler is not idempotent, but just
trapping EXIT guarantees that it's only run once.
Some haskell code starts silently hanging when not built with a
threaded runtime, so let’s assume people using `writeHaskell` don’t
care about micro-optimizations like this and do the expected thing.
Some architectures don’t support a threaded runtime, for these we
provide the `threadedRuntime` option to turn it off (it should fail at
build time in that case, easy to detect).
If somebody already passed `"-threaded"` before via ghcArgs, this
will not add the flag a second time. Thus it’s backward-compatible in
this regard.
I tested out both branches (with `-threaded` set and not set before),
on an example I had where the runtime would hang when not compiled
with `-threaded`.
Sometimes I want to pass a different implementation of `mkNugetDeps`.
For example in private repos, it can be handy to use `__noChroot = true`
and bypass the deps.nix generation altogether. Or some Nuget packages
ship with ELF binaries that need to be patched, and that's best done as
soon as possible.
If the package was not restored from nuget.org (determinted by checking
the "source" field of ".nupkg.metadata"), query the custom source for
the package endpoint (the way nuget api is built we can't determine it
without an API query) and build a custom package URL to save in the
generated deps file.
Noticed this bug when was trying to bootstrap m4 on darwin. That fixes
line 163: no such file or directory error
That does not solve all problems staging has on darwin.
With Rust 1.61, it is necessary to link to external static/dynamic libaries
when building the rlib that uses them, rather than when linking the final
binary. In fact, it is no longer necessary to specify the libraries to link
when building the final binary, but the library search path flags must still
be included.
The fetchgit function in nixpkgs sets the leaveDotGit argument to true
if deepClone is set to true. nix-prefetch-git did behave differently. It
would not assume --leave-dotGit if --deepClone is specified. With this
change the inconsistency is addressed by assuming --leave-dotGit if
--deepClone is specified.
When maintainers override stages of `fetchgit' (e.g. `postPatch`) it
is very easy for them to accidentally leak the outpath-hash of their
current `stdenv` into `fetchgit''s output, and therefore into the
value they paste into `sha256`.
This is a problem, because the resulting expression will break
whenever any change is made to `stdenv` or when anybody attempts to
build the expression on a different platform than the one used by the
original maintainer.
Almost as much of a problem is the fact that CI **does not catch**
these problems. The `fetchgit` is run only once, then its output goes
into cachix, and all future builds (hydra, CI, ofborg) pull from
cachix.
Let's offer maintainers the option to check that they aren't making
this mistake, by passing through `allowedRequisites`. The default
value is `null`, but it might be worth changing that at some point in
the future.
It is also sometimes difficult to communicate to package maintainers
why their expression is problematic. Having `allowedRequisites`
passed through makes it easier to do this: "look, when I switch on
`allowedRequisites` your package breaks; are you sure you meant to
hardcode the hash today's `x86_64-linux.stdenv` into your expression?`
For an example use case, see https://github.com/NixOS/nixpkgs/pull/171223
The issue above is part of a larger problem with nixpkgs infra: there
large parts of cachix cannot be reproduced easily if they are lost.
Once something ends goes into cachix, we never ever again reverify the
procedure by which it was placed into cachix.
musl and darwin support UTF-8 locales without any extras. As a result
unzip can unpack UTF-8 filenames there as is. But on glibc without
locale archive presence files get mangled as:
deps/αβ -> deps/#U03b1#U03b2
This makes `fetchzip` fixed-output derivations unstable.
Tested this change to fail in `coq.src` which was generated in system
that mangles UTF-8 symbols:
$ nix build -f. coq.src --rebuild -L
source> trying https://github.com/coq/coq/archive/V8.15.2.zip
source> % Total % Received % Xferd Average Speed Time Time Time Current
source> Dload Upload Total Spent Left Speed
source> 0 0 0 0 0 0 0 0 --:--:-- 0:00:01 --:--:-- 0
source> 100 8945k 100 8945k 0 0 1513k 0 0:00:05 0:00:05 --:--:-- 1989k
source> unpacking source archive /build/V8.15.2.zip
error: hash mismatch in fixed-output derivation '/nix/store/hrnyykm7wgw8vxisgq7hc2bg5gr0y6s8-source.drv':
specified: sha256-h81nFqkuvZkMR7YLHy7laTq5yOhjMW+w6rYzncxvyD4=
got: sha256-DTspmwyD3Evl1CUmvUy2MonbLGUezvsHN3prmP9eK2I=
Note: it means that some of existing caches for fixed output
derivations become incorrect. It should not break already cached
tarballs on cache.nixos.org thus the impact should not be widespread.
This enables users to make use of clang's multi-platform/target support
without having to go through full cross system setup. This is especially useful
for generating bpf object files, I'm not even usre what would a no-userland
cross compile system tuple even look like to even try going that route.
Fixes#176128
Otherwise, these warnings are emitted:
command-line option '-Wformat=1' is valid for C/C++/ObjC/ObjC++ but not for Fortran
command-line option '-Wformat-security' is valid for C/C++/ObjC/ObjC++ but not for Fortran
'-Werror=' argument '-Werror=format-security' is not valid for Fortran
Fixes part of #27218
- put `findlib` in `buildInputs` of `mkCoqDerivation` to make sure `coq` packages find their ocaml plugin dependencies,
- use `propagatedBuildInputs` to make sure ocaml plugin dependencies are in path,
- updated `coqPackage.heq` (broken url),
- fixed use of `DESTDIR` and `COQMF_COQLIB` in mkCoqDerivation,
- adding `COQCORELIB` environement variable to put ocaml plugin files in the right place,
- make `metaFetch` available from `coqPackages`
a67950f20b added `url` attribute
from `fetchurl` and therefore also from `fetchzip`.
We previously relied on `url` from fetchgit-based fetchers
to find the repo URL but now it will just return tarballs
in the case of `fetchFrom{GitHub,GitLab}`.
Let’s add an attribute to `fetch{git,FromGitHub,FromGitLab}`
to expose a repo URL consistently.
* origin/staging-next: (62 commits)
Re-Revert "lua: fix on darwin by using makeBinaryWrapper (#172749)"
openldap: fix cross-compilation
makeBinaryWrapper: fix codesign on aarch64-darwin
python3Packages.ldap: fix linking with openldap 2.5+
Revert "lua: fix on darwin by using makeBinaryWrapper (#172749)"
wine: enable parallel build again
pkgsi686Linux.gdb: fix formatting for 32-bit systems
gtk4: Fix incorrect merge
nixos/openldap: use upstream unit defaults
openldap: update maintainers
openldap: 2.4.58 -> 2.6.2
Revert "Add mingwW64-llvm cross-system."
lua: fix on darwin by using makeBinaryWrapper (#172749)
python310Packages.python-mimeparse: execute tests
pandas: fix darwin build
gtk3: 3.24.33 -> 3.24.33-2022-03-11
gtk4: patch fixing g-c-c crashes
e2fsprogs: patch for CVE-2022-1304
firefox-unwrapped: fix cross compilation
rustc: expose correct llvmPackages for cross compile
...
This should be a significant disk space saving for most NixOS
installations. This method is a bit more complicated than doing it in
the postInstall for the firmware derivations, but this way it's
automatic, so each firmware package doesn't have to separately
implement its compression.
Currently, only xz compression is supported, but it's likely that
future versions of Linux will additionally support zstd, so I've
written the code in such a way that it would be very easy to implement
zstd compression for those kernels when they arrive, falling back to
xz for older (current) kernels.
I chose the highest possible level of compression (xz -9) because even
at this level, decompression time is negligible. Here's how long it took
to decompress every firmware file my laptop uses:
i915/kbl_dmc_ver1_04.bin 2ms
regulatory.db 4ms
regulatory.db.p7s 3ms
iwlwifi-7265D-29.ucode 62ms
9d71-GOOGLE-EVEMAX-0-tplg.bin 22ms
intel/dsp_fw_kbl.bin 65ms
dsp_lib_dsm_core_spt_release.bin 6ms
intel/ibt-hw-37.8.10-fw-22.50.19.14.f.bseq 7ms
And since booting NixOS is a parallel process, it's unlikely (but
difficult to measure) that the time to user interaction was held up at
all by most of these.
Fixes (partially?) #148197
The old logic flow had the structure
if ( … ) {
if ( … ) {
…
} else {
…
}
} else {
…
}
which is quite hard to follow in Nix. Instead we ensure that no if
expression is inside a then branch.
This change is zero rebuild, as no logic was changed.
- use propagatedBuildInputs to make sure ocaml plugin stuff is in path
- updated coqPackage.heq (broken url)
- fixed use of `DESTDIR` and `COQMF_COQLIB` in mkCoqDerivation
- adding `COQCORELIB` environement variable to put ocaml plugin files in the right place
- make metaFetch available from `coqPackages`
A small shell script that can be used to extract a binary wrapper's
makeCWrapper call from its embedded docstring, without depending on
makeBinaryWrapper.
