This reverts commit 9f4b1064c6.
It caused problems: https://github.com/NixOS/nixpkgs/pull/318872#issuecomment-2196168990
FODs are not allowed to have references, but writeText is often used to create a file with references.
Leading to this error:
error: fixed output derivation 'packages.json' is not allowed to refer to other store paths.
Added a cross compilation test for wasm32-unknown-unknown. This also
required using crate.metadata instead of using the regex to get rid of
the hash in the library filename. It also required adding a mkCrate
argument to assertOutputs so we can override the buildRustCrate used.
With this change you will finally be able to use
buildRustCrate/crate2nix to build your wasm32-unknown-unknown
rust projects.
Simply import nixpkgs like so:
```
lib = <nixpkgs/lib>;
pkgsForWasm32 = import <nixpkgs> {
crossSystem = lib.examples.wasm32-unknown-none;
}
```
or use pkgsCross directly with
```
pkgsCross.wasm32-unknown-none.callPackage ./. { };
```
Allow the macOS GLUT framework to be used automatically in many
cases. Packages that specifically search for freeglut or require its
additional APIs should still explicitly depend on it.
Deliberately skip the Haskell package set, which is mostly
automatically generated, and mupdf, which has its own fork of freeglut.
* ocamlPackages.wtf8: use minimalOCamlVersion
* ocamlPackages.ppx_yojson_conv: minimalOCamlVersion
* ocamlPackages.postgresql: use minimalOCamlVersion
* ocamlPackages.opti: use minimalOCamlVersion
* ocamlPackages.opam-repository: use minimalOCamlVersion
* ocamlPackages.opam-format: use minimalOCamlVersion
* ocamlPackages.lwt-dllist: use minimalOCamlVersion
* ocamlPackages.lacaml: use minimalOCamlVersion
* ocamlPackages.gnuplot: use minimalOCamlVersion
* ocamlPackages.fix: use minimalOCamlVersion
* ocamlPackages.eigen: use minimalOCamlVersion
* ocamlPackages.earley: use minimalOCamlVersion
* ocamlPackages.directories: use minimalOCamlVersion
* ocamlPackages.cpuid: use minimalOCamlVersion
* build-support/ocaml: deprecate minimumOCamlVersion
* build-support/ocaml: deprecate minimumOCamlVersion
---------
Co-authored-by: Vincent Laporte <Vincent.Laporte@gmail.com>
In order to allow for the new `cargo::` prefix for build script outputs
we have to adjust the configure-crate bash scripts in buildRustCrate to
properly parse the new additional syntax.
These changes don't affect existing build scripts configured with the
old `cargo:` prefix.
For more information, see https://doc.rust-lang.org/cargo/reference/build-scripts.html#outputs-of-the-build-script
GStreamer searches for plugins relative to its binary's location.
However, since bd97973ce0, it uses its *real* binary location, breaking the FHS.
Fixes#311004 (tested on Heroic and Lutris too).
When cross compiling proc macros, the proc macro needs to be built for the
build platform's architecture.
Without this change cross compiling from Darwin to Linux would simply
fail because it tries to link to a library with a file extension that
doesn't exist on the builder's platform.
Seeing the following new warnings pop up on stderr when cargo was bumped
to 1.78:
```
warning: `/build/.cargo/config` is deprecated in favor of `config.toml`
note: if you need to support cargo 1.38 or earlier, you can symlink `config` to `config.toml`
```
which happens to break commitmsgfmt builds in nix (#320294).
closes#320294
For a long time, we've had `crossLibcStdenv`, `*Cross` libc attributes,
and `*bsdCross` pre-libc package sets. This was always bad because
having "cross" things is "not declarative": the naming doesn't reflect
what packages *need* but rather how we *provide* something. This is
ugly, and creates needless friction between cross and native building.
Now, almost all of these `*Cross` attributes are gone: just these are
kept:
- Glibc's and Musl's are kept, because those packages are widely used
and I didn't want to risk changing the native builds of those at this
time.
- generic `libcCross`, `theadsCross`, and friends, because these relate
to the convolulted GCC bootstrap which still needs to be redone.
The BSD and obscure Linux or freestnanding libcs have conversely all
been made to use a new `stdenvNoLibc`, which is like the old
`crossLibcStdenv` except:
1. It usable for native and cross alike
2. It named according to what it *is* ("a standard environment without
libc but with a C compiler"), rather than some non-compositional
jargon ("the stdenv used for building libc when cross compiling",
yuck).
I should have done this change long ago, but I was stymied because of
"infinite recursions". The problem was that in too many cases we are
overriding `stdenv` to *remove* things we don't need, and this risks
cyles since those more minimal stdenvs are used to build things in the
more maximal stdenvs.
The solution is to pass `stage.nix` `stdenvNoCC`, so we can override to
*build up* rather than *tear down*. For now, the full `stdenv` is also
passed, so I don't need to change the native bootstraps, but I can see
this changing as we make things more uniform and clean those up.
Finally, the BSDs also had to be cleaned up, since they have a few
pre-libc dependencies, demanding a systematic approach. I realized what
rhelmot did in 61202561d9 (specify what
packages just need `stdenvNoLibc`) is definitely the right approach for
this, and adjusted NetBSD and OpenBSD to likewise use it.
We already did this for -march in 12b0e8ac74 ("clang: don't set
-march for overridden target"), but it should have been done for all
machine flags, for the same reason.
Example bug this fixes:
nix-shell -E '
with import ./. {
crossSystem = {
system = "powerpc64le-linux";
gcc.cpu = "power10";
};
};
clangStdenv.mkDerivation { name = "test"; }
' --run '$CC -target wasm32-unknown-unknown -c /dev/null'
Which previously failed with:
clang: error: unsupported option '-mcpu=' for target 'wasm32-unknown-unknown'
This would cause any drv to be rebuilt when any part of the default.nix changes.
What we actually care about is the two JS files though, so simply reference them
directly.
Co-authored-by: Infinidoge <infinidoge@inx.moe>
Switched to a common attribute for library file extensions:
de70971c90
This makes buildRustCrate evaluate successfully when using
pkgsCross/pkgsStatic.
Without the change `updateSourceDateEpoch` fails on tarballs like
`diffoscope-269` that contain single top-level `-269` root as:
diffoscope> source root is -269
diffoscope> find: unknown predicate `-269'
diffoscope> chmod: invalid mode: ‘-269’
diffoscope> Try 'chmod --help' for more information.
Currently `diffoscope-269` has a `sourceRoot = "./-269";` workaround to
bypass the failure.
Co-authored-by: Philip Taron <philip.taron@gmail.com>
Most Linux distributions are enabling this these days and it does
protect against real world vulnerabilities as demonstrated by
CVE-2018-16864 and CVE-2018-16865.
Fix#53753.
Information on llvm version support gleaned from
6609892a2d68e07da3e5092507a730
Information on gcc version support a lot harder to gather,
but both 32bit and 64bit arm do appear to be supported
based on the test suite.
Before, an unstable version of melpaBuild has to be in MELPA format,
which has some disadvantages:
- existing update scripts cannot be reused[1]
- it causes inconsistence.
This patch fixes that by allowing an unstable version to be in Nix
format.
We heuristically detect if it is an unstable version in Nix format.
If so, we convert it to MELPA format.
This does not bring evaluation performance overhead. Here is my
benchmark result using hyperfine-1.18.0 and nix-2.18.1.
Without this patch:
Benchmark 1: NIXPKGS_ALLOW_BROKEN=1 nix eval --include nixpkgs=$PWD --file . emacs.pkgs.melpaPackages --apply 'pkgSet: map (drv: drv.drvPath) (builtins.filter (p: p.type or null == "derivation") (builtins.attrValues pkgSet))' --no-eval-cache >/dev/null
Time (mean ± σ): 9.126 s ± 0.048 s [User: 6.531 s, System: 0.997 s]
Range (min … max): 9.056 s … 9.186 s 10 runs
With this patch applied:
Benchmark 1: NIXPKGS_ALLOW_BROKEN=1 nix eval --include nixpkgs=$PWD --file . emacs.pkgs.melpaPackages --apply 'pkgSet: map (drv: drv.drvPath) (builtins.filter (p: p.type or null == "derivation") (builtins.attrValues pkgSet))' --no-eval-cache >/dev/null
Time (mean ± σ): 9.124 s ± 0.057 s [User: 6.522 s, System: 1.011 s]
Range (min … max): 9.040 s … 9.220 s 10 runs
[1]: 177c9eddad/pkgs/applications/editors/emacs/elisp-packages/manual-packages/lsp-bridge/default.nix (L93-L108)
Store buildFlagsArray as Bash global array variable instead of
creating/sourcing a file containing its declaration.
buildFlagsArray is short enough, and it is not assigned in a sub-shell.
There's no reason to pass it as a file. Storing it as a variable makes
the build more efficient.
Since moreutils depends on Perl and that causes #315990, drop moreutils
until Perl is fixed to handle cross compiling properly. We now create a
temporary file and write the jq result to it and then copy that to
`$out`.
This change refactors internal hooks used by buildDotnetModule to
support derivations with structured attributes. Note that this changes
variable names that the internal hooks expect.
This reverts commit 7173eb87b8.
Reason for revert: This causes too big a rebuild for master (since GHC
uses srcOnly). This went unnoticed due to a stale ofborg rebuild count.
This matches how the default unpackPhase would copy from a store path
into the build directory, so it seems wise to match this here. On file
systems that support reflinks, this should improve performance as well.
Some packages rely on `uname` to configure the host target which breaks cross-compilation.
We can have more control over the evaluation of `uname` by placing `deterministic-uname` into the package's `nativeBuildInputs`.
However the current `deterministic-uname` is hardcoded to `buildPlatform`.
This PR introduces a build argument `forPlatform` to `deterministic-uname` which allows you to override the platform it reports.
Example:
```nix
deterministic-uname.override { forPlatform = stdenv.hostPlatform; }
```
https://github.com/NixOS/nixpkgs/pull/246164 but for hardlinks.
Mesa, among other packages, has binaries that are linked together and
can end up corrupted when the same binary is stripped through two
different names.
To resolve this, print out the device and inode number before each file
name, sort/uniq based on that, then cut it back out before stripping.
The symlink resolution logic is removed as the same file accessed
through two different links in `$paths` will necessarily have the same
numbers. File/directory within the paths listed in `$paths` are
correctly not (and were never) processed due to the `-type f` predicate
and (implied) `-P` option to `find`.
This change fixes cross-compilation for .NET packages (that are not
using .sln as project files). See relevant comment in the change list
for more details.
In addition to that, it removes dotnet-test-sdk that appears to be
broken, that is, dotnet --list-sdks does not recognize SDKs from PATH,
and when propagated from the check hook it was shadowed by inputs from
preceding hooks.
Note that dotnet-test-sdk used to work when it was introduced in PR
144062, but PR 155257 probably overlooked this case. However, currently
it is not used in Nixpkgs and I think dotnetCorePackages.combinePackages
should cover the intended use case for dotnet-test-sdk.
There should be no reason to use env here:
1. In places where it is used to run dotnet with environment variables,
the same can be done with shell syntax.
For example, `env "FOO=$bar" baz` is equivalent to `FOO="$bar" baz`.
2. Otherwise, it just unnecessarily forces PATH lookup for dotnet
command. In addition to that, some dotnet invocations did not use
env.
--frozen is stricter than we need in Nixpkgs. If a Cargo.lock is
slightly wrong, or (in my use case) if building a subproject that is
not a member of the top-level workspace, but the correct Cargo.lock
can be entirely resolved from the existing top-level Cargo.lock, it
should be deterministic, and shouldn't cause any problems, to let
cargo generate the new Cargo.lock. This should result in less need to
bother upstreams about fixing their Cargo.lock files in cases where
they could have been automatically fixed.
Adding cc to propagatedBuildInputs makes derivations with
makeBinaryWrapper in nativeBuildInputs run the cc-wrapper
setup hook. This isn't an issue for derivations using
stdenv, as the cc setup hook is already run by default.
However, derivations that are made with stdenvNoCC, e.g.
because they're made with runCommand, will not run the
cc-wrapper setup hook without this change.
For some reason the FreeBSD native compiler will not work
correctly without the setup hook.
The $DISPLAY variable has a format of [host]:num[.screen]. Previously,
the number would only be extracted properly if it had the form :num.
Allow all forms but correctly discard the unused parts.
Thanks to Qyriad, it was found out that Nixpkgs was unable to cross
compile when CMake was needed at cross compilation time with Meson
because Meson refuse to use the ambiant CMake.
A simple fix is to always provide the ambiant CMake and trust the
packager to pass the right CMake, otherwise the rest of the build would
probably fail in mysterious ways.
An example of package that required this fix is the Lix, a Nix
implementation, that uses `toml11` discovered via CMake during the Meson
configure phase.
Co-authored-by: Qyriad <qyriad@qyriad.me>
Signed-off-by: Raito Bezarius <masterancpp@gmail.com>
This fixes at least one bug with default-features, and also
just aligns us more with what Cargo actually does.
Also some Python style fixes and a bit less mutating state.
Conflicts in luaPackages.luarocks-nix:
- 8b563cd9f9 switched to the new version format (prefixed by 0 since there was no relevant last tag in the branch).
bc4f6fa543 bumped version and switched to the new version format. But the tag used is not part of the branch the commit comes from (master).
Used the new version but replaced the tag prefix with 0.
- b73ec84b9e removed meta (it is already set in the overridden luarocks). But that would cause the update script to try to update the overridden luarocks (as did the previously used old.meta).
bf311d950e modified meta to fix the update script properly.
Kept the meta from the latter since it allows update script to work.
This fixes:
Could not load ICU data. UErrorCode: 2
We're using a hook instead of a wrapper because various things like to
reference the unwrapped dotnet executable.
The underlying goModules derivation must have the same source root as
the overall package. sourceRoot is being propagated already, but
setSourceRoot was not. This was surely an oversight.
This is implemented in such a way that it cannot break existing
vendorHashes unless the package is using setSourceRoot already. If
that's the case, then the package is either broken or found a workaround
to do deal with this very issue (e.g. by using `cd path` in an early
phase).
A simple search (`rg setSourceRoot $(rg -l buildGoModule)`) reveals that
no such packages obviously exist in Nixpkgs.
This was achieved using the following command:
sd 'wrapGAppsHook\b' wrapGAppsHook3 (rg -l 'wrapGAppsHook\b')
And then manually reverted the following changes:
- alias in top-level.nix
- function name in wrap-gapps-hook.sh
- comment in postFixup of at-spi2-core
- comment in gtk4
- comment in preFixup of 1password-gui/linux.nix
- comment in postFixup of qgis/unwrapped-ltr.nix and qgis/unwrapped.nix
- comment in postFixup of telegram-desktop
- comment in postFixup of fwupd
- buildCommand of mongodb-compass
- postFixup of xflux-gui
- comment in a patch in kdePackages.kde-gtk-config and plasma5Packages.kde-gtk-config
- description of programs.sway.wrapperFeatures.gtk NixOS option (manual rebuild)
Closes#267442
$ nix path-info -Sh /nix/store/qj1dm7wfw5m3mxf1gn3fdm0az9y1h5ny-linux-firmware-20240312-xz
/nix/store/qj1dm7wfw5m3mxf1gn3fdm0az9y1h5ny-linux-firmware-20240312-xz 440.3M
$ nix path-info -Sh /nix/store/c3szcjxb3g990dbiz7llwmkaf0bi98j2-linux-firmware-20240312-zstd
/nix/store/c3szcjxb3g990dbiz7llwmkaf0bi98j2-linux-firmware-20240312-zstd 460.6M
This is an increase of 4.4%, but OTOH zstd has a significantly higher
decompression speed[1].
[1] https://gregoryszorc.com/blog/2017/03/07/better-compression-with-zstandard/
Closes#302291Closes#301536
The following things have changed:
* For 5.7+: ZSWAP compressor uses zstd now.
* For 5.11+: ZRAM compressor uses zstd now.
* For 5.13+: kernel modules are compressed with zstd instead of xz.
* For 5.19+: support zstd-compressed firmware.
The modules-closure functionality needed explicit support for copying
over `.zst` files. Also, the VM image builder used busybox's `insmod`
before which doesn't support zstd. Switched to `kmod` and added xz/zstd
as dependencies for it, similar to how it's done for the actual stage1
in d33e52b253. The use of `kmod` here
doesn't seem to be such a big deal since it's only a build-time
dependency.
Since the tool is exposed more prominently now, we should clear up what
it is and note that it is to be considered unstable, i.e. we may change
it if the necessity arises. (In practice it is probably going to be
fairly stable though, as compiler interfaces tend to be quite stable.)
Should we add a version?
The cc and bintools wrapper contained ad hoc bootstrapping logic for
expand-response-params (which was callPackage-ed in a let binding). This
lead to the strange situation that the bootstrapping logic related to
expand-response-params is split between the wrapper derivations (where
it is duplicated) and the actual stdenv bootstrapping.
To clean this up, the wrappers simply should take expand-response-params
as an ordinary input: They need an adjacent expand-response-params (i.e.
one that runs on their host platform), but don't care about the how.
Providing this is only problematic during stdenv bootstrapping where we
have to pull it from the previous stage at times.
We don't need to artificially make sure that we can execute the wrapper
scripts on the build platform by using stdenv's shell (which comes from
buildPackages) since our cross infrastructure will get us the wrapper
from buildPackages. The upside of this change is that cross-compiled
wrappers (e.g. pkgsCross.aarch64-multiplatform.gcc) will actually work
when executed!
For bootstrapping this is also not a problem, since we have a long
build->build platform chain so runtimeShell is just as good as
stdenvNoCC.shell. We do fall back to old ways, though, by explicitly
using the bootstrap-tools shell in stage2, so the adjacent bash is only
used from stage4 onwards. This is unnecessary in principle (I'll try
removing this hack in the future), but ensures this change causes zero
rebuilds.
To reduce dependencies (mainly nix-prefetch-git and through that git,
git-lfs) when we just need to fixup a lock file, eg when building electron.
This also tries to avoid needless rebuilds when eg. golang is updated.
Also this cleans up and combined the build/installPhase of both tools to
be a lot simpler.
In delicate code like this, it seems unwise to pass something of as
something it isn't for convenience's (?) sake. It causes a slight
possibility for confusion with `buildPackages.stdenv`. However, it
should be possible to eliminate the need for this in a separate change.
`stdenv.is*` defaults to `hostPlatform` which is technically incorrect
here, since what we are gating concerns the target platform only.
This would be a problem in practice if cross compiling from
x86_64-darwin to aarch64-darwin whilst using gcc which is probably
rare enough.
Replace `src = ./.` instances with more explicit source listings.
Otherwise, derivations will be rebuilt on any change to the files
defining them (e.g. formatting via nixfmt-rfc-style).
After enabling a separate binary output for the `Agda` Haskell package,
the new `bin` output measures ~100MiB, compared to the ~4.5GiB before.
Using it in `agdaWithPackages` reduces the closure size of an Agda
installation from ~5GiB to ~3GiB. The remaining space is taken up
mostly by the GHC backend.
With this change, derivations depending on `haskellPackages.Agda`
directly need to pick the right (binary) output. This concerns in
particular `emacsPackages.agda2-mode`.
`lib.getExe pkgsCross.riscv64.buildPackages.pkg-config` should return
`<prefix>/bin/riscv64-unknown-linux-gnu-pkg-config` not `<prefix>/bin/pkg-config`
