mirror of
https://github.com/NixOS/nixpkgs.git
synced 2025-04-14 11:47:45 +00:00
haskell.lib.compose.justStaticExecutables: Forbid references to GHC
This makes `justStaticExecutables` error if the produced store path contains references to GHC. This is almost always erroneous and due to the generated `Paths_*` module being imported. This helps prevent `justStaticExecutables` from producing binaries with closure sizes in the gigabytes. See: https://github.com/NixOS/nixpkgs/issues/164630 Co-authored-by: sternenseemann <sternenseemann@systemli.org>
This commit is contained in:
parent
29eb2c04cf
commit
d2618822ab
@ -923,14 +923,59 @@ for this to work.
|
||||
|
||||
`justStaticExecutables drv`
|
||||
: Only build and install the executables produced by `drv`, removing everything
|
||||
that may refer to other Haskell packages' store paths (like libraries and
|
||||
documentation). This dramatically reduces the closure size of the resulting
|
||||
derivation. Note that the executables are only statically linked against their
|
||||
Haskell dependencies, but will still link dynamically against libc, GMP and
|
||||
other system library dependencies. If dependencies use their Cabal-generated
|
||||
`Paths_*` module, this may not work as well if GHC's dead code elimination
|
||||
is unable to remove the references to the dependency's store path that module
|
||||
contains.
|
||||
that may refer to other Haskell packages' store paths (like libraries and
|
||||
documentation). This dramatically reduces the closure size of the resulting
|
||||
derivation. Note that the executables are only statically linked against their
|
||||
Haskell dependencies, but will still link dynamically against libc, GMP and
|
||||
other system library dependencies.
|
||||
|
||||
If the library being built or its dependencies use their Cabal-generated
|
||||
`Paths_*` module, this may not work as well if GHC's dead code elimination is
|
||||
unable to remove the references to the dependency's store path that module
|
||||
contains. (See [nixpkgs#164630][164630] for more information.)
|
||||
|
||||
Importing the `Paths_*` module may cause builds to fail with this message:
|
||||
|
||||
```
|
||||
error: output '/nix/store/64k8iw0ryz76qpijsnl9v87fb26v28z8-my-haskell-package-1.0.0.0' is not allowed to refer to the following paths:
|
||||
/nix/store/5q5s4a07gaz50h04zpfbda8xjs8wrnhg-ghc-9.6.3
|
||||
```
|
||||
|
||||
If that happens, first disable the check for GHC references and rebuild the
|
||||
derivation:
|
||||
|
||||
```nix
|
||||
pkgs.haskell.lib.overrideCabal
|
||||
(pkgs.haskell.lib.justStaticExecutables my-haskell-package)
|
||||
(drv: {
|
||||
disallowGhcReference = false;
|
||||
})
|
||||
```
|
||||
|
||||
Then use `strings` to determine which libraries are responsible:
|
||||
|
||||
```
|
||||
$ nix-build ...
|
||||
$ strings result/bin/my-haskell-binary | grep /nix/store/
|
||||
...
|
||||
/nix/store/n7ciwdlg8yyxdhbrgd6yc2d8ypnwpmgq-hs-opentelemetry-sdk-0.0.3.6/bin
|
||||
...
|
||||
```
|
||||
|
||||
Finally, use `remove-references-to` to delete those store paths from the produced output:
|
||||
|
||||
```nix
|
||||
pkgs.haskell.lib.overrideCabal
|
||||
(pkgs.haskell.lib.justStaticExecutables my-haskell-package)
|
||||
(drv: {
|
||||
postInstall = ''
|
||||
${drv.postInstall or ""}
|
||||
remove-references-to -t ${pkgs.haskellPackages.hs-opentelemetry-sdk}
|
||||
'';
|
||||
})
|
||||
```
|
||||
|
||||
[164630]: https://github.com/NixOS/nixpkgs/issues/164630
|
||||
|
||||
`enableSeparateBinOutput drv`
|
||||
: Install executables produced by `drv` to a separate `bin` output. This
|
||||
|
@ -23,6 +23,14 @@
|
||||
before changing the package to `pkgs.stalwart-mail` in
|
||||
[`services.stalwart-mail.package`](#opt-services.stalwart-mail.package).
|
||||
|
||||
- `haskell.lib.compose.justStaticExecutables` now disallows references to GHC in the
|
||||
output by default, to alert users to closure size issues caused by
|
||||
[#164630](https://github.com/NixOS/nixpkgs/issues/164630). See ["Packaging
|
||||
Helpers" in the Haskell section of the Nixpkgs
|
||||
manual](https://nixos.org/manual/nixpkgs/unstable/#haskell-packaging-helpers)
|
||||
for information on working around `output '...' is not allowed to refer to
|
||||
the following paths` errors caused by this change.
|
||||
|
||||
## Other Notable Changes {#sec-release-24.11-notable-changes}
|
||||
|
||||
- Create the first release note entry in this section!
|
||||
|
@ -110,6 +110,28 @@ in
|
||||
# build products from that prior build as a starting point for accelerating
|
||||
# this build
|
||||
, previousIntermediates ? null
|
||||
# References to these store paths are forbidden in the produced output.
|
||||
, disallowedRequisites ? []
|
||||
# Whether to allow the produced output to refer to `ghc`.
|
||||
#
|
||||
# This is used by `haskell.lib.justStaticExecutables` to help prevent static
|
||||
# Haskell binaries from having erroneous dependencies on GHC.
|
||||
#
|
||||
# Generated `Paths_*` modules include paths for the runtime library
|
||||
# directory (and similar) of the package being built. If the `Paths_*`
|
||||
# module is imported, this creates a dependency from the static binary
|
||||
# being built to the _library_ being built (which is dynamically linked
|
||||
# and depends on the GHC used to build it).
|
||||
#
|
||||
# To avoid this:
|
||||
# 1. Build the impacted derivation.
|
||||
# 2. Run `strings` on the built binary of the impacted derivation to
|
||||
# locate the store paths it depends on.
|
||||
# 3. Add `remove-references-to -t ${bad-store-path-in-binary}` to the
|
||||
# impacted derivation's `postInstall`.
|
||||
#
|
||||
# See: https://github.com/NixOS/nixpkgs/issues/164630
|
||||
, disallowGhcReference ? false
|
||||
, # Cabal 3.8 which is shipped by default for GHC >= 9.3 always calls
|
||||
# `pkg-config --libs --static` as part of the configure step. This requires
|
||||
# Requires.private dependencies of pkg-config dependencies to be present in
|
||||
@ -680,9 +702,17 @@ stdenv.mkDerivation ({
|
||||
runHook postInstallIntermediates
|
||||
'';
|
||||
|
||||
disallowedRequisites =
|
||||
disallowedRequisites
|
||||
++ (
|
||||
if disallowGhcReference
|
||||
then [ghc]
|
||||
else []
|
||||
);
|
||||
|
||||
passthru = passthru // rec {
|
||||
|
||||
inherit pname version;
|
||||
inherit pname version disallowGhcReference;
|
||||
|
||||
compiler = ghc;
|
||||
|
||||
|
@ -290,7 +290,7 @@ rec {
|
||||
/* link executables statically against haskell libs to reduce
|
||||
closure size
|
||||
*/
|
||||
justStaticExecutables = overrideCabal (drv: {
|
||||
justStaticExecutables = overrideCabal (drv: {
|
||||
enableSharedExecutables = false;
|
||||
enableLibraryProfiling = false;
|
||||
isLibrary = false;
|
||||
@ -300,6 +300,7 @@ rec {
|
||||
# Remove every directory which could have links to other store paths.
|
||||
rm -rf $out/lib $out/nix-support $out/share/doc
|
||||
'';
|
||||
disallowGhcReference = true;
|
||||
});
|
||||
|
||||
/* Build a source distribution tarball instead of using the source files
|
||||
|
Loading…
Reference in New Issue
Block a user