If we don't check `hasCC`, we run into trouble on pkgsCross.ghcjs, one
of the few platforms where `cc` will throw
(so `stdenv.cc.isClang or false would` not be enough).
Problem introduced in 8164b190 / #266172.
This is a follow-up PR to #266172 to address the feedback left by
@sternenseemann regarding `env`.
Rebuilds on staging-next #263535 should be limited if there are any.
Since we can't reliably (or easily at least) tell what Cabal version is
used, we should allow overriding whether to propagate or not, although
GHC >= 9.3 is correct in most cases.
The main idea is to limit the amount of flags passed to Setup.hs as well
as, consequently, linkers and C compilers. E.g. in the case of
gi-javascriptcore, the default behavior causes the argv limit to be
exceeded.
Just getting the length of allPkgconfigDepends now may require a quite
expensive and involved function to be execute, so it should be put off
as long as possible. We can achieve this by moving the assert for
pkg-config being available next to its inclusion in nativeBuildInputs.
This solves the infinite recursion triggered by hercules-ci-cnix-store.
This change essentially amounts to inlining
__CabalEagerPkgConfigWorkaround into haskellPackages.mkDerivation and
applying it automatically for the affected GHC versions. This is a bit
overeager, but the best automatic solution we can come up with for now.
Consequently, we don't need __CabalEagerPkgConfigWorkaround in nixpkgs
anymore nor downstream at least for “standard” haskellPackages builds.
__CabalEagerPkgConfigWorkaround is preserved for now since it is still
necessary if using GHC < 9.4 with Cabal >= 3.10 or cabal-install >= 3.10.
The one thing that may or may not be negatively affected by this change
is ghcWithPackages. I doubt this is a problem in practice though, since
it didn't provide pkg-config in the first place. passthru.env and
shellFor do and work correctly since they rely on mkDerivation.
The main problem was GHC exceeding the Hydra output limit with profiling
libs on aarch64-linux which made us disable the feature. Nowadays the
limit is 3GB, the GHC output is a bit over 2GB, so easily under the
limit.
aarch64-darwin uses a different codegen backend and was never really
affected by the problem: Its output with profiling enabled is around
1.6GB.
Consequently we can enable profiling for all platforms again, as we have
no output size issues for those we build on Hydra.
Thanks to flokli for helping me track down these up to date numbers.
Whether profiling should or should not be enabled does indeed depend on
GHC's target platform (or GHC in general, i.e. if it happens to be
ghcjs, we need to disable it). The profiling objects it produces are
excessively big if it is producing them for aarch64.
However, in a Haskell packages' derivation, GHC's and the resulting
derivation's platforms are offset
GHC: BUILD--HOST---TARGET
DRV: BUILD--HOST--(TARGET)
since we need to execute GHC in the derivation (so its host is our build
platform) and the machine code it produces need to be executable on the
host platform we are targeting. Unless our derivation is building a
compiler (or similar), the target platform of the derivation doesn't
matter.
stdenv exposes the platforms of the current derivation, not of GHC, so
checking the host platform is prudent. Changing this doesn't have a lot
of impact, since when cross-compiling Haskell packages (e.g. via
pkgsCross), host and target platform will usually be the same.
It does save unnecessary rebuilds in the buildHaskellPackages set
though, since most of the packages in here don't care about the target
platform, so they should be the same as their corresponding packages in
the ordinary (natively compiled) haskellPackages set. In short, after
this change
pkgsCross.aarch64-multiplatform.hoogle == haskellPackages.hoogle
holds whereas it would erroneously be compiled with different profiling
settings before.
This line may look odd, but we should not set ghc.isGhcjs if we are
using the JavaScript backend. It is a normal cross backend and no
special code is required to make it work, i.e. everything will be named
as it would be normally. Additionally, passing --ghcjs to Cabal will
make it do the wrong thing.
We need to, of course, stop strip from being thrown at the JS objects in
both cases.
Previously, we would try to calculate the name of
buildPackages.stdenv.cc and then just hope that it is in PATH somehow.
This definitely doesn't work in all cases.
The new approach is to add buildPackages.stdenv.cc to depsBuildBuild
which also populates CC_FOR_BUILD which we can directly re-use.
Currently, the test output is only printed if the test suite fails. If a
test suite gets stuck, however, and is hit with a timeout by Hydra, it
can help to have the log available when diagnosing the issue.
Using $TMPDIR here is problematic because it is not always cleared at
the end of each build, for instance when using "nix-shell --run
genericBuild". This can cause confusing errors when a nix-shell build
is trying to pull in dependencies from a previous build since it tries
to use older package conf files.
To fix, we can just use mktemp which will guarantee us a clean
directory for each build. Should have no effect in nix-build, but will
fix a common issue with using generic-builder in nix-shell.
When cross compiling the pkg-config binary is prefixed and cabal
needs to be made aware of this.
Note: the `--with-pkg-config` flag can't be added unconditionally
because if the package doesn't need pkg-config (thus pkg-config
is not in the PATH) cabal consider this a hard failure.
This is the correctest and clearest way to do it I can think of at the
moment that doesn't need us to add anything.
"${ghcCommand}-${ghc.version}" also works, but is clunkier and harder to
replicate for downstream users.
This adds a new builder option `doHaddockInterfaces` to enable the -haddock flag in GHC,
which results in Haddock comments parsed at compile-time and embedded in
interface files. These are used by the :doc command in GHCi, as well as IDE
tools like ghcide and hls to display docs on hover.
The `-haddock` flag has been around since at least 8.2, even though it does not
get a mention in the GHC Users guide.
There are two downsides to turning on this flag:
1. Increased compile times, since Haddocks must be parsed and then encoded
2. Haddock parse errors now become compile errors for GHC < 9.0.1
(https://gitlab.haskell.org/ghc/ghc/-/issues/8944)
Thus we only enable the feature if we have GHC 9.0.1 and haddock is
enabled; when 9.0.1 becomes the default GHC, we may need to reevaluate
the performance concern.
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
meta.badPlatforms allows us to exclude specific platforms from the list
of supported platforms without the need to explicitly substract it from
lib.platforms.all (or our inferior equivalent allKnownPlatforms) in
platforms. Thus it'll map nicely to unsupported-platforms in the
hackage2nix configuration in the future.
This is a full set rebuild, however it improves the name generation
for the static and cross case since the respective additional
components are now inserted between pname and version instead of after
name like before. This prevents builtins.parseDrvName from mistaking a
platform config string for a version component.
Every flag the generic builder receives via `testFlags` is passed via
`--test-option` [1] to `Setup.hs` which in turn passes them to the
underlying test suite binary. These wrapped options are added to
`checkFlagsArray` in `checkPhase`. This needs to be done in bash since
without structuredAttrs in nixpkgs so far, Nix arrays aren't properly
translated into bash arrays, so we'd have all sorts of quoting issues
when spaces are involved.
Re-using `checkFlags` and `checkFlagsArray` from standard stdenv
setup.sh also results in an additional feature: Using `overrideAttrs`
`checkFlags` and `checkFlagsArray` can additionally be overridden,
which allows passing extra flags to `Setup.hs` whithout being wrapped
with `--test-option`.
[1]: See also https://cabal.readthedocs.io/en/3.4/setup-commands.html?highlight=test-option#cmdoption-runhaskell-Setup.hs-test-test-option
According to the cabal-install man page this also allows passing
special variables which are substituted for other values
depending on context.
continuation of #109595
pkgconfig was aliased in 2018, however, it remained in
all-packages.nix due to its wide usage. This cleans
up the remaining references to pkgs.pkgsconfig and
moves the entry to aliases.nix.
python3Packages.pkgconfig remained unchanged because
it's the canonical name of the upstream package
on pypi.
Comparing to a default value to detect if an argument was provided
forces it to at least WHNF, which can cause failure (e.g., if the
argument is a string with a quoted path from a broken package).
Allow the darwin links code to overwrite libs that were already
copied, because C dependencies can occur multiple times.
Solves errors like
ln: failed to create symbolic link '/nix/store/higpc9xavwcjjzdipz7m9ly03bh7iy2z-hercules-ci-agent-source-0.7.0/lib/links/libboost_context.dylib': File exists
Raise the maximum number of cores used for parallel building from 4 to 16.
Increase the size of the allocation area for GHC's garbage collector from 1 MiB
to 64 MiB.
See https://www.twitch.tv/videos/611899011 for the motivation for this change.
Recent updates to the generic builder have caused haskellPackages.lmdb-simple to
fail to build on Darwin, since it cannot see the lmdb C dynamic library included
by its dependent haskellPackages.lmdb.
The C dynamic library has suffix `.so` not `.dylib`, so this fix allows for
that.
Closes#80190, but that issue may identify a preferable solution.
Previously the package conf files were handled without paying attention
to the fact that it's pretty-printed output. One problem was discovered
with GHC 8.8.1 on Darwin, where the dynamic-library-dirs first field
seems to have increased in length, meaning while before it was
dynamic-library-dirs: some-small-directory-name
some-more-directories
Now it is
dynamic-library-dirs:
some-larger-directory-name
some-more-directories
Which breaks the code installed for https://github.com/NixOS/nixpkgs/pull/25537,
because that assumed the former format, resulting in the reoccurence of
the bug in https://github.com/NixOS/nixpkgs/issues/22810, see
https://github.com/Infinisil/all-hies/issues/43
This commit fixes this by "unprettyfying" the package conf files before
processing them.
Closes https://github.com/NixOS/nixpkgs/pull/78738.
