Unfortunately, it won't be possible to build 9.12 nor, in all
likelihood, 9.14 with Cabal 3.12 or 3.14 (which are shipped with GHC 9.8
and 9.10, respectively) due to
<145a647785>.
Since this has been confirmed by upstream, we'll have to bite the bullet
and adjust our hadrian bootstrapping logic to deal with that.
Fortunately, we at least have gotten a hackage released version of Cabal
3.14, so we don't need to bootstrap from an in tree Cabal version.
Backport a patch (which probably addresses a completely unrelated issue
with hadrian and system libffi for GHC >= 9.2.3) that stops Cabal from
handling libffi includes in libraries/ghci. This allows for cc-wrapper
to freely pick the correct libffi which is important when cross
compiling GHC as we need to correctly pick between build and host/target
libffi each time.
This fixes the build of pkgsCross.riscv64.haskell.compiler.ghc8107.
Tested
- haskell.packages.ghc8107.cabal2nix (x86_64-linux, aarch64-linux)
- pkgsCross.riscv64.haskell.compiler.ghc8107 (x86_64-linux)
Could not test darwin because rcodesign is broken at the moment (?).
Unfortunately, this is the newest GHC revision we can update to. After
145a647785,
it becomes necessary to bootstrap with an in tree Cabal version.
Supporting this in nixpkgs is a waste of time since it wouldn't be
necessary for any release version.
Curiously when building a cross compiled riscv64 GHC, the libraries (for
the target) turn out way bigger than when building a riscv64 cross
compiler. Profiling libraries are not necessary for bootstrapping GHC,
so we can disable them for now. We may want to revisit this condition,
though, once we have a native bootstrapping path for riscv64-linux set
up.
In preparation for the deprecation of `stdenv.isX`.
These shorthands are not conducive to cross-compilation because they
hide the platforms.
Darwin might get cross-compilation for which the continued usage of `stdenv.isDarwin` will get in the way
One example of why this is bad and especially affects compiler packages
https://www.github.com/NixOS/nixpkgs/pull/343059
There are too many files to go through manually but a treewide should
get users thinking when they see a `hostPlatform.isX` in a place where it
doesn't make sense.
```
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv.is" "stdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv'.is" "stdenv'.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "clangStdenv.is" "clangStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "gccStdenv.is" "gccStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenvNoCC.is" "stdenvNoCC.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "inherit (stdenv) is" "inherit (stdenv.hostPlatform) is"
fd --type f "\.nix" | xargs sd --fixed-strings "buildStdenv.is" "buildStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "effectiveStdenv.is" "effectiveStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "originalStdenv.is" "originalStdenv.hostPlatform.is"
```
Ideally we don't want to use bintools.bintools and also not really
encode knowledge of what is wrapped and what not in our GHC derivation.
Unfortunately, not all tools are part of the wrapper derivation as well.
This should be gradually improved (e.g. in the case of the darwin tools
and strip).
Some GHC bindists have a normal `$out/lib` directory which contains
symlinks to all core libs. Because it is a normal lib directory, the
bintools setup hook will pick up on it and cause ld to pass the
appropriate -L and -rpath flags. We do not want this to happen,
especially in the case of the stage2 compiler. Not only will the final
ghc have an unnecessary reference (and thus increased closure size) to
the binary ghc, but the extra libraries in the rpath mess with the rts
and cause e.g. segfaults in GHCi.
Unfortunately, there is no way to prevent this. It is a fundamental flaw
in the cc and bintools wrappers that they do not actually distinguish
between the roles of dependencies (build, host, target). Instead
the mangleVar* function will translate the dependencies split up by
roles into platforms. This means that the wrappers can't distinguish
between depsBuildBuild and depsHostTarget (== buildInputs) when natively
compiling. As long as we are natively compiling the wrappers will put
the stage0 ghc (be it in depsBuildBuild, nativeBuildInputs etc.) into
the linker flags of the final ghc.
The solution is to sidestep the issue. We just had ghc in depsBuildBuild
to have it added to PATH. GHC itself will pass the appropriate linker
flags if necessary. To avoid the setup hooks picking up on the GHC
libraries we just don't put it into depsBuildBuild or any other
dependency list. Since the GHC build system accepts the GHC binary via
an absolute path, we don't even need to add the stage0 GHC to PATH.
The stage0 ghc is build->build since it builds the stage1 compiler which
has build for its host platform (i.e. is build->target relative to the
entire GHC derivation).
Also annotate a bit more around the use of pkgsBuildBuild and the boot
compiler and make it more explicit where it comes from in the
derivation.
This is a safeguard against a problem we had with 9.6. Unfortunately,
since the cc wrapper emits `-L` and `-rpath` flags based on platform
config (e.g. aarch64-unknown-linux), not platform role (e.g. build),
stdenv itself doesn't prevent ghc from being linked against the boot
compiler when building a native or cross-compiling GHC (since host ==
build).
With disallowedReferences, the build will fail if such a problem is
re-introduced.
We reuse the targetLibs logic for this since it is more or less the same
story. However, the terminfo library is only built when GHC is neither a
cross-compiler nor being cross-compiled. Therefore ncurses (if used)
will only ever come from pkgsHostTarget. In the other cases ncurses is
still passed via depsBuildBuild for the stage1 compiler.
This commit tries to resolve the problem that the package-db doesn't
include library and include dirs of ncurses for the terminfo package,
causing library loading and linking problems in downstream packages,
e.g. dhall-docs and dhall-toml. This problem was introduced in
4b00fbf163. With this in mind, not passing
--with-curses-* – as long as the terminfo package isn't built – seems
fine.
The Darwin LLVM backend of GHC (which is mostly interesting for
GHC < 9.2) uses clang as configured via the CLANG environment variable
as an assembler. Since it processes outputs of clang as configured via
the CC variable, we need to make sure these versions match or risk CLANG
clang not understanding the output of CC clang.
In the past this wasn't really a problem as due to the fairly old
default clang version in the stdenv, clang 11 would be used for CC.
CLANG would always be a newer version and deal with the output without
any problems.
Ever since the upgrade of the default clang version for
darwin (bcbdb800cf), CC would often be a
newer version of clang than CLANG, causing build problems in some
packages like crypton (for GHC 8.10.7 and 9.0.2 on aarch64-darwin where
the darwin LLVM backend was actually used).
An oversight in 884a76c5e6: We need to
mirror the isGhcjs condition of targetCC (toolsForTarget) for installCC
as well. (Note that in practice, targetCC == installCC for ghcjs since
it is cross-only (?).)
The build platform doesn't matter for checking which stage is our final
stage! Stage2 means host and target are sufficiently similar, Stage1
means host and target may differ.
Hadrian started installing unlit with a targetPrefix (if applicable)
which wasn't the case with make before. Unfortunately, the logic to
generate the settings file wasn't updated, so GHC 9.6.* cross compilers
expect to find an unlit binary without a target prefix.
Upstream issue: https://gitlab.haskell.org/ghc/ghc/-/issues/23317
GHC's build system assumes that the C compiler, tools etc. discovered
during configure can also be used at runtime. This means that the CC,
LD, AR etc. variables given at runtime are used to populate the settings
file which GHC uses to lookup the tools it needs.
The implicit assumption of this mechanism is that the build and runtime
environment of GHC are similar enough and PATH is used to find the
tools. I. e. if we set CC=clang, we wouldn't need to worry about this as
much. We, however, pass absolute paths which is useful since it allows
GHC to work outside of stdenv (as long as e. g. no FFI is involved).
Even so, until now, we didn't really have any problems stemming from
this, as we used pkgsBuildTarget to get everything we need. The
compiler we'd want to execute would in principle need to come
from pkgsHostTarget.
1. For native compilers, all package sets are the same since
build == host == target.
2. For cross compilers build == host, so pkgsBuildTarget
is practically the same as pkgsHostTarget.
When cross-compiling a native compiler, build != host, so we need to
actually ensure that GHC uses different tools at runtime compared to
bootstrapping. There is currently no intended way to achieve this, so we
use a custom tool to edit the settings file. An alternative would be to
patch the build system, but this would be difficult to maintain. We
could go down this route if there's interest from upstream to provide a
proper way to specify the runtime tools.
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
The goal of this commit is basically to eliminate the use of
targetPackages for finding libraries. Instead, we introduce a
`targetLibs` set that can be used instead. The libraries in there
philosophically come from targetPackages since they are used by the core
libs and will be linked against user code. However, when cross compiling
GHC it's always a native compiler, so we can and have to use
pkgsHostTarget (targetPackages would be empty). This is explained more
in the acccompanying comment.
An alternative to this approach is not to pass in the libraries
explicitly via `--with-*` flags and rely on cc-wrapper and splicing to
pick the correct library. This works well for ncurses and probably
merits testing for other libraries as well since it's very simple. It
would need to be verified, however, that configure doesn't discover the
“wrong” library and leaks it somewhere.
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
1. Explicitly set WITH_TERMINFO. We usually match GHC's behavior well,
but it is better to tie the Nix option to make explicitly.
Unfortunately, the same is very complicated to achieve with
hadrian (iirc).
2. Disable enableTerminfo if we are cross-compiling. This matches
the behavior of GHC's build system, so we'll have to match it now.
It also reduces the ncurses-related headache a bit.
3. Stop passing --with-curses* flags. Unfortunately, GHC does not
account for the fact that different platforms need different ncurses
libraries. This is somewhat migitated by the fact that ncurses is
only ever needed for the build platform if we are cross compiling,
but I seem to remember it leaking into the final GHC somehow.
A more reliable alternative is relying on the cc/ld wrapper scripts,
as they'll always pull out the correct ncurses out of the environment
when GHC's build system passes -lcurses.
4. Unconditionally add ncurses to depsBuildBuild. Stage0 unconditionally
builds terminfo (maybe the stage1 compiler needs it?), so we need to
make sure that ncurses for the build platform is available.
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
This is easy in comparison since these tools won't end up in GHC's
settings nor need to be available at runtime, so we can use
the *_FOR_BUILD environment variables.
It is important to add buildCC to depsBuildBuild to engage the
stdenv/wrapper script machinery properly.
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
The bindist defaults to enabling `ar -L` if it detects a compatible `ar`. Suppress this behavior by overriding the setting. This allows the bindist to be used to bootstrap GHC 9.8.
Because the binaries in the bindist lack the linker-signed flag in their signatures, `install_name_tool` will invalidate them when it processes them. Repair the signatures using `rcodesign`, which can also set the linker-signed flag when it replaces the signatures. Having the flag present causes future invocations of `install_name_tool` and `strip` on them to update the signatures.
- Unconditionally get `install_name_tool` from cc.bintools.bintools since it is no longer wrapped; and
- Use the `strip` wrapper on both Darwin architectures. It’s the default one, and it’s the same between both.
This is the libffi version that GHC 8.10.7 was released with.
libffi >= 3.4 in nixpkgs is no longer getting configured with the
configure flag that enabled GHC to continue working.
See https://gitlab.haskell.org/ghc/ghc/-/issues/20051.
Fixes#324384.
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
This refactor should simplify the code a little bit and make future
changes easier. I. e. for cross compiling GHC we'll have to update the
tools in the GHC settings file and calculate the host->target tool paths
for later use. Having a ready function for this will make this a lot
easier.
This is a left over to do from #308776. Rebasing existing PR (prior
to #308776) would need to be rebased on a change before the commit
included in this PR.
elfutils is used in the RTS (rts/Libdw.c), i.e. it will be used on the
target platform.
Tested via pkgsCross.gnu32.haskellPackages.ghc [1], though #304605 needs
to be cherry-picked for elfutils to build.
[1]: nix-shell -E 'with import ./. { crossSystem = "i686-linux"; };
mkShell { nativeBuildInputs = [haskellPackages.ghc ]; }'
