rustdoc is built for native builds, because it's used to run doctests,
but it wasn't built for cross builds, since they don't run doctests.
This inconsistency led to wrapRustc not working for cross-compiled
rustc, because it expected there to be a rustdoc binary.
In the interests of consistency, let's instruct the build system to
always build rustdoc.
Link: https://github.com/NixOS/nixpkgs/pull/292777#issuecomment-2048071969
Fixes: 6f8fa05acf ("wrapRustc: wrap rustdoc")
We need to set -crt-static on musl for rustdoc as well, so let's unify
the wrappers. Ideally, rather than wrapping rustdoc, we'd have
rustdoc use the wrapped rustc, but that's currently only possible with
an unstable option (--test-builder).
The options set by the wrapper, -C target-feature and --sysroot, are
supported by both rustdoc and rustc, but other flags maybe not be
supported by both, so I've introduced different environment
variables (the existing NIX_RUSTFLAGS and a new NIX_RUSTDOCFLAGS) to
allow those to be set independently.
This fixes cargo-auditable in pkgsMusl., which broke because its
doctests stopped working when -crt-static was moved to the wrapper.
Fixes: 79156bf13a ("rustc: move crt-static default override to wrapper (#291829)")
Previously, when cross compiling from non-musl to musl, the crt-static
default override wouldn't be applied, because the compiler wouldn't
have been built with it due to fastCross. Moving it to the wrapper
fixes this without having to introduce extra compiler rebuilds. And
because the wrapper is applied even to the bootstrap rustc, we no
longer need special handling of crt-static in the Cargo expression.
Unlike --sysroot, rustc allows -C target-feature= to be passed
multiple times, with later instances taking precedence over earlier
ones. This means that it's very easy to set the default in the
wrapper, just by our overridden default before any other arguments.
This fixes pkgsCross.aarch64-multiplatform-musl.mesa from x86_64-linux.
When building a cross-compiler, the rustc derivation does some tricks to
only build the standard library and reuse the host's compiler, leading
to much faster build time.
Unfortunately, the way the build system was invoked, it would always
build the `std` crate, whether or not the target supports it. Some
bare-metal targets only support building the `core` and `alloc` crates.
By being more vague about the build command, using `library` instead of
`library/std`, Rust's build system is able to figure out exactly which
crates to build:
https://github.com/rust-lang/rust/blob/1.74.1/src/bootstrap/compile.rs#L370-L412
Oddly enough, the install command still needs to use `library/std`, even
if building just a subset:
https://github.com/rust-lang/rust/blob/1.74.1/src/bootstrap/install.rs#L207
The following command was used to reproduce the original issue. Without
this patch, it leads to a build failure when trying to compile one of
std's dependencies. With the patch it completes succesfully and produces
a working cross-compiler.
nix build --impure --expr '(import ./. {
crossSystem = {
config = "riscv32-none-elf";
rustc.config = "riscv32imc-unknown-none-elf";
};
}).buildPackages.rustc'
Since wasm32-unknown-unknown doesn't require any extra
platform-specific dependencies (e.g. libc), we might as well just
always include in with rustc. We could also include other targets
fitting these criteria, but I wasn't able to find any others that
actually build (not even wasm64-unknown-unknown).
The old rustc-wasm32 package disabled documentation, but we don't
actually need to do that — we just need to allow for some broken links
in the wasm32-unknown-unknown documentation. Broken links in
documentation are an upstream issue anyway. There's no need we need
to fail our build for them.
We keep running into situations where we can't get the right
combination of rustc flags through build systems into rustc.
RUSTFLAGS is the only variable supported across build systems, but if
RUSTFLAGS is set, Cargo will ignore all other ways of specifying rustc
flags, including the target-specific ones, which we need to make
dynamic musl builds work. (This is why pkgsCross.musl64.crosvm is
currently broken — it works if you unset separateDebugInfo, which
causes RUSTFLAGS not to be set.)
So, we need to do the same thing we do for C and C++ compilers, and
add a compiler wrapper so we can inject the flags we need, regardless
of the build system.
Currently the wrapper only supports a single mechanism for injecting
flags — the NIX_RUSTFLAGS environment variable. As time goes on,
we'll probably want to add additional features, like target-specific
environment variables.
The BOOTSTRAP_ARGS patch is no longer necessary since
863d2fddd79 ("Pass `-jN` from Make to `BOOTSTRAP_ARGS`").
We now have to set sysconfdir, because since
350ead87205 ("add sanity checks for user write access on `x install`"),
the build system will check it can write there, even though nothing
actually gets installed there. (Previously it defaulted to /etc.)
This commit adds the machinery required to get our `rustc.nix` to
work using upstream's git repository, directly, without them waving
dead chickens over it and performing whatever other occult rituals
are involved in creating a release tarball.
No chickens, live or dead, were harmed in the creation of this
commit.
We need this stuff to be available in lib so make-derivation.nix can
access it to construct the Meson cross file.
This has a couple of other advantages:
- It makes Rust less special. Now figuring out what Rust calls a
platform is the same as figuring out what Linux or QEMU call it.
- We can unify the schema used to define Rust targets, and the schema
used to access those values later. Just like you can set "config"
or "system" in a platform definition, and then access those same
keys on the elaborated platform, you can now set "rustcTarget" in
your crossSystem, and then access "stdenv.hostPlatform.rustcTarget"
in your code.
"rustcTarget", "rustcTargetSpec", "cargoShortTarget", and
"cargoEnvVarTarget" have the "rustc" and "cargo" prefixes because
these are not exposed to code by the compiler, and are not
standardized. The arch/os/etc. variables are all named to match the
forms in the Rust target spec JSON.
The new rust.target-family only takes a list, since we don't need to
worry about backwards compatibility when that name is used.
The old APIs are all still functional with no warning for now, so that
it's possible for external code to use a single API on both 23.05 and
23.11. We can introduce the warnings once 23.05 is EOL, and make them
hard errors when 23.11 is EOL.
This upgrade unfortunately removes MIPS support, as it has been
dropped to Tier 3[1] and so bootstrap tarballs are no longer provided.
It looks like it was dropped due to multiple codegen bugs, and lack of
maintenance, so bringing it back would probably involve engaging with
Rust/LLVM upstream on those.
[1]: https://github.com/rust-lang/compiler-team/issues/648
When cross-compiling a rust package, all we need is the std library compiled
for the target. This uses the final stage compiler which was built for Build
and then uses that as a stage0 compiler for target std library.
It also copies the rust binary from pkgsBuildBuild so that it find the new
lib/rustlib directory.
We also need to create a cargo wrapper which will use the "new" rust compiler
Also makes sure man pages and doc pages are propagated
Co-authored-by: Alyssa Ross <hi@alyssa.is>
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
Co-authored-by: Rick van Schijndel <Mindavi@users.noreply.github.com>
armv6l-linux was incorrectly added to the list of platforms without host
tools in #227987. arm-unknown-linux-gnueabihf is present in the list of
Tier 2 targets with host tools, and this target corresponds to our
armv6l-linux platform.
I made a mistake when trying to add the target prefix to Rust cross
compilers: pkgsCross.aarch64-multiplatform.rustc ended up being called
"aarch64-unknown-linux-gnu-rustc-aarch64-unknown-linux-gnu-1.69.0",
which is a bit verbose.
With this change:
nix-repl> rustc.name
"rustc-1.69.0"
nix-repl> pkgsCross.aarch64-multiplatform.buildPackages.rustc.name
"aarch64-unknown-linux-gnu-rustc-1.69.0"
nix-repl> pkgsCross.aarch64-multiplatform.rustc.name
"rustc-aarch64-unknown-linux-gnu-1.69.0"
As intended.
Fixes: 57e73d23bb ("rustc,rustPlatform.buildRustPackage: broaden platforms")
rustc supports way more platforms than Linux and Darwin. We might not
be able to build it for every platform at the moment, but that's what
meta.broken is for.
There are other platforms that rustc can produce binaries for, but
can't run on itself, so those are listed in the defaults for
buildRustPackage.
Our `rustc.nix` adds a `--target` flag for the host when doing a
host!=target build, but neglects to add a `--target` flag for the
buildPlatform when doing a build!=(host==target) build. This commit
corrects that.
Before rustc 1.68 omitting the --target flag for the buildPlatform
did not cause any problems. As of rustc 1.68, build!=host without a
--target for the build will fail like below (with hundreds more
"cannot find std::" errors.
```
$ nix build -f . -L pkgsCross.aarch64-multiplatform.rustc
...
Copying stage1 library from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / aarch64-unknown-linux-gnu)
Uplifting stage1 library (x86_64-unknown-linux-gnu -> aarch64-unknown-linux-gnu)
Copying stage2 library from stage1 (x86_64-unknown-linux-gnu -> x86_64-unknown-linux-gnu / aarch64-unknown-linux-gnu)
Building stage2 tool rust-analyzer-proc-macro-srv (aarch64-unknown-linux-gnu)
Compiling autocfg v1.1.0
Compiling libc v0.2.135
Compiling cfg-if v1.0.0
Compiling proc-macro2 v1.0.47
Compiling quote v1.0.21
Compiling unicode-ident v1.0.5
Compiling syn v1.0.102
Compiling once_cell v1.15.0
Compiling parking_lot_core v0.9.4
Compiling serde_derive v1.0.145
Compiling hashbrown v0.12.3
Compiling scopeguard v1.1.0
Compiling smallvec v1.10.0
Compiling log v0.4.17
Compiling serde v1.0.145
Compiling rustc-hash v1.1.0
error[E0463]: can't find crate for `std`
error: cannot find macro `println` in this scope
--> /nix/tmp/nix-build-rustc-aarch64-unknown-linux-gnu-1.68.2.drv-0/rustc-1.68.2-src/vendor/libc-0.2.135/build.rs:7:5
|
7 | println!("cargo:rerun-if-changed=build.rs");
| ^^^^^^^
error: cannot find macro `println` in this scope
--> /nix/tmp/nix-build-rustc-aarch64-unknown-linux-gnu-1.68.2.drv-0/rustc-1.68.2-src/vendor/libc-0.2.135/build.rs:16:9
|
16 | println!(
| ^^^^^^^
error: cannot find macro `println` in this scope
--> /nix/tmp/nix-build-rustc-aarch64-unknown-linux-gnu-1.68.2.drv-0/rustc-1.68.2-src/vendor/libc-0.2.135/build.rs:29:13
|
29 | println!("cargo:rustc-cfg=freebsd10")
| ^^^^^^^
```
Our rustc package is not universal, because we only build std for the
host and target platforms. This means that a build graph where cross
is involved will end up with multiple rustc packages in it, so it
would be helpful to have a way to tell them apart, just like we do for
e.g. gcc.
If RUSTFLAGS is set in the environment, Cargo will ignore rustflags
settings in its TOML configuration. So setting RUSTFLAGS=-g (like
separateDebugInfo does) to generate debug info breaks
dynamically-linked Rust packages on musl. This breakage is visible
for any packages that call into C dynamic libraries. If the binary is
linked directly to a C dynamic library, it will fail to build, and if
it depends on a Rust library which links a C dynamic library, it will
segfault at runtime when it tries to call a function from the C
library. I noticed this because pkgsMusl.crosvm is broken for this
reason, since it sets separateDebugInfo = true.
It shouldn't be possible to end up with broken binaries just by using
RUSTFLAGS to do something innocuous like enable debug info, so I think
that, even though we liked the approach of modiyfing .cargo/config
better at the time, it's become clear that it's too brittle, and we
should bite the bullet and patch the compiler instead when targetting
musl. It does not appear to be necessary to modify the compiler at
all when cross-compiling /from/ dynamically-linked Musl to another
target, so I'm only checking whether the target system is
dynamically-linked Musl when deciding whether to make the modification
to the compiler.
This reverts commit c2eaaae50d
("cargoSetupHook: pass host config flags"), and implements the
compiler patching approach instead.
We previously disabled this based on a now-closed issue from 2015 [0].
I think enough time has passed that we can give it a shot again, given
that the in the worst case scenario we revert, and in the best case
scenario we get a performance boost.
[0]: https://github.com/rust-lang/rust/issues/30181
ripgrep is a very popular grep replacement (similar to fd and find)
and wezterm is a popular terminal emulator which has a big codebase with
lots of features tested (it also broke in the past multiple times on
rustc upgrades.).
This reverts commit edfbbaf282.
I mistakingly believed that once 1.66.0 was used to bootstrap, we'd be
able to remove libiconv from rustc's build-time dependency tree on Darwin.
Sadly, this isn't the case, because src/tools/bootstrap depends on libc.
Additionally, it seems that my assessment in b1834a461e
was wrong -- *any* dependency on `libc` will cause a requirement on
libiconv, due to rustc unconditionally linking every library specified
in `link` directives, no matter if the function is actually used.
This was worked around somewhat in https://github.com/rust-lang/libc/pull/2944
by not linking libiconv if libc is only a dependency of std, but this
doesn't apply when `libc` is a dependency of anything else.
Maybe one day we'll just rip out libiconv from `libc` entirely (or hide it
behind a feature flag), but for now, we can just keep it in `buildRustPackage`'s
`buildInputs` by default.
This change switches to using GCC 11 by default on aarch64-linux, as well as passing `-lgcc` to the linker, per #201485.
See #201254 and #208412 for wider context on the issue.
This reverts commit b6fc00b8f4.
Rust 1.66.0 contains a fix for libiconv being linked unconditionally on macOS, but this only applies to packages that don't depend on older versions of `libc`.
For now, let's go back to including libiconv in `buildInputs` by default for packages that use `buildRustPackage`. As packages bump their `libc` versions, we can eventually stop including it by default, and manually add it where needed.
The "bootstrap" and "installer" crates depend on lzma-sys, which will
build its own version of xz if it can't find the liblzma.pc through
pkg-config. Even though it's used as a library, xz here is a native
build input, as it is used by the build system rather than the end
product.
