See previous commit for what was done to `binutils` to make this
possible.
There were some uses of `forcedNativePackages` added. The
combination of overrides with that attribute is highly spooky: it's
often important that if an overridden package comes from it, the
replaced arguments for that package come from it. Long term this
package set and all the spookiness should be gone and irrelevant:
"Move along, nothing to see here!"
No hashes should be changed with this commit
Before all overrides were also pruned in the previous stage, now
only gcc and binutils are, because they alone care about about the
target platform. The rest of the overrides don't, so it's better to
preserve them in order to avoid spurious rebuilds.
This is required for Aarch64 since a lot of source tarballs ship with
outdated configure scripts that don't recognize aarch64. Simply
replacing the config.guess and config.sub with new versions from
upstream makes them build again.
This same approach is used by at least Buildroot and Fedora. In
principle this could be enabled for all architectures but
conditionalizing this on aarch64 avoids a mass rebuild on x86.
The long term goal is a big replace:
{ inherit system platform; } => buildPlatform
crossSystem => hostPlatform
stdenv.cross => targetPlatform
And additionally making sure each is defined even when not cross compiling.
This commit refactors the bootstrapping code along that vision, but leaves
the old identifiers with their null semantics in place so packages can be
modernized incrementally.
[N.B., this package also applies to the commits that follow it in the same
PR.]
In most cases, buildPackages = pkgs so things work just as before. For
cross compiling, however, buildPackages is resolved as the previous
bootstrapping stage. This allows us to avoid the mkDerivation hacks cross
compiling currently uses today.
To avoid a massive refactor, callPackage will splice together both package
sets. Again to avoid churn, it uses the old `nativeDrv` vs `crossDrv` to do
so. So now, whether cross compiling or not, packages with get a `nativeDrv`
and `crossDrv`---in the non-cross-compiling case they are simply the same
derivation. This is good because it reduces the divergence between the
cross and non-cross dataflow. See `pkgs/top-level/splice.nix` for a comment
along the lines of the preceding paragraph, and the code that does this
splicing.
Also, `forceNativeDrv` is replaced with `forceNativePackages`. The latter
resolves `pkgs` unless the host platform is different from the build
platform, in which case it resolves to `buildPackages`. Note that the
target platform is not important here---it will not prevent
`forcedNativePackages` from resolving to `pkgs`.
--------
Temporarily, we make preserve some dubious decisions in the name of preserving
hashes:
Most importantly, we don't distinguish between "host" and "target" in the
autoconf sense. This leads to the proliferation of *Cross derivations
currently used. What we ought to is resolve native deps of the cross "build
packages" (build = host != target) package set against the "vanilla
packages" (build = host = target) package set. Instead, "build packages"
uses itself, with (informally) target != build in all cases.
This is wrong because it violates the "sliding window" principle of
bootstrapping stages that shifting the platform triple of one stage to the
left coincides with the next stage's platform triple. Only because we don't
explicitly distinguish between "host" and "target" does it appear that the
"sliding window" principle is preserved--indeed it is over the reductionary
"platform double" of just "build" and "host/target".
Additionally, we build libc, libgcc, etc in the same stage as the compilers
themselves, which is wrong because they are used at runtime, not build
time. Fixing this is somewhat subtle, and the solution and problem will be
better explained in the commit that does fix it.
Commits after this will solve both these issues, at the expense of breaking
cross hashes. Native hashes won't be broken, thankfully.
--------
Did the temporary ugliness pan out? Of the packages that currently build in
`release-cross.nix`, the only ones that have their hash changed are
`*.gcc.crossDrv` and `bootstrapTools.*.coreutilsMinimal`. In both cases I
think it doesn't matter.
1. GCC when doing a `build = host = target = foreign` build (maximally
cross), still defines environment variables like `CPATH`[1] with
packages. This seems assuredly wrong because whether gcc dynamically
links those, or the programs built by gcc dynamically link those---I
have no idea which case is reality---they should be foreign. Therefore,
in all likelihood, I just made the gcc less broken.
2. Coreutils (ab)used the old cross-compiling infrastructure to depend on
a native version of itself. When coreutils was overwritten to be built
with fewer features, the native version it used would also be
overwritten because the binding was tight. Now it uses the much looser
`BuildPackages.coreutils` which is just fine as a richer build dep
doesn't cause any problems and avoids a rebuild.
So, in conclusion I'd say the conservatism payed off. Onward to actually
raking the muck in the next PR!
[1]: https://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html
Stdenv adapters are kinda weird and un-idiomatic (especially when they
don't actually change stdenv). It's more idiomatic to say
buildInputs = [ makeCoverageAnalysisReport ];
This removes the need for hacks like stdenv.regenerate. It also
ensures that overrideGCC is now stackable (so ‘stdenv = useGoldLinker
clangStdenv’ works).
This is for consistency with terminology in stdenv (and the terms
"hostDrv" and "buildDrv" are not very intuitive, even if they're
consistent with GNU terminology).
This adapter causes the resulting binaries to have debug info and no
optimisations. Example use (in all-packages.nix):
foo = callPackage ./foo.nix {
stdenv = keepDebugInfo stdenv;
};
some redundant builds (e.g., GMP was built three times).
* Updated GMP to 5.0.2.
* Updated PPL to 0.11.2.
* Remove ad hoc flags to build GCC's dependencies statically.
Instead, use the ‘makeStaticLibraries’ stdenv adapter.
* Build GMP with C++ support by default.
svn path=/nixpkgs/branches/stdenv-updates/; revision=30891
does not override NIX_CFLAGS_COMPILE anymore in the mkDerivation parameter
attributes. This way, apacheHttpd can be built properly with coverage
information.
An indication of this problem came from the nixos tests.subversion failure.
svn path=/nixpkgs/branches/stdenv-updates/; revision=24938
"meta" and "passthru", and these attributes will be appended to the usual
mkDerivation attributes only if the package is cross built.
This allows putting some of the cross-building logic in the mkDerivation
nix parameters, and not only in the final builder script, as it was until now.
svn path=/nixpkgs/trunk/; revision=20272
* libpng updated to 1.4.0.
* For libjpegStatic, use a stdenv adapter to build a static library.
svn path=/nixpkgs/branches/stdenv-updates/; revision=19605
I fixed conflicts regarding the renaming 'kernel' -> 'linux' in all-packages.
Also a small conflict in all-packages about making openssl overridable.
And I some linux 2.6.31-zen kernel files also marked in conflict.
svn path=/nixpkgs/branches/stdenv-updates/; revision=19438
renaming.
I think directory renaming breaks the usual merges... because it leaves the
'to be removed' directory in the working directory still. A manual 'rm' of the
'to be removed' directory fixed the commit.
svn merge ^/nixpkgs/trunk
svn path=/nixpkgs/branches/stdenv-updates/; revision=18661
this because 'ld' wants to know the path of every library involved in a dynamic
linking. I imagine that ld does not need that in native builds because it can
call the loader for it to resolve the library rpaths, but this is not the case
for cross-building.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18577
- Disabling guile test, because one fails. I commented on that in the source.
On cross builds:
- Adding stripping
- Updating the glibc-2.11 expression to match the parameters of glibc-2.9,
which I was updating more.
- Renaming from selfNativeBuildInput to selfBuildNativeInput, so this matches
better the pattern buildNativeInputs.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18550
- Before this changes, cflags and ldflags for the native and the cross compiler
got mixed. Not all the gcc-wrapper/gcc-cross-wrapper variables are
independant now, but enough, I think.
- Fixed the generic stdenv expression, which did a big mess on buildInputs and
buildNativeInputs. Now it distinguishes when there is a stdenvCross or not.
Maybe we should have a single stdenv and forget about the stdenvCross
adapter - this could end in a stdenv a bit complex, but simpler than the
generic stdenv + adapter.
- Added basic support in pkgconfig for cross-builds: a single PKG_CONFIG_PATH
now works for both the cross and the native compilers, but I think this
should work well for most cases I can think of.
- I tried to fix the guile expression to cross-biuld; guile is built, but not
its manual, so the derivation still fails. Guile requires patching to
cross-build, as far as I understnad.
- Made the glibcCross build to be done through the usage of a
gcc-cross-wrapper over the gcc-cross-stage-static, instead of using it
directly.
- Trying to make physfs (a neverball dependency) cross build.
- Updated the gcc expression to support building a cross compiler without getting
derivation variables mixed with those of the stdenvCross.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18534
on a derivation with a meta.license attribute which does not satisfy the
license predicate.
With this adapter you can abort any install which depends on software
which are not free by default. You can try it with MPlayer, because
MPlayer depends of win32codecs flagged as "unfree".
svn path=/nixpkgs/trunk/; revision=18530
its dependencies. To make it works, you need to change the default
stdenv as documented in the error message.
./maintainers/scripts/dep-licenses.sh <attribute name>
svn path=/nixpkgs/trunk/; revision=18508
true/false, which tells whether the derivation needs itself as
buildNativeInput.
For example, in order to build cross ncurses, we need the a native build
ncurses.
(As libtool does not work in stdenv, I have not tested this change, to check
whether finally ncurses cross-build)
svn path=/nixpkgs/branches/stdenv-updates/; revision=18489
derivation, the "buildInputs" in every stdenv mkDerivation don't map now
directly to the environment
variable "buildInputs" in the builder, but "buildNativeInputs". So, the inputs
build by the native compiler.
When cross compiling, they will map to the environment variable "buildInputs"
(yes, now the same name), which means does to be built with the cross compiler.
I think I improved the naming of variables a bit. There was a big mess,
specially in the stdenv adapter for cross building, and also in the default
builder script.
I also tried to add proper manager of propagatedInputBuilds, these being
propagated considering the host or build origin of that input build (so, at the
end, being those propagatedInputBuilds being propagated properly to the native
or the cross compiler.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18477
the cross compilation functionality.
- I renamed some expected stdenv.mkDerivation parameter attributes so we can
keep this branch properly updated from trunk. We agreed with Nicolas Pierron
doing a massive renaming, so all current buildInputs become hostInputs (input
as build for the host machine, in autotools terminology) , and
then buildInputs would mean "input as for the build machine".
By now, the specific "input as for the build machine" is specified through
buildNativeInputs. We should fix this in the merge to trunk.
- I made the generic stdenv understand the buildNativeInputs, otherwise if
we start changing nixpkgs expressions so they distinguish the current
buildInputs into buildInputs and buildNativeInputs, we could break even more
nixpkgs for other platforms.
- I changed the default result of mkDerivation so it becomes the derivation for
to be run in the build machine. This allows, without any special rewriting,
"fetchurl" derivations to be always results for the build machine to use
them.
- The change above implies that, for anyone wanting to cross-compile, has to
build the hostDrv of the wanted derivation. For example, after this commit,
the usual test of "nix-build -A bison.hostDrv arm.nix" works. I described
the contents of this arm.nix in r18398.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18471
I made the stdenvCross adapter simpler, according to Nicolas Pierron comments,
and I commented it a bit.
There are still jobs to do. At least:
- Plan for the general renaming from buildInputs to hostInputs
- We should not break merges from trunk.
- Make the generic stdenv understand about host/buildInputs, at least for
native builds, because it is used in the always-native building of
stdenvLinux. This should allow us to remove all duplications of "*NoCross" in
nixpkgs.
svn path=/nixpkgs/branches/stdenv-updates/; revision=18449
needing to keep a new tree of expressions apart for the expressions to get
cross-compiled.
I changed the whole way of using cross compilation with nixpkgs, which before
was done through a simple adapter.
Now the adapter became complex, and I've tried to avoid the most obvious
recursivities. For example, the fetchurl expression should
never be cross-compiled, as the gmp, mpfr, and some others, like
some ncurses, perl, ... I made overrided copies of those necessary as
perlNoCross, ncursesNoCross, as stdenvNoCross, keeping in mind that
the stdenv (capable of cross compilation) is built upon stdenvNoCross using
an adapter.
So, to cross compile, instead of building using "nixpkgs/default.nix",
you should build with your
own "myarchiteture.nix", which should have contents like these, for example:
import /etc/nixos/nixpkgs/default.nix
{
crossSystem = {
config = "armv5tel-unknown-linux-gnueabi";
bigEndian = false;
arch = "arm";
float = "soft";
};
}
svn path=/nixpkgs/branches/stdenv-updates/; revision=18398
tree under $out into a separate stdenv adapter named keepBuildTree.
* makeModulesClosure: support building an initrd for a kernel that has
been compiled with coverage instrumentation.
svn path=/nixpkgs/trunk/; revision=16916
