BEFORE:
the python derivation did not contain .nativeDrv and .crossDrv because
it was not from the __splicedPackages set
the python used in mk-python-derivation.nix was for host
```
nix-repl> pkgsCross.aarch64-multiplatform.python3Packages.xpybutil.nativeBuildInputs
[ «derivation /nix/store/bhz39ds4v02hn6x4py4mzjyilw4a589h-python3-aarch64-unknown-linux-gnu-3.10.7.drv» «derivation /nix/store/v880cnh4ml7czmivfbk3cdh93hz9yvbn-hook.drv» «derivation /nix/store/f243ab7wv92gqsmc9h7gr0qcnj0xcgdb-hook.drv» «derivation /nix/store/880lf8895bzn8d94lrr2y7ilgkxq0lc4-python-remove-tests-dir-hook.drv» «derivation /nix/store/fzjnhawfs1wpw58hcd1vxd9y750dc08y-python-remove-bin-bytecode-hook.drv» «derivation /nix/store/n8l59iparx98yfw8g5ydqmzmk3fdic75-setuptools-setup-hook.drv» «deri→
```
AFTER:
the python derivation does contain .nativeDrv and .crossDrv because
it is from the __splicedPackages set
those 2 are what makes nativeBuildInputs and buildInputs function
properly
the python used in mk-python-derivation.nix is for build
```
nix-repl> pkgsCross.aarch64-multiplatform.python3Packages.xpybutil.nativeBuildInputs
[ «derivation /nix/store/hvb9yxgv1133cfhxxd869sibldvv2vdx-python3-3.10.7.drv» «derivation /nix/store/v880cnh4ml7czmivfbk3cdh93hz9yvbn-hook.drv» «derivation /nix/store/f243ab7wv92gqsmc9h7gr0qcnj0xcgdb-hook.drv» «derivation /nix/store/880lf8895bzn8d94lrr2y7ilgkxq0lc4-python-remove-tests-dir-hook.drv» «derivation /nix/store/fzjnhawfs1wpw58hcd1vxd9y750dc08y-python-remove-bin-bytecode-hook.drv» «derivation /nix/store/n8l59iparx98yfw8g5ydqmzmk3fdic75-setuptools-setup-hook.drv» «derivation /nix/store/7vyhynla→
```
Since the Nix command used is an implementation detail of the script,
it's the script that will have to be updated if the command changes,
it's the script that should be opting into this, rather than making
the user do it globally.
Since c0972c16dc ("update-python-libraries: add missing dependency nix"),
Nix will be provided as a dependency, so we know we'll always be using
a modern version and no longer need the fallback.
*Flags implies a list
slightly relevant:
> stdenv: start deprecating non-list configureFlags https://github.com/NixOS/nixpkgs/pull/173172
the makeInstalledTests function in `nixos/tests/installed-tests/default.nix` isn't available outside of nixpkgs so
it's not a breaking change
By default buildPython* runs a hook for detecting conflicting packages.
This hook needs pkg_resources which is part of setuptools.
Before this commit, setuptools was simply added to the build. This meant
that when setuptools was forgotten to be added to the build, the build
and installation would still succeed because of this package from the
hook. During runtime (and cross-compilation) one would notice the
missing package.
Store the intermediate artifacts. In time, we should build, install and
test in separate derivations as that reduces circular dependencies,
avoids rebuilds when fixing tests, and makes it possible to use the
wheels for creating say virtualenv's.
Removes the up until now unused option to specify a `sphinxOutdir` in
favor of allowing to specify multiple builders, which is for example
useful for generating both documentation and manpages using the hook.
Since the output path cannot be determined from within the package we
automatically generate it and add a diversion for manpages, so they land
in the correct output and path.
Before the change separate-debug-info.sh did the stripping itself.
This scheme has a few problems:
1. Stripping happens only on ELF files. *.a and *.o files are skipped.
Derivations have to do it manually. Usually incorrectly
as they don't run $RANLIB (true for `glibc` and `musl`).
2. Stripping happens on all paths. Ideally only `stripDebugList` paths
should be considered.
3. Host strip is called on Target files.
This change offloads stripping logic to strip hook. This strips more
files for `glibc` and `musl`. Now we can remove most $STRIP calls
from individual derivations.
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
Python package sets can be overridden by overriding an interpreter
and passing in `packageOverrides = self: super: {...};`. This is fine
in case you need a single interpreter, however, it does not help you
when you want to override all sets.
With this change it is possible to override all sets at once by
appending a list of "extensions" to `pythonPackagesExtensions`.
From reading the implementation you might wonder why a list is used, and
not
`lib.composeExtensions`? The reason is the latter requires knowledge of
the library function. This approach should be easier for most users
as it is similar to how we append to lists of e.g. inputs or patches
when overriding a derivation.
The PEP600 standard gives Python's naming scheme for various
architectures; it follows the convention which was in use by Fedora in
2014. According to PEP600, the architecture name for Power PC is
`ppc64le`, not `powerpc64le`. This is also how python3 declares its
"supported wheels" under Debian on PowerPC, as checked with `pip debug
--verbose`
$ pip debug --verbose | grep powerpc
$ pip debug --verbose | grep ppc | head
cp39-cp39-manylinux_2_31_ppc64le
cp39-cp39-manylinux_2_30_ppc64le
cp39-cp39-manylinux_2_29_ppc64le
cp39-cp39-manylinux_2_28_ppc64le
cp39-cp39-manylinux_2_27_ppc64le
cp39-cp39-manylinux_2_26_ppc64le
cp39-cp39-manylinux_2_25_ppc64le
cp39-cp39-manylinux_2_24_ppc64le
cp39-cp39-manylinux_2_23_ppc64le
Let's adjust the `pythonHostPlatform` expression in
cpython/default.nix to pass the architecture using the naming scheme
Python expects.
Verified on a Raptor Computing Systems Talos II. Without this commit,
PyQt5 fails to build, failing with "unsupported wheel". With this
commit, it builds successfully.
Every package should have a maintainer and we should not throw it away if we arleady have one.
Also in reality the maintainer of python3 is also the one of python3-minimal
GCC does not come with a `libgcc_eh.a` for the target platform if
it was built without `--enable-shared`. That flag was removed with
c6dd11ca39, meaning we should no longer
attempt to link against that lib.
We have a common pattern here in nixpkgs for Python applications: when a
Python package ships with either a requirements.txt or setup.py file, we
generally end up having to modify its version restriction, otherwise we have
build failures since we package only one specific version of each package
normally.
However, this end up being done in a completely ad-hoc way: some people
use substituteInPlace, some others use sed, others uses patches, etc.
In many cases, the code ends up being buggy, so it may work in one
version and breaks on the next one. We can instead implement one
standard way of doing this, and trying to be a correct as possible.
So this is what this commit does: it implements a new build hook, that
when called will automatically patch the wheel file. This is one of the
most generic ways to patch Python dependencies, and should work in
multiple cases.
Noticed option globbing when tried to enable parallelism by default
locally for most packages by default. python3Packages.yt-dlp failed as:
python3.9-yt-dlp> /nix/store/5mywvxdjkk1q6srwwwgdkzc37ibla801-python3.9-setuptools-61.2.0/lib/python3.9/site-packages/setuptools/dist.py:516: UserWarning: Normalizing '2022.04.08' to '2022.4.8'
python3.9-yt-dlp> warnings.warn(tmpl.format(**locals()))
python3.9-yt-dlp> invalid command name 'build_lazy_extractors--parallel'
The change adds leading whitespace everywhere where options might
already be present.
Overriding the interpreters did not work correctly. When overriding
packages would end up twice in the build time closure: one corresponding
to the overridden interpreter and one corresponding to the original
interpreter. The reason is that the override was not applied to the
interpreters in the spliced package sets.
This simplifies usages and makes the default value consistent.
In a few cases, the default value was interpreted to be `false`,
but this is useless, because virtually nobody will explicitly
set `allowAliases = true;`.
It seems the additional linker flags were added in 9d3b0a2 (May 2008) as
a workaround for readline support. They were kept since then, but it
also means that `python3-config` outputs these flags
```console
$ python3-config --libs
-lpthread -ldl -lcrypt -lncurses -lutil -lm -lm
```
while other GNU/Linux distros do not. For example, Debian 11:
```console
$ sudo apt install python3-dev
$ python3-config --libs
-lcrypt -lpthread -ldl -lutil -lm -lm
```
This change removes the `-lncurses` flag and aligns Nix/NixOS with other
distros.
Conflict in pkgs/development/libraries/libvirt/default.nix
required manual adjustments. The fetched patch is already in src.
I checked that libvirt builds.
Avoids situations where a branch and tag share the same
label. Github will silently return a non-valid artifact and
will fail the build.
Since all github releases correspond to an annotated tag, it's
safe to assume that if we found a release for a tag, that the
tag will exist.
they ship their own openssl
as of this commit, that is actually openssl 1.1.1m, the same version we have in openssl_1_1
maybe the package should still be marked as vulnerable, because they ship their own openssl
The original motivation behind removing that was to appease a consumer
of python-config's output. That issue was probably resolved by now, so
let's bring the build in sync with what python is doing by default.
This reverts b7819e38 ('python3.x: Patch extra stack size on darwin')
Setting an old target conflicts with the -stack_size,1000000 linker
flag. This fixes build of python310.
This reverts ce59dec5 ('python34: as a workaround, explicitly set MACOSX_DEPLOYMENT_TARGET')
