Closes#150801
Note: I decided against resuming directly on #150801 because the
conflict was too big (and resolving it seemed too error-prone to me).
Also the `this`-refactoring could be done in an easier manner, i.e. by
exposing JIT attributes with the correct configuration. More on that
below.
This patch creates variants of the `postgresql*`-packages with JIT[1]
support. Please note that a lot of the work was derived from previous
patches filed by other contributors, namely dasJ, andir and abbradar,
hence the co-authored-by tags below.
Effectively, the following things have changed:
* For JIT variants an LLVM-backed stdenv with clang is now used as
suggested by dasJ[2]. We need LLVM and CLang[3] anyways to build the
JIT-part, so no need to mix this up with GCC's stdenv. Also, using the
`dev`-output of LLVM and clang's stdenv for building (and adding llvm
libs as build-inputs) seems more cross friendly to me (which will
become useful when cross-building for JIT-variants will actually be
supported).
* Plugins inherit the build flags from the Makefiles in
`$out/lib/pgxs/src` (e.g. `-Werror=unguarded-availability-new`). Since
some of the flags are clang-specific (and stem from the use of the
CLang stdenv) and don't work on gcc, the stdenv of `pkgs.postgresql`
is passed to the plugins. I.e., plugins for non-JIT variants are built
with a gcc stdenv on Linux and plugins for JIT variants with a clang
stdenv.
Since `plv8` hard-codes `gcc` as `$CC` in its Makefile[4], I marked it
as broken for JIT-variants of postgresql only.
* Added a test-matrix to confirm that JIT works fine on each
`pkgs.postgresql_*_jit` (thanks Andi for the original test in
#124804!).
* For each postgresql version, a new attribute
`postgresql_<version>_jit` (and a corresponding
`postgresqlPackages<version>JitPackages`) are now exposed for better
discoverability and prebuilt artifacts in the binary cache.
* In #150801 the `this`-argument was replaced by an internal recursion.
I decided against this approach because it'd blow up the diff even
more which makes the readability way harder and also harder to revert
this if necessary.
Instead, it is made sure that `this` always points to the correct
variant of `postgresql` and re-using that in an additional
`.override {}`-expression is trivial because the JIT-variant is
exposed in `all-packages.nix`.
* I think the changes are sufficiently big to actually add myself as
maintainer here.
* Added `libxcrypt` to `buildInputs` for versions <v13. While
building things with an LLVM stdenv, these versions complained that
the extern `crypt()` symbol can't be found. Not sure what this is
exactly about, but since we want to switch to libxcrypt for `crypt()`
usage anyways[5] I decided to add it. For >=13 it's not relevant
anymore anyways[6].
* JIT support doesn't work with cross-compilation. It is attempted to
build LLVM-bytecode (`%.bc` is the corresponding `make(1)`-rule) for
each sub-directory in `backend/` for the JIT apparently, but with a
$(CLANG) that can produce binaries for the build, not the host-platform.
I managed to get a cross-build with JIT support working with
`depsBuildBuild = [ llvmPackages.clang ] ++ buildInputs`, but
considering that the resulting LLVM IR isn't platform-independent this
doesn't give you much. In fact, I tried to test the result in a VM-test,
but as soon as JIT was used to optimize a query, postgres would
coredump with `Illegal instruction`.
A common concern of the original approach - with llvm as build input -
was the massive increase of closure size. With the new approach of using
the LLVM stdenv directly and patching out references to the clang drv in
`$out` the effective closure size changes are:
$ nix path-info -Sh $(nix-build -A postgresql_14)
/nix/store/kssxxqycwa3c7kmwmykwxqvspxxa6r1w-postgresql-14.7 306.4M
$ nix path-info -Sh $(nix-build -A postgresql_14_jit)
/nix/store/xc7qmgqrn4h5yr4vmdwy56gs4bmja9ym-postgresql-14.7 689.2M
Most of the increase in closure-size stems from the `lib`-output of
LLVM
$ nix path-info -Sh /nix/store/5r97sbs5j6mw7qnbg8nhnq1gad9973ap-llvm-11.1.0-lib
/nix/store/5r97sbs5j6mw7qnbg8nhnq1gad9973ap-llvm-11.1.0-lib 349.8M
which is why this shouldn't be enabled by default.
While this is quite much because of LLVM, it's still a massive
improvement over the simple approach of adding llvm/clang as
build-inputs and building with `--with-llvm`:
$ nix path-info -Sh $(nix-build -E '
with import ./. {};
postgresql.overrideAttrs ({ configureFlags ? [], buildInputs ? [], ... }: {
configureFlags = configureFlags ++ [ "--with-llvm" ];
buildInputs = buildInputs ++ [ llvm clang ];
})' -j0)
/nix/store/i3bd2r21c6c3428xb4gavjnplfqxn27p-postgresql-14.7 1.6G
Co-authored-by: Andreas Rammhold <andreas@rammhold.de>
Co-authored-by: Janne Heß <janne@hess.ooo>
Co-authored-by: Nikolay Amiantov <ab@fmap.me>
[1] https://www.postgresql.org/docs/current/jit-reason.html
[2] https://github.com/NixOS/nixpkgs/pull/124804#issuecomment-864616931
& https://github.com/NixOS/nixpkgs/pull/150801#issuecomment-1467868321
[3] This fails with the following error otherwise:
```
configure: error: clang not found, but required when compiling --with-llvm, specify with CLANG=
```
[4] https://github.com/plv8/plv8/blob/v3.1.5/Makefile#L14
[5] https://github.com/NixOS/nixpkgs/pull/181764
[6] c45643d618
The keyd package already exists, but without a systemd service.
Keyd requires write access to /var/run to create its socket. Currently
the directory it uses can be changed with an environment variable, but
the keyd repo state suggests that this may turn into a compile-time
option. with that set, and some supplementary groups added, we can run
the service under DynamicUser.
Co-authored-by: pennae <82953136+pennae@users.noreply.github.com>
...but still allow for setting `dataDir` to a custom path. This gets
rid of the use of the deprecated option PermissionsStartOnly. Also, add
the ability to customize user and group, since that could be useful
with a custom `dataDir`.
By default, pgadmin4 uses SERVER_MODE = True. This requires
access to system directories (e.g. /var/lib/pgadmin). There is
no easy way to change this mode during runtime. One has to change
or add config files withing pgadmin's directory structure to change it
or add a system-wide config file under `/etc/pgadmin`[1].
This isn't always easy to achive or may not be possible at all. For
those usecases this implements a switch in the pgadmin4 derivation and
adds a new top-level package `pgadmin4-desktopmode`. This builds in
DESKTOP MODE and allows the usage of pgadmin4 without the nixOS module
and without access to system-wide directories.
pgadmin4 module saves the configuration to /etc/pgadmin/config_system.py
pgadmin4-desktopmode tries to read that as well. This normally fails with
a PermissionError, as the config file is owned by the user of the pgadmin module.
With the check-system-config-dir.patch this will just throw a warning
but will continue and not read the file.
If we run pgadmin4-desktopmode as root
(something one really shouldn't do), it can read the config file and fail,
because of the wrong config for desktopmode.
[1]https://www.pgadmin.org/docs/pgadmin4/latest/config_py.html
Signed-off-by: Florian Brandes <florian.brandes@posteo.de>
We test pgadmin in nixosTests, because it needs a running postgresql instance.
This is now unnecessary since we can do so in the package itself.
This reduces the complexity of pgadmin and removes the need for the extra
nixosTests.
Also setting SERVER_MODE in `pkg/pip/setup_pip.py` does not have any effect
on the final package, so we remove it.
In NixOS, we use the module, which expects SERVER_MODE to be true (which it defaults to).
In non-NixOS installations, we will need the directory /var/lib/pgadmin and /var/log/pgadmin
Signed-off-by: Florian Brandes <florian.brandes@posteo.de>
Previously, secrets were named according to the initrd they were
associated with. This created a problem: If secrets were changed whilst
the initrd remained the same, there were two versions of the secrets
with one initrd. The result was that only one version of the secrets would
by recorded into the /boot partition and get used. AFAICT this would
only be the oldest version of the secrets for the given initrd version.
This manifests as #114594, which I found frustrating while trying to use
initrd secrets for the first time. While developing the secrets I found
I could not get new versions of the secrets to take effect.
Additionally, it's a nasty issue to run into if you had cause to change
the initrd secrets for credential rotation, etc, if you change them and
discover you cannot, or alternatively that you can't roll back as you
would expect.
Additional changes in this patch.
* Add a regression test that switching to another grub configuration
with the alternate secrets works. This test relies on the fact that it
is not changing the initrd. I have checked that the test fails if I
undo my change.
* Persist the useBootLoader disk state, similarly to other boot state.
* I had to do this, otherwise I could not find a route to testing the
alternate boot configuration. I did attempt a few different ways of
testing this, including directly running install-grub.pl, but what
I've settled on is most like what a user would do and avoids
depending on lots of internal details.
* Making tests that test the boot are a bit tricky (see hibernate.nix
and installer.nix for inspiration), I found that in addition to
having to copy quite a bit of code I still couldn't get things to
work as desired since the bootloader state was being clobbered.
My change to persist the useBootLoader state could break things,
conceptually. I need some help here discovering if that is the case,
possibly by letting this run through a staging CI if there is one.
Fix#114594.
cc potential reviewers:
@lopsided98 (original implementer) @joachifm (original reviewer),
@wkennington (numerous fixes to grub-install.pl), @lheckemann (wrote
original secrets test).
This commit fixes a papercut in nixos-rebuild where people wanting to
switch to a specialisation (or test one) were forced to manually figure
out the specialisation's path and run its activation script - since now,
there's a dedicated option to do just that.
This is a backwards-compatible change which doesn't affect the existing
behavior, which - to be fair - might still be considered sus by some
people, the painful scenario here being:
- you boot into specialisation `foo`,
- you run `nixos-rebuild switch`,
- whoops, you're no longer at specialisation `foo`, but you're rather
brought back to the base system.
(it's especially painful for cases where specialisation is used to load
extra drivers, e.g. Nvidia, since then launching `nixos-rebuild switch`,
while forgetting that you're inside a specialisation, can cause some
parts of your system to get accidentally unloaded.)
I've tried to mitigate that by improving specialisations so that they
create a dedicated file somewhere in `/run/current-system` containing
the specialisation's name (which `nixos-rebuild` could then use as the
default value for `--specialisation`), but I haven't been able to come
up with anything working (plus it would be a breaking change then).
Closes https://github.com/NixOS/nixpkgs/issues/174065
This commit upgrades headscale to the newest version, 0.17.0 and updates
the module with the current breaking config changes.
In addition, the module is rewritten to conform with RFC0042 to try to
prevent some drift between the module and the upstream.
A new maintainer, Misterio77, is added as maintainer.
Signed-off-by: Kristoffer Dalby <kristoffer@tailscale.com>
Co-authored-by: Gabriel Fontes <hi@m7.rs>
Co-authored-by: Geoffrey Huntley <ghuntley@ghuntley.com>
On x86_64-linux only because bootspec is for NixOS (for the moment?),
and NixOS is really only a Linux concept (for the moment?).
Not on aarch64-linux because it fails for whatever reason 🤷
This splits the tests into two: one where cups.socket is started
normally, the order with socket activation.
Why? It's almost impossible to follow the test with 4 different
machines printing at the same time. It should also be more efficient
because only two VMs at a time were needed anyway.
Changes sgx-psw to append `aesm` to `LD_LIBRARY_PATH`:
- Append instead of prepend to allow for overriding in service config
- As we already add a wrapper to add `aesm` to `LD_LIBRARY_PATH` it is
not necessary to also set in `LD_LIBRARY_PATH` of the systemd service.
Co-authored-by: Vincent Haupert <mail@vincent-haupert.de>
