nixpkgs/pkgs/os-specific/linux/systemd/default.nix

588 lines
21 KiB
Nix
Raw Normal View History

2021-02-03 19:25:52 +00:00
# NOTE: Make sure to (re-)format this file on changes with `nixpkgs-fmt`!
2020-10-27 22:28:29 +00:00
{ stdenv
, lib
, fetchFromGitHub
2020-11-01 14:35:11 +00:00
, fetchpatch
, buildPackages
2020-10-27 22:28:29 +00:00
, ninja
, meson
, m4
, pkg-config
2020-10-27 22:28:29 +00:00
, coreutils
, gperf
, getent
, glibcLocales
, glib
, substituteAll
, gettext
, python3Packages
# Mandatory dependencies
, libcap
2020-11-24 15:29:28 +00:00
, util-linux
, kbd
, kmod
2020-10-27 22:28:29 +00:00
# Optional dependencies
, pam
, cryptsetup
, lvm2
, audit
, acl
, lz4
, libgcrypt
, libgpg-error
2020-10-27 22:28:29 +00:00
, libidn2
, curl
, gnutar
, gnupg
, zlib
, xz
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
, tpm2-tss
2020-10-27 22:28:29 +00:00
, libuuid
, libapparmor
, intltool
, bzip2
, pcre2
, e2fsprogs
, linuxHeaders ? stdenv.cc.libc.linuxHeaders
, gnu-efi
, iptables
2020-10-27 22:28:29 +00:00
, withSelinux ? false
, libselinux
2021-02-25 16:21:13 +00:00
, withLibseccomp ? lib.meta.availableOn stdenv.hostPlatform libseccomp
2020-10-27 22:28:29 +00:00
, libseccomp
, withKexectools ? lib.meta.availableOn stdenv.hostPlatform kexec-tools
, kexec-tools
, bashInteractive
, libmicrohttpd
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
# the (optional) BPF feature requires bpftool, libbpf, clang and llmv-strip to be avilable during build time.
# Only libbpf should be a runtime dependency.
, bpftools
, libbpf
, llvmPackages
2020-10-27 23:29:07 +00:00
, withAnalyze ? true
2020-10-28 00:26:24 +00:00
, withApparmor ? true
, withCompression ? true # adds bzip2, lz4 and xz
, withCoredump ? true
, withCryptsetup ? true
2020-10-27 23:54:08 +00:00
, withDocumentation ? true
, withEfi ? stdenv.hostPlatform.isEfi
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
, withFido2 ? true
, withHomed ? false
, withHostnamed ? true
, withHwdb ? true
, withImportd ? true
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
, withLibBPF ? false # currently fails while generating BPF objects
, withLocaled ? true
, withLogind ? true
2020-10-28 00:08:29 +00:00
, withMachined ? true
, withNetworkd ? true
2020-10-28 00:05:14 +00:00
, withNss ? true
2020-11-01 14:35:11 +00:00
, withOomd ? false
2020-10-28 00:36:02 +00:00
, withPCRE2 ? true
2020-10-27 23:48:51 +00:00
, withPolkit ? true
, withPortabled ? false
, withRemote ? true
, withResolved ? true
2020-10-27 23:51:39 +00:00
, withShellCompletions ? true
, withTimedated ? true
, withTimesyncd ? true
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
, withTpm2Tss ? true
2020-10-28 00:00:44 +00:00
, withUserDb ? true
, libfido2
2020-11-01 14:35:11 +00:00
, p11-kit
2020-10-27 22:28:29 +00:00
# name argument
, pname ? "systemd"
2020-10-27 22:28:29 +00:00
, libxslt
, docbook_xsl
, docbook_xml_dtd_42
, docbook_xml_dtd_45
}:
assert withResolved -> (libgcrypt != null && libgpg-error != null);
assert withImportd ->
2020-10-27 22:28:29 +00:00
(curl.dev != null && zlib != null && xz != null && libgcrypt != null
&& gnutar != null && gnupg != null && withCompression);
assert withEfi -> (gnu-efi != null);
assert withRemote -> lib.getDev curl != null;
assert withCoredump -> withCompression;
assert withHomed -> withCryptsetup;
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
assert withCryptsetup -> (cryptsetup != null);
let
wantCurl = withRemote || withImportd;
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
version = "249.4";
2020-10-27 22:28:29 +00:00
in
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
stdenv.mkDerivation {
inherit pname version;
# We use systemd/systemd-stable for src, and ship NixOS-specific patches inside nixpkgs directly
# This has proven to be less error-prone than the previous systemd fork.
src = fetchFromGitHub {
owner = "systemd";
2020-12-13 15:35:01 +00:00
repo = "systemd-stable";
rev = "v${version}";
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
sha256 = "0pqi9gbk9kgwvd0idf13ybxz7s4h5przn01bwj6fna44jr0wy41c";
};
# If these need to be regenerated, `git am path/to/00*.patch` them into a
# systemd worktree, rebase to the more recent systemd version, and export the
2021-03-31 16:24:19 +00:00
# patches again via `git -c format.signoff=false format-patch v${version}`.
2020-11-01 14:35:11 +00:00
# Use `find . -name "*.patch" | sort` to get an up-to-date listing of all patches
patches = [
./0001-Start-device-units-for-uninitialised-encrypted-devic.patch
./0002-Don-t-try-to-unmount-nix-or-nix-store.patch
./0003-Fix-NixOS-containers.patch
./0004-Look-for-fsck-in-the-right-place.patch
./0005-Add-some-NixOS-specific-unit-directories.patch
./0006-Get-rid-of-a-useless-message-in-user-sessions.patch
./0007-hostnamed-localed-timedated-disable-methods-that-cha.patch
./0008-Fix-hwdb-paths.patch
./0009-Change-usr-share-zoneinfo-to-etc-zoneinfo.patch
./0010-localectl-use-etc-X11-xkb-for-list-x11.patch
./0011-build-don-t-create-statedir-and-don-t-touch-prefixdi.patch
2020-11-01 14:35:11 +00:00
./0012-inherit-systemd-environment-when-calling-generators.patch
./0013-add-rootprefix-to-lookup-dir-paths.patch
./0014-systemd-shutdown-execute-scripts-in-etc-systemd-syst.patch
./0015-systemd-sleep-execute-scripts-in-etc-systemd-system-.patch
./0016-kmod-static-nodes.service-Update-ConditionFileNotEmp.patch
./0017-path-util.h-add-placeholder-for-DEFAULT_PATH_NORMAL.patch
2021-03-31 16:24:19 +00:00
./0019-pkg-config-derive-prefix-from-prefix.patch
2021-04-30 12:23:00 +00:00
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
# In v249 a bunch of meson files had been touched as part of the migration to
# jinja2 for templating. Unfortunately some of those files lost the `install_sysconfdir_samples` check.
# The following two patches are part of a PR that was filed to fix those cases.
# https://github.com/systemd/systemd/pull/20303
./0020-core-respect-install_sysconfdir_samples-in-meson-fil.patch
./0021-login-respect-install_sysconfdir_samples-in-meson-fi.patch
# In v248 or v249 we started to get in trouble due to our /etc/systemd/sytem being
# a symlink and thus being treated differently by systemd. With the below
# patch we mitigate that effect by special casing all our root unit dirs
# if they are symlinks. This does exactly what we need (AFAICT).
./0022-core-Handle-lookup-paths-being-symlinks.patch
# The way files are being tested for being executable changed in v248/v249
# which caused our confinement setup to fail as we do not mount /proc by
# default.
# The issue has been reported upstream and this patch carries the upstream
# fix for the same. Upstream now has a test for this scenario.
# https://github.com/systemd/systemd/issues/20514
./0023-path-util-make-find_executable-work-without-proc-mounted.patch
];
postPatch = ''
substituteInPlace src/basic/path-util.h --replace "@defaultPathNormal@" "${placeholder "out"}/bin/"
substituteInPlace src/boot/efi/meson.build \
--replace \
"find_program('objcopy'" \
"find_program('${stdenv.cc.bintools.targetPrefix}objcopy'"
'' + (
let
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
# The folllowing patches references to dynamic libraries to ensure that
# all the features that are implemented via dlopen(3) are available (or
# explicitly deactivated) by pointing dlopen to the absolute store path
# instead of relying on the linkers runtime lookup code.
#
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
# All of the shared library references have to be handled. When new ones
# are introduced by upstream (or one of our patches) they must be
# explicitly declared, otherwise the build will fail.
#
# As of systemd version 247 we've seen a few errors like `libpcre2.… not
# found` when using e.g. --grep with journalctl. Those errors should
# become less unexpected now.
#
# There are generally two classes of dlopen(3) calls. Those that we want to
# support and those that should be deactivated / unsupported. This change
# enforces that we handle all dlopen calls explicitly. Meaning: There is
# not a single dlopen call in the source code tree that we did not
# explicitly handle.
#
# In order to do this we introduced a list of attributes that maps from
# shared object name to the package that contains them. The package can be
# null meaning the reference should be nuked and the shared object will
# never be loadable during runtime (because it points at an invalid store
# path location).
#
# To get a list of dynamically loaded libraries issue something like
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
# `grep -ri '"lib[a-zA-Z0-9-]*\.so[\.0-9a-zA-z]*"'' $src` and update the below list.
dlopenLibs =
let
opt = condition: pkg: if condition then pkg else null;
in
[
# bpf compilation support
{ name = "libbpf.so.0"; pkg = opt withLibBPF libbpf; }
# We did never provide support for libxkbcommon & qrencode
{ name = "libxkbcommon.so.0"; pkg = null; }
{ name = "libqrencode.so.4"; pkg = null; }
# We did not provide libpwquality before so it is safe to disable it for
# now.
{ name = "libpwquality.so.1"; pkg = null; }
# Only include cryptsetup if it is enabled. We might not be able to
# provide it during "bootstrap" in e.g. the minimal systemd build as
# cryptsetup has udev (aka systemd) in it's dependencies.
{ name = "libcryptsetup.so.12"; pkg = opt withCryptsetup cryptsetup; }
# We are using libidn2 so we only provide that and ignore the others.
# Systemd does this decision during configure time and uses ifdef's to
# enable specific branches. We can safely ignore (nuke) the libidn "v1"
# libraries.
{ name = "libidn2.so.0"; pkg = libidn2; }
{ name = "libidn.so.12"; pkg = null; }
{ name = "libidn.so.11"; pkg = null; }
# journalctl --grep requires libpcre so lets provide it
{ name = "libpcre2-8.so.0"; pkg = pcre2; }
# Support for TPM2 in systemd-cryptsetup, systemd-repart and systemd-cryptenroll
{ name = "libtss2-esys.so.0"; pkg = opt withTpm2Tss tpm2-tss; }
{ name = "libtss2-rc.so.0"; pkg = opt withTpm2Tss tpm2-tss; }
{ name = "libtss2-mu.so.0"; pkg = opt withTpm2Tss tpm2-tss; }
{ name = "libtss2-tcti-"; pkg = opt withTpm2Tss tpm2-tss; }
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
{ name = "libfido2.so.1"; pkg = opt withFido2 libfido2; }
];
patchDlOpen = dl:
let
library = "${lib.makeLibraryPath [ dl.pkg ]}/${dl.name}";
in
if dl.pkg == null then ''
# remove the dependency on the library by replacing it with an invalid path
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
for file in $(grep -lr '"${dl.name}"' src); do
echo "patching dlopen(\"${dl.name}\", ) in $file to an invalid store path ("/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-not-implemented/${dl.name}")"
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
substituteInPlace "$file" --replace '"${dl.name}"' '"/nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-not-implemented/${dl.name}"'
done
'' else ''
# ensure that the library we provide actually exists
if ! [ -e ${library} ]; then
# exceptional case, details:
# https://github.com/systemd/systemd-stable/blob/v249-stable/src/shared/tpm2-util.c#L157
if ! [[ "${library}" =~ .*libtss2-tcti-$ ]]; then
echo 'The shared library `${library}` does not exist but was given as subtitute for `${dl.name}`'
exit 1
fi
fi
# make the path to the dependency explicit
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
for file in $(grep -lr '"${dl.name}"' src); do
echo "patching dlopen(\"${dl.name}\", ) in $file to ${library}"
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
substituteInPlace "$file" --replace '"${dl.name}"' '"${library}"'
done
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
'';
in
# patch all the dlopen calls to contain absolute paths to the libraries
lib.concatMapStringsSep "\n" patchDlOpen dlopenLibs
)
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
# finally ensure that there are no left-over dlopen calls (or rather strings pointing to shared libraries) that we didn't handle
+ ''
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
if grep -qr '"lib[a-zA-Z0-9-]*\.so[\.0-9a-zA-z]*"' src; then
echo "Found unhandled dynamic library calls: "
grep -r '"lib[a-zA-Z0-9-]*\.so[\.0-9a-zA-z]*"' src
exit 1
fi
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
''
# Finally patch shebangs that might need patching.
# Should no longer be necessary with v250.
# https://github.com/systemd/systemd/pull/19638
+ ''
patchShebangs .
'';
outputs = [ "out" "man" "dev" ];
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
nativeBuildInputs =
[
pkg-config
gperf
ninja
meson
glibcLocales
getent
m4
intltool
gettext
libxslt
docbook_xsl
docbook_xml_dtd_42
docbook_xml_dtd_45
(buildPackages.python3Packages.python.withPackages (ps: with ps; [ lxml jinja2 ]))
]
++ lib.optional withLibBPF [
bpftools
llvmPackages.clang
llvmPackages.libllvm
]
;
2021-07-26 11:12:36 +00:00
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
buildInputs =
[
acl
audit
glib
kmod
libcap
libgcrypt
libidn2
libuuid
linuxHeaders
pam
]
++ lib.optional withApparmor libapparmor
++ lib.optional wantCurl (lib.getDev curl)
++ lib.optionals withCompression [ bzip2 lz4 xz ]
++ lib.optional withCryptsetup (lib.getDev cryptsetup.dev)
++ lib.optional withEfi gnu-efi
++ lib.optional withKexectools kexec-tools
++ lib.optional withLibseccomp libseccomp
++ lib.optional withNetworkd iptables
++ lib.optional withPCRE2 pcre2
++ lib.optional withResolved libgpg-error
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
++ lib.optional withSelinux libselinux
++ lib.optional withRemote libmicrohttpd
++ lib.optionals withHomed [ p11-kit ]
++ lib.optionals (withHomed || withCryptsetup) [ libfido2 ]
++ lib.optionals withLibBPF [ libbpf ]
++ lib.optional withTpm2Tss tpm2-tss
;
#dontAddPrefix = true;
mesonFlags = [
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
"-Dversion-tag=${version}"
"-Ddbuspolicydir=${placeholder "out"}/share/dbus-1/system.d"
2019-02-22 08:23:48 +00:00
"-Ddbussessionservicedir=${placeholder "out"}/share/dbus-1/services"
"-Ddbussystemservicedir=${placeholder "out"}/share/dbus-1/system-services"
"-Dpamconfdir=${placeholder "out"}/etc/pam.d"
"-Drootprefix=${placeholder "out"}"
"-Dpkgconfiglibdir=${placeholder "dev"}/lib/pkgconfig"
"-Dpkgconfigdatadir=${placeholder "dev"}/share/pkgconfig"
"-Dloadkeys-path=${kbd}/bin/loadkeys"
"-Dsetfont-path=${kbd}/bin/setfont"
"-Dtty-gid=3" # tty in NixOS has gid 3
"-Ddebug-shell=${bashInteractive}/bin/bash"
"-Dglib=${lib.boolToString (glib != null)}"
# while we do not run tests we should also not build them. Removes about 600 targets
"-Dtests=false"
2020-10-27 23:29:07 +00:00
"-Danalyze=${lib.boolToString withAnalyze}"
2020-10-27 23:29:34 +00:00
"-Dgcrypt=${lib.boolToString (libgcrypt != null)}"
"-Dimportd=${lib.boolToString withImportd}"
"-Dlz4=${lib.boolToString withCompression}"
2021-01-15 14:45:37 +00:00
"-Dhomed=${lib.boolToString withHomed}"
"-Dlogind=${lib.boolToString withLogind}"
"-Dlocaled=${lib.boolToString withLocaled}"
"-Dhostnamed=${lib.boolToString withHostnamed}"
2020-10-28 00:08:29 +00:00
"-Dmachined=${lib.boolToString withMachined}"
"-Dnetworkd=${lib.boolToString withNetworkd}"
2020-11-01 14:35:11 +00:00
"-Doomd=${lib.boolToString withOomd}"
2020-10-27 23:48:51 +00:00
"-Dpolkit=${lib.boolToString withPolkit}"
"-Dcryptsetup=${lib.boolToString withCryptsetup}"
"-Dportabled=${lib.boolToString withPortabled}"
"-Dhwdb=${lib.boolToString withHwdb}"
"-Dremote=${lib.boolToString withRemote}"
"-Dsysusers=false"
"-Dtimedated=${lib.boolToString withTimedated}"
"-Dtimesyncd=${lib.boolToString withTimesyncd}"
2020-10-28 00:00:44 +00:00
"-Duserdb=${lib.boolToString withUserDb}"
"-Dcoredump=${lib.boolToString withCoredump}"
"-Dfirstboot=false"
"-Dresolve=${lib.boolToString withResolved}"
"-Dsplit-usr=false"
"-Dlibcurl=${lib.boolToString wantCurl}"
"-Dlibidn=false"
"-Dlibidn2=true"
"-Dquotacheck=false"
"-Dldconfig=false"
"-Dsmack=true"
2019-02-22 08:23:48 +00:00
"-Db_pie=true"
2020-11-01 14:35:11 +00:00
"-Dinstall-sysconfdir=false"
"-Defi-ld=${stdenv.cc.bintools.targetPrefix}ld"
/*
2021-07-26 11:12:36 +00:00
As of now, systemd doesn't allow runtime configuration of these values. So
the settings in /etc/login.defs have no effect on it. Many people think this
should be supported however, see
- https://github.com/systemd/systemd/issues/3855
- https://github.com/systemd/systemd/issues/4850
- https://github.com/systemd/systemd/issues/9769
- https://github.com/systemd/systemd/issues/9843
- https://github.com/systemd/systemd/issues/10184
*/
"-Dsystem-uid-max=999"
"-Dsystem-gid-max=999"
# "-Dtime-epoch=1"
"-Dsysvinit-path="
"-Dsysvrcnd-path="
"-Dkill-path=${coreutils}/bin/kill"
"-Dkmod-path=${kmod}/bin/kmod"
2020-11-24 15:29:28 +00:00
"-Dsulogin-path=${util-linux}/bin/sulogin"
"-Dmount-path=${util-linux}/bin/mount"
"-Dumount-path=${util-linux}/bin/umount"
2019-09-03 23:35:49 +00:00
"-Dcreate-log-dirs=false"
# Use cgroupsv2. This is already the upstream default, but better be explicit.
"-Ddefault-hierarchy=unified"
2019-09-03 23:35:49 +00:00
# Upstream defaulted to disable manpages since they optimize for the much
# more frequent development builds
"-Dman=true"
"-Defi=${lib.boolToString withEfi}"
"-Dgnu-efi=${lib.boolToString withEfi}"
] ++ lib.optionals withEfi [
"-Defi-libdir=${toString gnu-efi}/lib"
"-Defi-includedir=${toString gnu-efi}/include/efi"
"-Defi-ldsdir=${toString gnu-efi}/lib"
2020-10-27 23:51:39 +00:00
] ++ lib.optionals (withShellCompletions == false) [
"-Dbashcompletiondir=no"
"-Dzshcompletiondir=no"
2020-10-28 00:05:14 +00:00
] ++ lib.optionals (!withNss) [
"-Dnss-myhostname=false"
"-Dnss-mymachines=false"
"-Dnss-resolve=false"
"-Dnss-systemd=false"
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
] ++ lib.optionals withLibBPF [
"-Dbpf-framework=true"
] ++ lib.optional withTpm2Tss "-Dtpm2=true";
preConfigure = ''
mesonFlagsArray+=(-Dntp-servers="0.nixos.pool.ntp.org 1.nixos.pool.ntp.org 2.nixos.pool.ntp.org 3.nixos.pool.ntp.org")
export LC_ALL="en_US.UTF-8";
# FIXME: patch this in systemd properly (and send upstream).
# already fixed in f00929ad622c978f8ad83590a15a765b4beecac9: (u)mount
for i in \
src/core/mount.c \
src/core/swap.c \
src/cryptsetup/cryptsetup-generator.c \
src/journal/cat.c \
src/nspawn/nspawn.c \
src/remount-fs/remount-fs.c \
src/shared/generator.c \
src/shutdown/shutdown.c \
units/emergency.service.in \
units/modprobe@.service \
units/rescue.service.in \
units/systemd-logind.service.in \
units/systemd-nspawn@.service.in; \
do
test -e $i
substituteInPlace $i \
--replace /usr/bin/getent ${getent}/bin/getent \
2020-11-24 15:29:28 +00:00
--replace /sbin/mkswap ${lib.getBin util-linux}/sbin/mkswap \
--replace /sbin/swapon ${lib.getBin util-linux}/sbin/swapon \
--replace /sbin/swapoff ${lib.getBin util-linux}/sbin/swapoff \
--replace /bin/echo ${coreutils}/bin/echo \
--replace /bin/cat ${coreutils}/bin/cat \
2020-11-24 15:29:28 +00:00
--replace /sbin/sulogin ${lib.getBin util-linux}/sbin/sulogin \
--replace /sbin/modprobe ${lib.getBin kmod}/sbin/modprobe \
--replace /usr/lib/systemd/systemd-fsck $out/lib/systemd/systemd-fsck \
--replace /bin/plymouth /run/current-system/sw/bin/plymouth # To avoid dependency
done
2020-11-01 14:35:11 +00:00
for dir in tools src/resolve test src/test src/shared; do
2019-02-22 08:23:48 +00:00
patchShebangs $dir
done
# absolute paths to gpg & tar
substituteInPlace src/import/pull-common.c \
--replace '"gpg"' '"${gnupg}/bin/gpg"'
for file in src/import/{{export,import,pull}-tar,import-common}.c; do
substituteInPlace $file \
--replace '"tar"' '"${gnutar}/bin/tar"'
done
systemd: 247.6 -> 249.4 This updates systemd to version v249.4 from version v247.6. Besides the many new features that can be found in the upstream repository they also introduced a bunch of cleanup which ended up requiring a few more patches on our side. a) 0022-core-Handle-lookup-paths-being-symlinks.patch: The way symlinked units were handled was changed in such that the last name of a unit file within one of the unit directories (/run/systemd/system, /etc/systemd/system, ...) is used as the name for the unit. Unfortunately that code didn't take into account that the unit directories themselves could already be symlinks and thus caused all our units to be recognized slightly different. There is an upstream PR for this new patch: https://github.com/systemd/systemd/pull/20479 b) The way the APIVFS is setup has been changed in such a way that we now always have /run. This required a few changes to the confinement tests which did assert that they didn't exist. Instead of adding another patch we can just adopt the upstream behavior. An empty /run doesn't seem harmful. As part of this work I refactored the confinement test just a little bit to allow better debugging of test failures. Previously it would just fail at some point and it wasn't obvious which of the many commands failed or what the unexpected string was. This should now be more obvious. c) Again related to the confinement tests the way a file was tested for being accessible was optimized. Previously systemd would in some situations open a file twice during that check. This was reduced to one operation but required the procfs to be mounted in a units namespace. An upstream bug was filed and fixed. We are now carrying the essential patch to fix that issue until it is backported to a new release (likely only version 250). The good part about this story is that upstream systemd now has a test case that looks very similar to one of our confinement tests. Hopefully that will lead to less friction in the long run. https://github.com/systemd/systemd/issues/20514 https://github.com/systemd/systemd/pull/20515 d) Previously we could grep for dlopen( somewhat reliably but now upstream started using a wrapper around dlopen that is most of the time used with linebreaks. This makes using grep not ergonomic anymore. With this bump we are grepping for anything that looks like a dynamic library name (in contrast to a dlopen(3) call) and replace those instead. That seems more robust. Time will tell if this holds. I tried using coccinelle to patch all those call sites using its tooling but unfornately it does stumble upon the _cleanup_ annotations that are very common in the systemd code. e) We now have some machinery for libbpf support in our systemd build. That being said it doesn't actually work as generating some skeletons doesn't work just yet. It fails with the below error message and is disabled by default (in both minimal and the regular build). > FAILED: src/core/bpf/socket_bind/socket-bind.skel.h > /build/source/tools/build-bpf-skel.py --clang_exec /nix/store/x1bi2mkapk1m0zq2g02nr018qyjkdn7a-clang-wrapper-12.0.1/bin/clang --llvm_strip_exec /nix/store/zm0kqan9qc77x219yihmmisi9g3sg8ns-llvm-12.0.1/bin/llvm-strip --bpftool_exec /nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool --arch x86_64 ../src/core/bpf/socket_bind/socket-bind.bpf.c src/core/bpf/socket_bind/socket-bind.skel.h > libbpf: elf: socket_bind_bpf is not a valid eBPF object file > Error: failed to open BPF object file: BPF object format invalid > Traceback (most recent call last): > File "/build/source/tools/build-bpf-skel.py", line 128, in <module> > bpf_build(args) > File "/build/source/tools/build-bpf-skel.py", line 92, in bpf_build > gen_bpf_skeleton(bpftool_exec=args.bpftool_exec, > File "/build/source/tools/build-bpf-skel.py", line 63, in gen_bpf_skeleton > skel = subprocess.check_output(bpftool_args, universal_newlines=True) > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 424, in check_output > return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, > File "/nix/store/81lwy2hfqj4c1943b1x8a0qsivjhdhw9-python3-3.9.6/lib/python3.9/subprocess.py", line 528, in run > raise CalledProcessError(retcode, process.args, > subprocess.CalledProcessError: Command '['/nix/store/l6dg8jlbh8qnqa58mshh3d8r6999dk0p-bpftools-5.13.11/bin/bpftool', 'g', 's', '../src/core/bpf/socket_bind/socket-bind.bpf.o']' returned non-zero exit status 255. > [102/1457] Compiling C object src/journal/libjournal-core.a.p/journald-server.c.oapture output)put)ut) > ninja: build stopped: subcommand failed. f) We do now have support for TPM2 based disk encryption in our systemd build. The actual bits and pieces to make use of that are missing but there are various ongoing efforts in that direction. There is also the story about systemd in our initrd to enable this being used for root volumes. None of this will yet work out of the box but we can start improving on that front. g) FIDO2 support was added systemd and consequently we can now use that. Just with TPM2 there hasn't been any integration work with NixOS and instead this just adds that capability to work on that. Co-Authored-By: Jörg Thalheim <joerg@thalheim.io>
2021-08-30 13:10:54 +00:00
substituteInPlace src/libsystemd/sd-journal/catalog.c \
--replace /usr/lib/systemd/catalog/ $out/lib/systemd/catalog/
'';
# These defines are overridden by CFLAGS and would trigger annoying
# warning messages
postConfigure = ''
substituteInPlace config.h \
--replace "POLKIT_AGENT_BINARY_PATH" "_POLKIT_AGENT_BINARY_PATH" \
--replace "SYSTEMD_BINARY_PATH" "_SYSTEMD_BINARY_PATH" \
--replace "SYSTEMD_CGROUP_AGENT_PATH" "_SYSTEMD_CGROUP_AGENT_PATH"
'';
2019-10-29 23:53:51 +00:00
NIX_CFLAGS_COMPILE = toString [
# Can't say ${polkit.bin}/bin/pkttyagent here because that would
# lead to a cyclic dependency.
2020-10-27 22:28:29 +00:00
"-UPOLKIT_AGENT_BINARY_PATH"
"-DPOLKIT_AGENT_BINARY_PATH=\"/run/current-system/sw/bin/pkttyagent\""
2019-10-29 23:53:51 +00:00
# Set the release_agent on /sys/fs/cgroup/systemd to the
# currently running systemd (/run/current-system/systemd) so
# that we don't use an obsolete/garbage-collected release agent.
2020-10-27 22:28:29 +00:00
"-USYSTEMD_CGROUP_AGENT_PATH"
"-DSYSTEMD_CGROUP_AGENT_PATH=\"/run/current-system/systemd/lib/systemd/systemd-cgroups-agent\""
2020-10-27 22:28:29 +00:00
"-USYSTEMD_BINARY_PATH"
"-DSYSTEMD_BINARY_PATH=\"/run/current-system/systemd/lib/systemd/systemd\""
2019-10-29 23:53:51 +00:00
];
doCheck = false; # fails a bunch of tests
# trigger the test -n "$DESTDIR" || mutate in upstreams build system
preInstall = ''
export DESTDIR=/
'';
postInstall = ''
# sysinit.target: Don't depend on
# systemd-tmpfiles-setup.service. This interferes with NixOps's
# send-keys feature (since sshd.service depends indirectly on
# sysinit.target).
mv $out/lib/systemd/system/sysinit.target.wants/systemd-tmpfiles-setup-dev.service $out/lib/systemd/system/multi-user.target.wants/
mkdir -p $out/example/systemd
mv $out/lib/{modules-load.d,binfmt.d,sysctl.d,tmpfiles.d} $out/example
mv $out/lib/systemd/{system,user} $out/example/systemd
rm -rf $out/etc/systemd/system
# Fix reference to /bin/false in the D-Bus services.
for i in $out/share/dbus-1/system-services/*.service; do
substituteInPlace $i --replace /bin/false ${coreutils}/bin/false
done
rm -rf $out/etc/rpm
# "kernel-install" shouldn't be used on NixOS.
find $out -name "*kernel-install*" -exec rm {} \;
2020-10-27 23:54:08 +00:00
'' + lib.optionalString (!withDocumentation) ''
rm -rf $out/share/doc
'';
# The interface version prevents NixOS from switching to an
# incompatible systemd at runtime. (Switching across reboots is
# fine, of course.) It should be increased whenever systemd changes
# in a backwards-incompatible way. If the interface version of two
# systemd builds is the same, then we can switch between them at
# runtime; otherwise we can't and we need to reboot.
passthru.interfaceVersion = 2;
meta = with lib; {
systemd: 243.4 -> 243.7 This bumps to the latest state of the systemd 242 stable, published at https://github.com/systemd/systemd-stable/tree/v243-stable. Should cover CVE-2020-1712. Git Log: f8dd0f2f15 (tag: v243.7, systemd-stable/v243-stable) Revert "Support Plugable UD-PRO8 dock" 1a5428c2ab hibernate-resume-generator: wait "infinitely" for the resume device eb3148c468 (tag: v243.6) hwdb: update to v245-rc1 f14fa558ae Fix typo in function name fb21e13e8e polkit: when authorizing via PK let's re-resolve callback/userdata instead of caching it 2e504c92d1 sd-bus: introduce API for re-enqueuing incoming messages 4d80c8f158 polkit: use structured initialization 54791aff01 polkit: on async pk requests, re-validate action/details 81532beddc polkit: reuse some common bus message appending code 4441844d58 bus-polkit: rename return error parameter to ret_error 31a1d569db shared: split out polkit stuff from bus-util.c → bus-polkit.c 560eb5babf test: adapt to the new capsh format 275b266bde meson: update efi path detection to gnu-efi-3.0.11 9239154545 presets: "disable" all passive targets by default a827c41851 shared/sysctl-util: normalize repeated slashes or dots to a single value fb1bfd6804 dhcp6: do not use T1 and T2 longer than one provided by the lease ca43a515c6 network: fix implicit type conversion warning by GCC-10 421eca7edf bootspec: parse random-seed-mode line in loader.conf 34e21fc6de sd-boot: fix typo df7b3a05c9 test: Synchronize journal before reading from it 9326efee71 sd-bus: fix introspection bug in signal parameter names 7bbdc56aaf efi: fix build. 486f8ca365 generator: order growfs for the root fs after systemd-remount-fs 56d442e29d loginctl: use /org/freedesktop/login1/session/auto when "lock-session" is called without argument 6ed1152282 Documentation update for x-systemd.{before,after} dba3efa34a man: fix typo in systemd.netdev Xfrm example 6f9a8621d8 timesyncd: log louder when we refuse a server due to root distance 0637255d3b resolved: drop DNSSEC root key that is not valid anymore 9a135baa40 journal: don't use startswith() on something that is not a NUL-terminated string 1ff3972a0f test: add test for https://github.com/systemd/systemd/issues/14560 cac79b606b core: make sure StandardInput=file: doesn't get dup'ed to stdout/stderr by default 906ba9a67d pkgconf: add full generator paths 01b93e2c68 tree-wide: we forgot to destroy some bus errors 5c9455657e mount: make checks on perpetual mount units more lax 28c58beca1 core: never allow perpetual units to be masked d3b044b3e7 typo: "May modify to" -> "May modify" fd378d3d3c sysctl: downgrade message when we have no permission db4fbf5c61 Clarify journald.conf MaxLevelStore documentation c8365f71c0 logind: refuse overriding idle hint on tty sessions cd91f567b6 cgroup: update only siblings that got realized once c672dcd212 mount: mark an existing "mounting" unit from /proc/self/mountinfo as "just_mounted" a592a40564 journalctl: Correctly handle combination of --reverse and --lines (fixes #1596) 0aa144ab1d journalctl: Correctly handle --show-cursor in combination with --until or --since and --reverse 3b803a5e66 core: fix re-realization of cgroup siblings 7549dd40fc core: propagate service state to socket in more load states af6df343b2 man: describe "symlink" and "systemctl link" explicitly in UNIT FILE LOAD PATH a3c1ce25a7 core: be more restrictive on the dependency types we allow to be created transiently 2b9ec8384c udev: don't import parent ID_FS_ data on partitions ecd95c507c man: fix option name 0d4f06156b Support Plugable UD-PRO8 dock 7fba869abd gpt-auto: don't assume XBOOTLDR is vfat 494c281b67 man: fix documentation of IBM VIO device naming 7271fb056a man: slightly extend documentation on difference between ID_NET_NAME_ONBOARD and ID_NET_LABEL_ONBOARD 852ae28e68 boot: fix osrel parser 2613200370 udev: do not use exact match of file permission 46477397c1 network: lower the log-level of harmless message 7163b1fe86 hwdb: ignore keys added in kernel 5.5 92f90837dc systemctl: skip non-existent units in the 'cat' verb a67227cc99 systemd.exec: document the file system for EnvironmentFile paths cfb4c0aca5 systemd-analyze: fixed typo in documentation 017fddd998 test-condition: fix group check condition 9d5e3cb774 umount: show correct error message 252f1a5277 Revert "Drop dbus activation stub service" 20bbfac95e man: add section about user manager units c93ef60212 man: add remote-*.targets to the bootup sequence 55e0f99689 time-util: also use 32bit hack on EOVERFLOW 7afe2ecb02 [man] note which UID ranges will get user journals a43b67a4c9 [man] fix URL dedb26a8d6 analyze: badness if neither of RootImage and RootDirectory exists 714c93862a initrd: make udev cleanup service confict trigger and settle too 8932407ae1 man: we support growing xfs too these days 19af11dc07 time-util: deal with systems where userspace has 64bit time_t but kernel does not c90229d81d [import] fix stdin/stdout pipe behavior in import/export tar/raw 39910328da cryptsetup-generator: unconfuse writing of the device timeout fc5e6c87a4 shared/install: log syntax error for invalid DefaultInstance= 409c94a407 shared/install: provide a nicer error message for invalid WantedBy=/Required= values 70e8c1978a seccomp: real syscall numbers are >= 0 a0a1977d9a seccomp: more comprehensive protection against libseccomp's __NR_xyz namespace invasion 7f936c60d5 network: set ipv6 mtu after link-up or device mtu change b59d88cc62 man: fix typo in net-naming-scheme man page c5e5ac0958 man: fix typos (#14304) 9a2f26564d ipv4ll: do not reset conflict counter on restart bc9e1ebfdd Fix typo (duplicate "or") c6cb71b7e7 network: if /sys is rw, then udev should be around 67dcdfd956 nspawn: do not fail if udev is not running a7938a1bc6 Create parent directories when creating systemd-private subdirs 53aa44f873 network: do not return error but return UINT64_MAX if speed meter is disabled 65abf12674 core: swap priority can be negative b1cf452ff5 systemctl: enhance message about kexec missing kernel 07a0e5b425 man: use mkswap@ instead of makeswap@ 57dc017c6b journald: don't ask for the machine ID if we don't need it ac392a57c0 journalctl: pager_close() calls fflush(stdout) anyway as first thing ee7dfadc82 journald: remove unused field 471073f1b5 journalctl: return EOPNOTSUPP if pcre is not enabled 002ededb61 man: drop reference to machined, add one for journald instead fd3bd4be3b pid1: make TimeoutAbortSec settable for transient units eb2ef4d664 pid1: fix setting of DefaultTimeoutAbortSec 1d75e29b23 shared/ask-password-api: modify keyctl break value a16b1ee7e5 cryptsetup: reduce the chance that we will be OOM killed 4836fb010a core: write out correct field name when creating transient service units 3e2c547f6d udevd: don't use monitor after manager_exit() d42f7d45a8 Revert "udevd: fix crash when workers time out after exit is signal caught" c9a287eee8 man/systemd.link: Add missing verb *be* a67a3ae04b man: document all pager variables for systemctl and systemd 3a8fce3f38 core.timer: fix "systemd-analyze dump" and docs syntax inconsistencies wrt OnTimezoneChange= fdffd284b6 core/service: downgrade "scheduling restart" message to debug 733e7f19d3 travis: add missing closing quote sign 0d7b7817fc systemd-tmpfiles: don't install timer when service isn't installed either 0e7f83cd2b pam_systemd: prolong method call timeout when allocating session
2020-02-18 20:46:40 +00:00
homepage = "https://www.freedesktop.org/wiki/Software/systemd/";
description = "A system and service manager for Linux";
2018-08-17 22:32:36 +00:00
license = licenses.lgpl21Plus;
platforms = platforms.linux;
priority = 10;
maintainers = with maintainers; [ andir eelco flokli kloenk ];
};
}