Hooks are essentially implemented as special shell packages that run on
their respective host platform. When they are used, they appear as
nativeBuildInputs (as they need to be executed as part of the build of a
package using them) so are taken from buildPackages relative to the
derivation using them.
Since the override in buildNpmPackage nullifies splicing, we take
npmHooks from buildPackages manually.
Fixes pkgsCross.ghcjs.buildPackages.emscripten and thus
pkgsCross.ghcjs.haskellPackages.ghc.
The implicit contract of buildFHSUserEnv was that it allows to run
software built for a typical GNU/Linux distribution (not NixOS) without
patching it (patchelf, autoPatchelfHook, etc.). Note that this does not
inherently imply running untrusted programs.
buildFHSUserEnv was implemented by using chroot and assembling a
standard-compliant FHS environment in the new root. As expected, this
did not provide any kind of isolation between the system and the
programs.
However, when it was later reimplemented using bubblewrap
(PR #225748), which *is* a security tool, several isolation features
involving detaches Linux namespaces were turned on by default.
This decision has introduced a number of breakages that are very
difficult to debug and trace back to this change.
For example: `unshareIPC` breaks software audio mixing in programs using
ALSA (dmix) and `unsharePID` breaks gdb,
Since:
1. the security features were enable without any clear threat model;
2. `buildFHSEnvBubblewrap` is supposed to be a drop-in replacement of
`buildFHSEnvChrootenv` (see the release notes for NixOS 23.05);
3. the change is breaking in several common cases (security does not
come for free);
4. the contract was not changed, or at least communicated in a clear
way to the users;
all security features should be turned off by default.
P.S. It would be useful to create a variant of buildFHSEnv that does
provide some isolation. This could unshare some namespaces and mount
only limited parts of the filesystem.
Note that buildFHSEnv mounts every directory in / under the new root, so
again, very little is gained by unsharing alone.
I do not think there is a good reason for it to be optional.
There were only two packages without a version attr. The version attr
is added to them in this patch.
`$OBJCOPY` is not available in bootstrap tools
`stdenv.__bootPackages.stdenv.__bootPackages.stdenv.__bootPackages.stdenv.__bootPackages.stdenv.__bootPackages.bash.stdenv.cc.bintools`
Without this, pkgsStatic.pkgsLLVM.hello fails with segfaulting binaries
because of the issue described at [0].
In summary, llvm's linker has a different behaviour to GCC's when
supplied with both -static and -Wl,-dynamic-linker=...; GCC copes with
it, but LLVM produces a binary which segfaults on startup. It appears to
be necessary to omit the dynamic linker in this case.
nixpkgs' static adaptor passes -static via NIX_CFLAGS_LINK which was not
accounted for prior to this commit in the checkLinkType logic. For good
measure I put the other NIX_ flags affecting link in the same logic.
Additionally, $NIX_CFLAGS_LINK_@suffixSalt@ is not available until later
than it was originally set, so set $linkType close to its point of use.
I checked for earlier uses by studying the shell trace output and
couldn't find any.
[0] https://github.com/NixOS/nixpkgs/issues/111010#issuecomment-1536424163
Signed-off-by: Peter Waller <p@pwaller.net>
The first log in *Message* before this patch:
Loading /nix/store/bikm18vy6v07hmrvrll501i68440w9iw-emacs-29.1-rc1/share/emacs/site-lisp/site-start.el (source)...done
and after this patch:
Loading /nix/store/bikm18vy6v07hmrvrll501i68440w9iw-emacs-29.1-rc1/share/emacs/site-lisp/site-start (native compiled elisp)...done
I see no reason to symlink this dir.
Doing so may shadow unwanted libraries since the site-start.el of
Emacs adds paths under NIX_PROFILES to load-path.
It is added in [1] to fix "building emacs". However, I have no issue
in building and using Emacs after removing it.
[1]: https://github.com/NixOS/nixpkgs/pull/89351
"$out/share/emacs/site-lisp" is added to load-path in wrapper.sh[1]
using EMACSLOADPATH and "$out/share/emacs/native-lisp/" is added to
native-comp-eln-load-path in wrapper.sh[2] using
EMACSNATIVELOADPATH. There is no point to add them again here.
Additionally, the trailing "/" in "$out/share/emacs/native-lisp/"
causes duplicate entries in native-comp-eln-load-path:
("/nix/store/hash1-emacs-packages-deps/share/emacs/native-lisp/" ; [3]
"/home/user/.emacs.d/eln-cache/"
"/nix/store/hash1-emacs-packages-deps/share/emacs/native-lisp" ; [2]
"/nix/store/hash2-emacs-29.1-rc1/lib/emacs/29.1/native-lisp/")
load-path does not change with this patch applied.
[1]: 1476c13422/pkgs/build-support/emacs/wrapper.sh (L47)
[2]: 1476c13422/pkgs/build-support/emacs/wrapper.sh (L50)
[3]: 1476c13422/pkgs/build-support/emacs/wrapper.nix (L166)
Add this hook to checkPhase to allow for running MPI application in
the sandbox. It detects the MPI implementations and sets the respective
environment variables.
Before the change the following command did not work:
$ nix develop -i --impure --expr 'with import <nixpkgs> { system = "i686-linux"; }; (buildFHSUserEnv { name = "t"; targetPkgs = ps: with ps; [ libmpc stdenv.cc ]; }).env'
$ ld -lmpc -o a
ld: cannot find -lmpc: No such file or directory
It is expected to work as `NIX_LDFLAGS` does contain valid values:
$ echo $NIX_LDFLAGS
-L/usr/lib -L/usr/lib32
Note that for `gcc` it does work:
$ printf "int main(){}" | gcc -x c - -lmpc -o a
It happens because `HOST` role is enabled for `cc`:
$ echo $NIX_CC_WRAPPER_TARGET_HOST_i686_unknown_linux_gnu
1
But not for `BINTOOLS`:
$ echo $NIX_BINTOOLS_WRAPPER_TARGET_HOST_i686_unknown_linux_gnu
<empty>
The change adds BINTOOLS role and fixes linking:
$ nix develop -i --impure --expr 'with import ~/nm { system = "i686-linux"; }; (buildFHSUserEnv { name = "t"; targetPkgs = ps: with ps; [ libmpc stdenv.cc ]; }).env'
$ printf "int main(){}" | gcc -x c - -lmpc -o a
$ ld -lmpc -o a
ld: warning: cannot find entry symbol _start; not setting start address
Previously `writeDarwinBundle` used a handcrafted shell wrapper, however
this causes issues on Apple Silicon Macs as script-only application
bundles are always run under Rosetta[0][1].
Replacing the handcrafted shell wrapper with a binary wrapper allows
apps to run natively instead of requiring Rosetta. However, this means
we can no longer use `$1` and `$@`.
After checking nearly every current usage of `desktopToDarwinBundle`,
there were no apps that used `%[fFuU]` before the last argument, meaning
removing them naively is good enough for the current apps.
[0]: https://developer.apple.com/documentation/apple-silicon/building-a-universal-macos-binary
[1]: https://stackoverflow.com/a/68208374
The `name` & `version` attributes only existed in a previous form of
the API before it was switched over to using `fetchzip` &
`applyPatches`[1]. The attributes existed to be able to throw an
evaluation error with upgrade instructions when this was used. However,
this was before 22.11, so this shouldn't be in use anymore (and if
somebody is doing a migration from a very old NixOS, this is still
documented in the 22.11 release-notes[2]).
Anyways, this simplifies the code a little bit and also having both
`appName`/`appVersion` and `name`/`version` in there is quite confusing. But
still, I figured it's less confusing to not re-use attributes that were
deprecated in the past, hence the alternative naming.
[1] 3ca9b9a8ad
[2] d41b381310
This change makes sure that each Nextcloud plugin installed provides a
`meta` section with proper license information.
Unfortunately, the metadata from the appstore is useless for this
purpose since it doesn't differentiate between e.g. AGPL 3.x and AGPL
3.x or any later version. In fact, this isn't consistent between their
software, e.g. `bookmarks` has `agpl3Plus` according to the files'
headers[1] whereas `twofactor_nextcloud_notification` is AGPL 3 only[2].
I don't think there's any trivial and reasonable way to retrieve this
information programatically, so I decided to change the format of
`nextcloud-apps.json`[3] to also contain the license in the form of the
license attribute we have in `lib/licenses.nix`, i.e. GNU AGPL 3 or
later is `agpl3Plus`.
I retrieved the information using the following approach:
* go to the source code of $app at the revision we currently have
packaged
* check for a license identifier (does it specify the license only or
the license "or any later version")?
* first in `src/main.js` because most apps from Nextcloud itself used
actual spdx identifiers in the frontend's source-code.
* then in `lib/AppInfo/Application.php` which each Nextcloud app has.
License changes should be updated accordingly when updating the apps. As
with any other package in nixpkgs as well, this currently needs to be
done manually (or as part of the review process)[4].
Also, I decided to change the `name` of the `applyPatches` derivation
from `source-patched` to `nextcloud-app-${appName}-${appVersion}`. When
deploying a lot of apps (and probably displaying the diff using
`nix store diff-closures` on deploy), the current output isn't very
helpful. This is purely optional because I didn't want to break the
interface of `fetchNextcloudApp` again.
[1] https://github.com/nextcloud/bookmarks/blob/v13.1.0/lib/AppInfo/Application.php#L6
[2] https://github.com/nextcloud/twofactor_nextcloud_notification/blob/v3.7.0/lib/AppInfo/Application.php
[3] This isn't really well-defined since it's preprocessed with `jq(1)`
before passing the apps to nc4nix.
[4] Though notable license changes (e.g. agpl -> gpl) would also pop up
in the diff of <nextcloudversion>.json, so this is pretty easy to
catch.
The command `fakechroot` errored with buffer overflows. The `proot`
command doesn't seem to suffer from the same problem. The tar command
creating the layer errors with "permission denied" on a bunch of paths
in /proc but the layer seems to get built anyway.
this splits hardeningCFlags into hardeningCFlagsAfter and
hardeningCFlagsBefore (where most flags still remain) to allow
us to *append* `-D_FORTIFY_SOURCE=` values to the command-line,
forcing our choice of fortify level and avoiding potential
redefinition warnings/errors through use of `-U_FORTIFY_SOURCE`
While using libredirect in conjunction with geckodriver, I stumbled on
odd segfaults that happened when running the wrapped statx() call from
libredirect:
0x00007ffff7ddd541 in __strncmp_avx2 () from .../lib/libc.so.6
0x00007ffff7f6fe57 in statx () from .../lib/libredirect.so
0x00005555558d35bd in std::sys::unix::fs::try_statx::h2045d39b0c66d4e8 ()
0x00005555558d2230 in std::sys::unix::fs::stat::ha063998dfb361520 ()
0x0000555555714019 in mozversion::firefox_version::hdc3b57eb04947426 ()
0x00005555556a603c in geckodriver::capabilities::FirefoxCapabilities::version::h58e289917bd3c721 ()
0x00005555556a77f5 in <geckodriver::capabilities::FirefoxCapabilities as webdriver::capabilities::BrowserCapabilities>::validate_custom::h62d23cf9fd63b719 ()
0x000055555562a7c8 in webdriver::capabilities::SpecNewSessionParameters::validate::h60da250d33f0989f ()
0x00005555556d7a13 in <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold::h9427a360a3d0bf8f ()
0x0000555555669d85 in <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter::hd274d536ea29bb33 ()
0x00005555555c05ef in core::iter::adapters::try_process::hdf96a01ec1f9b8bd ()
0x000055555561768d in <webdriver::capabilities::SpecNewSessionParameters as webdriver::capabilities::CapabilitiesMatching>::match_browser::hfbd8c38f6db17e9f ()
0x00005555555ca6ef in <geckodriver::marionette::MarionetteHandler as webdriver::server::WebDriverHandler<geckodriver::command::GeckoExtensionRoute>>::handle_command::h13b98b9cb87a69d6 ()
0x00005555555e859e in webdriver::server::Dispatcher<T,U>::run::h746a8bf2f0bc24fd ()
0x000055555569ff0f in std::sys_common::backtrace::__rust_begin_short_backtrace::h3b920773bd467d2a ()
0x00005555555dbc99 in core::ops::function::FnOnce::call_once{{vtable.shim}}::h81ba7228877515f7 ()
0x00005555558d31a3 in std::sys::unix:🧵:Thread:🆕:thread_start::h4514580219a899c5 ()
0x00007ffff7d0ce24 in start_thread () from .../lib/libc.so.6
0x00007ffff7d8e9b0 in clone3 () from .../lib/libc.so.6
The reason why I found this odd was because it happens in the following
piece of code (shortened a bit):
1 static const char * rewrite(const char * path, char * buf)
2 {
3 if (path == NULL) return path;
4 for (int n = 0; n < nrRedirects; ++n) {
5 int len = strlen(from[n]);
6 if (strncmp(path, from[n], len) != 0) continue;
7 if (snprintf(buf, PATH_MAX, "%s%s", to[n], path + len) >= PATH_MAX)
8 abort();
9 return buf;
10 }
11 return path;
12 }
When inspecting the assembly, I found that the check for the null
pointer in line 3 was completely missing and the code was directly
entering the loop and then eventually segfault when running strncmp()
with a null pointer as its first argument.
I confirmed that indeed that check was missing by compiling libredirect
with "-O0" and comparing the generated assembly with the optimized one.
The one compiled with "-O0" had that check while the optimized one did
not and indeed when running geckodriver with the unoptimized version it
worked fine.
Digging in the Git history, I found 5677ce2008,
which actually introduced the null pointer check. Going back to that
commit however, the check actually was still in the generated assembly.
So I bisected between that commit and the most recent one and ended up
with commit ca8aa5dc87, which moved
everything to use GCC 7.
I haven't found out why *exactly* GCC was optimizing the check away, but
playing around on Godbolt with various other compilers seems that other
compilers such as Clang are doing it as well. Additionally, given that
passing NULL to stat() is UB, my guess is that compilers tend to assume
that such an argument can't be NULL. My assumption is based on the fact
that GCC warns with "argument 1 null where non-null expected" when
passing NULL to eg. stat().
To address this for now, I marked the path argument of the rewrite()
volatile and also added a test that should cause a segfault in case this
would regress again as it already did.
Signed-off-by: aszlig <aszlig@nix.build>
Since coreutils v9.2 the `--no-clobber` flag results in a non-zero exit
code when the destination files exist. Using `--update=none` will now
reproduce the old behavior of `--no-clobber`.
However, the `--update=none` flag was introduced in coreutils v9.3 and
thus `mergeImages` will fail if you have an older version than v9.3 in
stdenv after applying this commit.
[coreutils v9.3 changelog](f386722dc0/NEWS (L48))
Derivations built with `writeShellScriptBin`
should always be runnable with `nix run`. At present,
the derivation is missing both `meta.mainProgram`
and `pname`– this means that `nix run` falls back
to inferring the bin path from `name`. This is
unreliable and depends on faulty heuristics.
For context, reference the following snippet from
`nix run --help`:
If installable evaluates to a derivation, it will try to execute the
program <out>/bin/<name>, where out is the primary output store path
of the derivation, and name is the first of the following that exists:
· The meta.mainProgram attribute of the derivation.
· The pname attribute of the derivation.
· The name part of the value of the name attribute of the derivation.
This is very useful in conjunction with meta.pkgConfigModules, as the
new tester can use the list provided by this meta attribute as a default
value for moduleNames, making its usage in passthru.tests very
convenient.
For backwards compatibility, a shim under the old name is maintained
with a warning.
vcunat said
> This invocation of mktemp creates the file in the current directory, which is bad practice. We should add "--tmpdir=$TMPDIR" or make the template absolute.
> I noticed because one package did cd $src during installing, which is a read-only path...
The Darwin stdenv rework conditionally sets `NIX_CC_USE_RESPONSE_FILE`
depending on the `ARG_MAX` of the build system. If it is at least 1 MiB,
the stdenv passes the arguments on the command-line (like Linux).
Otherwise, it falls back to the response file. This was done to prevent
intermitent failures with clang 16 being unable to read the response
file. Unfortunately, this breaks `gccStdenv` on older Darwin platforms.
Note: While the stdenv logic will also be reverted, this change is
needed for compatibility with clang 16.
GCC is capable of using a response file, but it does not work correctly
when the response file is a file descriptor. This can be reproduced
using the following sequence of commands:
$ nix shell nixpkgs#gcc; NIX_CC_USE_RESPONSE_FILE=1 gcc
# Linux
/nix/store/9n9gjvzci75gp2sh1c4rh626dhizqynl-binutils-2.39/bin/ld: unrecognized option '-B/nix/store/vnwdak3n1w2jjil119j65k8mw1z23p84-glibc-2.35-224/lib/'
/nix/store/9n9gjvzci75gp2sh1c4rh626dhizqynl-binutils-2.39/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
# Darwin
ld: unknown option: -mmacosx-version-min=11.0
collect2: error: ld returned 1 exit status
Instead of using process substitution, create a temporary file and
remove it in a trap. This should also prevent the intermitent build
failures with clang 16 on older Darwin systems.
Fixes#245167
Before the change the hook had a chance to run `strip` against the same
file using multiple link paths. In case of `gcc` `libgcc.a` was stripped
multiple times in parallel and produces corrupted archive.
The change runs inputs via `realpath | uniq` to make sure we don't
attempt to strip the same files multiple times.
* Preferentially use the stdenv clang if it is new enough to produce
arm64e binaries; and
* Fix incompatible function pointer conversions (results in an error
with clang 16).
In 787af0f79f
I had to change ${go-modules} to $goModules to allow overrideAttrs to work;
However, env vars cannot contain -, so i had to change go-modules too.
This in turn broke nix-update because it uses the go-modules attr.
Instead of making nix-update more complicated, make go-modules naming match cargoDeps.
`fd --type f | xargs sd '\bgo-modules\b' 'goModules'`
and revert change to pkgs/applications/misc/dstask/default.nix
and pkgs/servers/http/dave/default.nix
and pkgs/os-specific/darwin/plistwatch/default.nix
release note added
While trying to exclude a generated `/share/info/dir` in a package
from symlinking into a profile, we also ended up excluding the Emacs
info manual for `dired-x`. This change excludes only the dir file.
Most FHSEnv-wrapped packages in Nixpkgs wrap a x86_64-linux binary; making
multiArch unnecessary bloat closure size. Saves about 1G in anki-bin.
This makes multiArch FHSEnvs the exception rather than the rule.
; Conflicts:
; pkgs/development/libraries/nss/generic.nix
- bb53634671 removed the conditionals since firefox-esr-91 and nss <3.69 has been dropped a while ago.
- cb3762857d updated the conditionals to always partition the tests based on the nss_latest version. Chosen that since it will remain future proof.
