So in php we have a bootstrapping composer (that lives in
composer-phar.nix) that downloads the binary distribution of composer
to be able to bootstrap and build a composer built from source.
However, it's desirable to keep composer updated at all time, and this
includes the phar one that isn't used by users directly. So this makes
sure we don't "forget" about the phar one since it now borrows the
version from the source version. However, then it also made sense to
move the hash for the phar to the same file as the source
composer. Then we can control the full upgrade life cycle of both the
phar and source versions of composer from the same file.
It's useful to have access to these attributes from packages built with
buildFHSEnvBubblewrap, and it reduces the difference between FHS and
non-FHS packages.
'name' is already handled by runCommandLocal.
This fixes a regression introduced in https://github.com/NixOS/nixpkgs/pull/256628
which broke fetching with private = true through a netrc file.
Tested locally with a really special github enterprise.
The commit 6f2b3ba027 introduced a
`mktemp` invokation that uses the `--tmpdir` flag, which is not
available on MacOS.
This changes the invokation to a portable one based on the following
StackOverflow answer https://stackoverflow.com/a/31397073/841562 .
Derivations affected by this patch set `__structuredAttrs = true;` and
provide their own `builder`, i.e. it's necessary to `source .attrs.sh`.
Rather than adding even more `if`-`source` monstrums, I decided to
modify all of those derivations to use `buildCommand` or `runCommand`,
without `builder` being set.
Then, `$stdenv/setup` is sourced already and as a result it's safe to
assume that `NIX_ATTRS_JSON_FILE`/`NIX_ATTRS_SH_FILE` point to a usable
location both in a build and a shell session.
When specifying the `builder` attribute in `stdenv.mkDerivation`, this
will be effectively transformed into
builtins.derivation {
builder = stdenv.shell;
args = [ "-e" builder ];
}
This also means that `default-builder.sh` is never sourced and as a
result it's not guaranteed that `$NIX_ATTRS_SH_FILE` is set to a correct
location[1].
Also, we need to source `.attrs.sh` to source `$stdenv`. So, the
following is done now:
* If `$NIX_ATTRS_SH_FILE` points to a correct location, then use it.
Directly using `.attrs.sh` is problematic for `nix-shell(1)` usage
(see previous commit for more context), so prefer the environment
variable if possible.
* Otherwise, if `.attrs.sh` exists, then use it. See [1] for when this
can happen.
* If neither applies, it can be assumed that `__structuredAttrs` is
turned off and thus nothing needs to be done.
[1] It's possible that it doesn't exist at all - in case of Nix 2.3 or
it can point to a wrong location on older Nix versions with a bug in
`__structuredAttrs`.
> If using a target spec JSON file, the <triple> value is the filename
> stem. For example --target foo/bar.json would match [target.bar].
- https://doc.rust-lang.org/cargo/reference/config.html#target
I've also exposed toRustTargetSpecShort as a public function, because
it's useful to be able to know what the target subdirectory will be.
This function helps building an OCaml package that builds with topkg.
There are currently many such packages in nixpkgs and this function
would greatly simplify adding more.
This is heavily inspired by `ocamlPackages.buildDunePackage`.
This initially may look like a downgrade, but this is caused by how
upstream is tagging versions.
Before they would have the GraalVM having its own version (e.g. 22.3.1),
and each version would support multiple JVM versions (e.g. 11, 17, 19).
Now each release supports only one JVM version (e.g.: 21), and they
track the same version as the JVM.
They also changed packaging, making all sub-products (e.g.: GraalPy,
GraalRuby, etc.) standalone, so they do not depend in GraalVM anymore
and have their own version. Thanks to this change, we will need to
repackage everything.
To simplify, this commit will remove all sub-products and only care
about the GraalVM/Native Image (that is back to GraalVM itself) part.
Other commits will re-added each sub-product.
Fix (partial): https://github.com/NixOS/nixpkgs/issues/257292
Currently there is a state of severe confusion in
pkgs/build-support/rust/hooks/ regarding host vs target; right now
there is only "host" defined, but whether it means "host" or
"target" seems to fluctuate.
This commit corrects that, ensuring that all variables come in all
three flavors (build, host, target) and are used consistently with
the nixpkgs convention.
This also fixes the cross-compilation of packages which use
`maturinBuildHook` -- hooks go in `nativeBuildInputs` and are
phase-shifted backwards by one platform, so they need to be careful
about distinguishing between build and host.
Closes#247441
`buildFHSEnvBubblewrap { pname = ...; }` currently results in eval error
because args.name doesn't exist then. Fix it by only using args.name if
it exists.
It's useful to be able to introspect all packages which are available
in the fhsenv. I've renamed basePkgs and baseMultiPkgs to be
consistent with the naming scheme used for the bits that were
previously public — names ending in "Pkgs" are for functions, and
names ending in "Paths" are the results of those functions.
nix-prefetch-git is either run as part of a build, usually sandboxed,
or outside a build, unsandboxed, to prefetch something that will later
be used in a build. It's important that the latter use produces
hashes that can be reproduced by the former.
One way that they can differ is if the user's git config does
something that changes the result of git clone. I ran into this,
because my global git config automatically enables git-lfs, whereas
nix-prefetch-git otherwise only uses git-lfs if specifically
requested. This led to very confusing hash mismatches.
Removed workaround from llvm 16.
Fixes including cxxabi.h on llvm >=15 libcxxStdenv.
```c
int main() {}
```
```
/nix/store/qwnvng0cbyx0bijm654jpmpl0516hfhx-libcxxabi-15.0.7-dev/include/cxxabi.h:20:10: fatal error: '__cxxabi_config.h' file not found
```
Before llvm 15 this used to work because `libcxx` copied the headers
from `cxxabi` to it's own `include`, which was then picked up by the
line above this one
Alternative fix would be to copy all files from `${cxxabi.dev}/include/c++/v1` to `${cxxabi.dev}/include` so the cc-wrapper setup hook would pick them up, but that would depend on in cxxabi being in buildInputs.
Signed-off-by: Amneesh Singh <natto@weirdnatto.in>
Hooks are essentially implemented as special shell packages that run on
their respective host platform. When they are used, they appear as
nativeBuildInputs (as they need to be executed as part of the build of a
package using them) so are taken from buildPackages relative to the
derivation using them.
Since the override in buildNpmPackage nullifies splicing, we take
npmHooks from buildPackages manually.
Fixes pkgsCross.ghcjs.buildPackages.emscripten and thus
pkgsCross.ghcjs.haskellPackages.ghc.
The implicit contract of buildFHSUserEnv was that it allows to run
software built for a typical GNU/Linux distribution (not NixOS) without
patching it (patchelf, autoPatchelfHook, etc.). Note that this does not
inherently imply running untrusted programs.
buildFHSUserEnv was implemented by using chroot and assembling a
standard-compliant FHS environment in the new root. As expected, this
did not provide any kind of isolation between the system and the
programs.
However, when it was later reimplemented using bubblewrap
(PR #225748), which *is* a security tool, several isolation features
involving detaches Linux namespaces were turned on by default.
This decision has introduced a number of breakages that are very
difficult to debug and trace back to this change.
For example: `unshareIPC` breaks software audio mixing in programs using
ALSA (dmix) and `unsharePID` breaks gdb,
Since:
1. the security features were enable without any clear threat model;
2. `buildFHSEnvBubblewrap` is supposed to be a drop-in replacement of
`buildFHSEnvChrootenv` (see the release notes for NixOS 23.05);
3. the change is breaking in several common cases (security does not
come for free);
4. the contract was not changed, or at least communicated in a clear
way to the users;
all security features should be turned off by default.
P.S. It would be useful to create a variant of buildFHSEnv that does
provide some isolation. This could unshare some namespaces and mount
only limited parts of the filesystem.
Note that buildFHSEnv mounts every directory in / under the new root, so
again, very little is gained by unsharing alone.
I do not think there is a good reason for it to be optional.
There were only two packages without a version attr. The version attr
is added to them in this patch.
`$OBJCOPY` is not available in bootstrap tools
`stdenv.__bootPackages.stdenv.__bootPackages.stdenv.__bootPackages.stdenv.__bootPackages.stdenv.__bootPackages.bash.stdenv.cc.bintools`
Without this, pkgsStatic.pkgsLLVM.hello fails with segfaulting binaries
because of the issue described at [0].
In summary, llvm's linker has a different behaviour to GCC's when
supplied with both -static and -Wl,-dynamic-linker=...; GCC copes with
it, but LLVM produces a binary which segfaults on startup. It appears to
be necessary to omit the dynamic linker in this case.
nixpkgs' static adaptor passes -static via NIX_CFLAGS_LINK which was not
accounted for prior to this commit in the checkLinkType logic. For good
measure I put the other NIX_ flags affecting link in the same logic.
Additionally, $NIX_CFLAGS_LINK_@suffixSalt@ is not available until later
than it was originally set, so set $linkType close to its point of use.
I checked for earlier uses by studying the shell trace output and
couldn't find any.
[0] https://github.com/NixOS/nixpkgs/issues/111010#issuecomment-1536424163
Signed-off-by: Peter Waller <p@pwaller.net>
The first log in *Message* before this patch:
Loading /nix/store/bikm18vy6v07hmrvrll501i68440w9iw-emacs-29.1-rc1/share/emacs/site-lisp/site-start.el (source)...done
and after this patch:
Loading /nix/store/bikm18vy6v07hmrvrll501i68440w9iw-emacs-29.1-rc1/share/emacs/site-lisp/site-start (native compiled elisp)...done
I see no reason to symlink this dir.
Doing so may shadow unwanted libraries since the site-start.el of
Emacs adds paths under NIX_PROFILES to load-path.
It is added in [1] to fix "building emacs". However, I have no issue
in building and using Emacs after removing it.
[1]: https://github.com/NixOS/nixpkgs/pull/89351
"$out/share/emacs/site-lisp" is added to load-path in wrapper.sh[1]
using EMACSLOADPATH and "$out/share/emacs/native-lisp/" is added to
native-comp-eln-load-path in wrapper.sh[2] using
EMACSNATIVELOADPATH. There is no point to add them again here.
Additionally, the trailing "/" in "$out/share/emacs/native-lisp/"
causes duplicate entries in native-comp-eln-load-path:
("/nix/store/hash1-emacs-packages-deps/share/emacs/native-lisp/" ; [3]
"/home/user/.emacs.d/eln-cache/"
"/nix/store/hash1-emacs-packages-deps/share/emacs/native-lisp" ; [2]
"/nix/store/hash2-emacs-29.1-rc1/lib/emacs/29.1/native-lisp/")
load-path does not change with this patch applied.
[1]: 1476c13422/pkgs/build-support/emacs/wrapper.sh (L47)
[2]: 1476c13422/pkgs/build-support/emacs/wrapper.sh (L50)
[3]: 1476c13422/pkgs/build-support/emacs/wrapper.nix (L166)
Add this hook to checkPhase to allow for running MPI application in
the sandbox. It detects the MPI implementations and sets the respective
environment variables.
Before the change the following command did not work:
$ nix develop -i --impure --expr 'with import <nixpkgs> { system = "i686-linux"; }; (buildFHSUserEnv { name = "t"; targetPkgs = ps: with ps; [ libmpc stdenv.cc ]; }).env'
$ ld -lmpc -o a
ld: cannot find -lmpc: No such file or directory
It is expected to work as `NIX_LDFLAGS` does contain valid values:
$ echo $NIX_LDFLAGS
-L/usr/lib -L/usr/lib32
Note that for `gcc` it does work:
$ printf "int main(){}" | gcc -x c - -lmpc -o a
It happens because `HOST` role is enabled for `cc`:
$ echo $NIX_CC_WRAPPER_TARGET_HOST_i686_unknown_linux_gnu
1
But not for `BINTOOLS`:
$ echo $NIX_BINTOOLS_WRAPPER_TARGET_HOST_i686_unknown_linux_gnu
<empty>
The change adds BINTOOLS role and fixes linking:
$ nix develop -i --impure --expr 'with import ~/nm { system = "i686-linux"; }; (buildFHSUserEnv { name = "t"; targetPkgs = ps: with ps; [ libmpc stdenv.cc ]; }).env'
$ printf "int main(){}" | gcc -x c - -lmpc -o a
$ ld -lmpc -o a
ld: warning: cannot find entry symbol _start; not setting start address
Previously `writeDarwinBundle` used a handcrafted shell wrapper, however
this causes issues on Apple Silicon Macs as script-only application
bundles are always run under Rosetta[0][1].
Replacing the handcrafted shell wrapper with a binary wrapper allows
apps to run natively instead of requiring Rosetta. However, this means
we can no longer use `$1` and `$@`.
After checking nearly every current usage of `desktopToDarwinBundle`,
there were no apps that used `%[fFuU]` before the last argument, meaning
removing them naively is good enough for the current apps.
[0]: https://developer.apple.com/documentation/apple-silicon/building-a-universal-macos-binary
[1]: https://stackoverflow.com/a/68208374
The `name` & `version` attributes only existed in a previous form of
the API before it was switched over to using `fetchzip` &
`applyPatches`[1]. The attributes existed to be able to throw an
evaluation error with upgrade instructions when this was used. However,
this was before 22.11, so this shouldn't be in use anymore (and if
somebody is doing a migration from a very old NixOS, this is still
documented in the 22.11 release-notes[2]).
Anyways, this simplifies the code a little bit and also having both
`appName`/`appVersion` and `name`/`version` in there is quite confusing. But
still, I figured it's less confusing to not re-use attributes that were
deprecated in the past, hence the alternative naming.
[1] 3ca9b9a8ad
[2] d41b381310
This change makes sure that each Nextcloud plugin installed provides a
`meta` section with proper license information.
Unfortunately, the metadata from the appstore is useless for this
purpose since it doesn't differentiate between e.g. AGPL 3.x and AGPL
3.x or any later version. In fact, this isn't consistent between their
software, e.g. `bookmarks` has `agpl3Plus` according to the files'
headers[1] whereas `twofactor_nextcloud_notification` is AGPL 3 only[2].
I don't think there's any trivial and reasonable way to retrieve this
information programatically, so I decided to change the format of
`nextcloud-apps.json`[3] to also contain the license in the form of the
license attribute we have in `lib/licenses.nix`, i.e. GNU AGPL 3 or
later is `agpl3Plus`.
I retrieved the information using the following approach:
* go to the source code of $app at the revision we currently have
packaged
* check for a license identifier (does it specify the license only or
the license "or any later version")?
* first in `src/main.js` because most apps from Nextcloud itself used
actual spdx identifiers in the frontend's source-code.
* then in `lib/AppInfo/Application.php` which each Nextcloud app has.
License changes should be updated accordingly when updating the apps. As
with any other package in nixpkgs as well, this currently needs to be
done manually (or as part of the review process)[4].
Also, I decided to change the `name` of the `applyPatches` derivation
from `source-patched` to `nextcloud-app-${appName}-${appVersion}`. When
deploying a lot of apps (and probably displaying the diff using
`nix store diff-closures` on deploy), the current output isn't very
helpful. This is purely optional because I didn't want to break the
interface of `fetchNextcloudApp` again.
[1] https://github.com/nextcloud/bookmarks/blob/v13.1.0/lib/AppInfo/Application.php#L6
[2] https://github.com/nextcloud/twofactor_nextcloud_notification/blob/v3.7.0/lib/AppInfo/Application.php
[3] This isn't really well-defined since it's preprocessed with `jq(1)`
before passing the apps to nc4nix.
[4] Though notable license changes (e.g. agpl -> gpl) would also pop up
in the diff of <nextcloudversion>.json, so this is pretty easy to
catch.
The command `fakechroot` errored with buffer overflows. The `proot`
command doesn't seem to suffer from the same problem. The tar command
creating the layer errors with "permission denied" on a bunch of paths
in /proc but the layer seems to get built anyway.
this splits hardeningCFlags into hardeningCFlagsAfter and
hardeningCFlagsBefore (where most flags still remain) to allow
us to *append* `-D_FORTIFY_SOURCE=` values to the command-line,
forcing our choice of fortify level and avoiding potential
redefinition warnings/errors through use of `-U_FORTIFY_SOURCE`
While using libredirect in conjunction with geckodriver, I stumbled on
odd segfaults that happened when running the wrapped statx() call from
libredirect:
0x00007ffff7ddd541 in __strncmp_avx2 () from .../lib/libc.so.6
0x00007ffff7f6fe57 in statx () from .../lib/libredirect.so
0x00005555558d35bd in std::sys::unix::fs::try_statx::h2045d39b0c66d4e8 ()
0x00005555558d2230 in std::sys::unix::fs::stat::ha063998dfb361520 ()
0x0000555555714019 in mozversion::firefox_version::hdc3b57eb04947426 ()
0x00005555556a603c in geckodriver::capabilities::FirefoxCapabilities::version::h58e289917bd3c721 ()
0x00005555556a77f5 in <geckodriver::capabilities::FirefoxCapabilities as webdriver::capabilities::BrowserCapabilities>::validate_custom::h62d23cf9fd63b719 ()
0x000055555562a7c8 in webdriver::capabilities::SpecNewSessionParameters::validate::h60da250d33f0989f ()
0x00005555556d7a13 in <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold::h9427a360a3d0bf8f ()
0x0000555555669d85 in <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter::hd274d536ea29bb33 ()
0x00005555555c05ef in core::iter::adapters::try_process::hdf96a01ec1f9b8bd ()
0x000055555561768d in <webdriver::capabilities::SpecNewSessionParameters as webdriver::capabilities::CapabilitiesMatching>::match_browser::hfbd8c38f6db17e9f ()
0x00005555555ca6ef in <geckodriver::marionette::MarionetteHandler as webdriver::server::WebDriverHandler<geckodriver::command::GeckoExtensionRoute>>::handle_command::h13b98b9cb87a69d6 ()
0x00005555555e859e in webdriver::server::Dispatcher<T,U>::run::h746a8bf2f0bc24fd ()
0x000055555569ff0f in std::sys_common::backtrace::__rust_begin_short_backtrace::h3b920773bd467d2a ()
0x00005555555dbc99 in core::ops::function::FnOnce::call_once{{vtable.shim}}::h81ba7228877515f7 ()
0x00005555558d31a3 in std::sys::unix:🧵:Thread:🆕:thread_start::h4514580219a899c5 ()
0x00007ffff7d0ce24 in start_thread () from .../lib/libc.so.6
0x00007ffff7d8e9b0 in clone3 () from .../lib/libc.so.6
The reason why I found this odd was because it happens in the following
piece of code (shortened a bit):
1 static const char * rewrite(const char * path, char * buf)
2 {
3 if (path == NULL) return path;
4 for (int n = 0; n < nrRedirects; ++n) {
5 int len = strlen(from[n]);
6 if (strncmp(path, from[n], len) != 0) continue;
7 if (snprintf(buf, PATH_MAX, "%s%s", to[n], path + len) >= PATH_MAX)
8 abort();
9 return buf;
10 }
11 return path;
12 }
When inspecting the assembly, I found that the check for the null
pointer in line 3 was completely missing and the code was directly
entering the loop and then eventually segfault when running strncmp()
with a null pointer as its first argument.
I confirmed that indeed that check was missing by compiling libredirect
with "-O0" and comparing the generated assembly with the optimized one.
The one compiled with "-O0" had that check while the optimized one did
not and indeed when running geckodriver with the unoptimized version it
worked fine.
Digging in the Git history, I found 5677ce2008,
which actually introduced the null pointer check. Going back to that
commit however, the check actually was still in the generated assembly.
So I bisected between that commit and the most recent one and ended up
with commit ca8aa5dc87, which moved
everything to use GCC 7.
I haven't found out why *exactly* GCC was optimizing the check away, but
playing around on Godbolt with various other compilers seems that other
compilers such as Clang are doing it as well. Additionally, given that
passing NULL to stat() is UB, my guess is that compilers tend to assume
that such an argument can't be NULL. My assumption is based on the fact
that GCC warns with "argument 1 null where non-null expected" when
passing NULL to eg. stat().
To address this for now, I marked the path argument of the rewrite()
volatile and also added a test that should cause a segfault in case this
would regress again as it already did.
Signed-off-by: aszlig <aszlig@nix.build>
Since coreutils v9.2 the `--no-clobber` flag results in a non-zero exit
code when the destination files exist. Using `--update=none` will now
reproduce the old behavior of `--no-clobber`.
However, the `--update=none` flag was introduced in coreutils v9.3 and
thus `mergeImages` will fail if you have an older version than v9.3 in
stdenv after applying this commit.
[coreutils v9.3 changelog](f386722dc0/NEWS (L48))
Derivations built with `writeShellScriptBin`
should always be runnable with `nix run`. At present,
the derivation is missing both `meta.mainProgram`
and `pname`– this means that `nix run` falls back
to inferring the bin path from `name`. This is
unreliable and depends on faulty heuristics.
For context, reference the following snippet from
`nix run --help`:
If installable evaluates to a derivation, it will try to execute the
program <out>/bin/<name>, where out is the primary output store path
of the derivation, and name is the first of the following that exists:
· The meta.mainProgram attribute of the derivation.
· The pname attribute of the derivation.
· The name part of the value of the name attribute of the derivation.
This is very useful in conjunction with meta.pkgConfigModules, as the
new tester can use the list provided by this meta attribute as a default
value for moduleNames, making its usage in passthru.tests very
convenient.
For backwards compatibility, a shim under the old name is maintained
with a warning.
vcunat said
> This invocation of mktemp creates the file in the current directory, which is bad practice. We should add "--tmpdir=$TMPDIR" or make the template absolute.
> I noticed because one package did cd $src during installing, which is a read-only path...
