Commit Graph

10399 Commits

Author SHA1 Message Date
Tom Bereknyei
aab801db98 fix: check to see if there are any lines before
(cherry picked from commit 59db8fd62b)
2024-08-19 13:40:32 +00:00
Andrew Marshall
7043e7311f libstore: fix port binding in __darwinAllowLocalNetworking sandbox
In d60c3f7f7c, this was changed to close a
hole in the sandbox. Unfortunately, this was too restrictive such that it
made local port binding fail, thus making derivations that needed
`__darwinAllowLocalNetworking` gain nearly nothing, and thus largely
fail (as the primary use for it is to enable port binding).

This unfortunately does mean that a sandboxed build process can, in
coordination with an actor outside the sandbox, escape the sandbox by
binding a port and connecting to it externally to send data. I do not
see a way around this with my experimentation and understanding of the
(quite undocumented) macOS sandbox profile API. Notably it seems not
possible to use the sandbox to do any of:

- Restrict the remote IP of inbound network requests
- Restrict the address being bound to

As such, the `(local ip "*:*")` here appears to be functionally no
different than `(local ip "localhost:*")` (however it *should* be
different than removing the filter entirely, as that would make it also
apply to non-IP networking). Doing `(allow network-inbound (require-all
(local ip "localhost:*") (remote ip "localhost:*")))` causes listening
to fail.

Note that `network-inbound` implies `network-bind`.

(cherry picked from commit 00f6db36fd)
2024-08-17 03:17:40 +00:00
Eelco Dolstra
8a80f8a6ac Show when we're unpacking an archive into the Git cache
This happens in parallel with the download (which starts later), so
you only see this message when the download has finished but the
import hasn't.

(cherry picked from commit 01839b525c)
2024-07-29 13:02:55 +00:00
Eelco Dolstra
331bcdb831 Warn if the download buffer is full
(cherry picked from commit f6a9a71b38)
2024-07-29 13:02:55 +00:00
Eelco Dolstra
a57ded2403 Add 'download-buffer-size' setting
We are piping curl downloads into `unpackTarfileToSink()`, but the
latter is typically slower than the former if you're on a fast
connection. So the download could appear unnecessarily slow. (There is
even a risk that if the Git import is *really* slow for whatever
reason, the TCP connection could time out.)

So let's make the download buffer bigger by default - 64 MiB is big
enough for the Nixpkgs tarball. Perhaps in the future, we could have
an unlimited buffer that spills data to disk beyond a certain
threshold, but that's probably overkill.

(cherry picked from commit 8ffea0a018)
2024-07-29 13:02:55 +00:00
Eelco Dolstra
cfef5cacda Log download durations
(cherry picked from commit caf4e98f0c)
2024-07-29 13:02:55 +00:00
Emily
a500c90375 libstore: fix sandboxed builds on macOS
The recent fix for CVE-2024-38531 broke the sandbox on macOS
completely. As it’s not practical to use `chroot(2)` on
macOS, the build takes place in the main filesystem tree, and the
world‐unreadable wrapper directory prevents the build from accessing
its `$TMPDIR` at all.

The macOS sandbox probably shouldn’t be treated as any kind of a
security boundary in its current state, but this specific vulnerability
wasn’t possible to exploit on macOS anyway, as creating `set{u,g}id`
binaries is blocked by sandbox policy.

Locking down the build sandbox further may be a good idea in future,
but it already has significant compatibility issues. For now, restore
the previous status quo on macOS.

Thanks to @alois31 for helping me come to a better understanding of
the vulnerability.

Fixes: 1d3696f0fb
Closes: #11002
(cherry picked from commit af2e1142b1)
2024-07-05 15:09:00 +00:00
Emily
1a46fb95dd libstore: clean up the build directory properly
After the fix for CVE-2024-38531, this was only removing the nested
build directory, rather than the top‐level temporary directory.

Fixes: 1d3696f0fb
(cherry picked from commit 76e4adfaac)
2024-07-05 15:09:00 +00:00
kn
a06453e45f Use proper struct sockpeercred for SO_PEERCRED for OpenBSD
getsockopt(2) documents this;  ucred is wrong ("cr_" member prefix, no pid).

(cherry picked from commit 10ccdb7a41)
2024-07-03 15:57:13 +00:00
John Ericson
e39cf2fa86 Ident some CPP in nix daemon
Makes it easier for me to read.

(cherry picked from commit a09360400b)
2024-07-03 15:57:13 +00:00
Eelco Dolstra
54b27fcc60 Fix --no-sandbox
When sandboxing is disabled, we cannot put $TMPDIR underneath an
inaccessible directory.

(cherry picked from commit d54590fdf3)
2024-06-21 17:16:52 +02:00
Eelco Dolstra
a82010789e Formatting
(cherry picked from commit 58b7b3fd15)
2024-06-21 17:16:48 +02:00
Eelco Dolstra
f5f0d30597 Put the chroot inside a directory that isn't group/world-accessible
Previously, the .chroot directory had permission 750 or 755 (depending
on the uid-range system feature) and was owned by root/nixbld. This
makes it possible for any nixbld user (if uid-range is disabled) or
any user (if uid-range is enabled) to inspect the contents of the
chroot of an active build and maybe interfere with it (e.g. via /tmp
in the chroot, which has 1777 permission).

To prevent this, the root is now a subdirectory of .chroot, which has
permission 700 and is owned by root/root.

(cherry picked from commit ede95b1fc1)
2024-06-21 17:16:44 +02:00
Théophane Hufschmitt
8c20f0fc33 Run the builds in a daemon-controled directory
Instead of running the builds under
`$TMPDIR/{unique-build-directory-owned-by-the-build-user}`, run them
under `$TMPDIR/{unique-build-directory-owned-by-the-daemon}/{subdir-owned-by-the-build-user}`
where the build directory is only readable and traversable by the daemon user.

This achieves two things:

1. It prevents builders from making their build directory world-readable
   (or even writeable), which would allow the outside world to interact
   with them.
2. It prevents external processes running as the build user (either
   because that somehow leaked, maybe as a consequence of 1., or because
   `build-users` isn't in use) from gaining access to the build
   directory.

(cherry picked from commit 1d3696f0fb)
2024-06-21 17:16:37 +02:00
Enno Richter
0a78a55d51 flake check: Recognize well known homeModule/homeModules attributes
(cherry picked from commit 80ba7778e7)
2024-06-05 11:15:00 +00:00
Bryan Lai
4f68558c9a libutil/url: fix git+file:./ parse error
Previously, the "file:./" prefix was not correctly recognized in
fixGitURL; instead, it was mistaken as a file path, which resulted in a
parsed url of the form "file://file:./".

This commit fixes the issue by properly detecting the "file:" prefix.
Note, however, that unlike "file://", the "file:./" URI is _not_
standardized, but has been widely used to referred to relative file
paths. In particular, the "git+file:./" did work for nix<=2.18, and was
broken since nix 2.19.0.

Finally, this commit fixes the issue completely for the 2.19 series, but
is still inadequate for the 2.20 series due to new behaviors from the
switch to libgit2. However, it does improve the correctness of parsing
even though it is not yet a complete solution.

(cherry picked from commit 8594f3cd5a)
2024-06-04 08:27:15 +00:00
eihqnh
f00aa37873 nix repl: make runNix() isInteractive is true by default
(cherry picked from commit bb1a4ea21a)
2024-05-30 19:15:37 +02:00
Sarah Brofeldt
12967aea53 nix repl: hide progress bar during :edit
(cherry picked from commit e5f509ef0b)
2024-05-30 19:15:28 +02:00
Eelco Dolstra
92b4adcab8 Handle zip files containing symlinks
In streaming mode, libarchive doesn't handle symlinks in zip files
correctly. So write the entire file to disk so libarchive can access
it in random-access mode.

Fixes #10649. This was broken in cabee98152.

(cherry picked from commit 9951e14ae0)
2024-05-15 20:07:41 +00:00
Graham Christensen
0b245b7ad1 git putFile: support flake maximalists
Passing the commit message as an argument causes update failures on repositories with lots of flake inputs. In some cases, the commit message is over 250,000 bytes.

(cherry picked from commit 8b5e8f4fba)
2024-05-13 13:32:11 +00:00
Robert Hensing
293d59382e Fix fetchGit nested submodules
(cherry picked from commit 750bcaa330)
2024-05-09 09:07:45 +00:00
Dylan Green
aa165301d1
Pathlocks Implementation for Windows (#10586)
Based on Volth's original port.

Co-authored-by: volth <volth@volth.com>
2024-04-22 15:08:10 +00:00
Eelco Dolstra
cf1adbdd64
Merge pull request #10572 from NorfairKing/no-more-prev
Remove the 'prev' check entirely.
2024-04-22 10:53:31 +02:00
Robert Hensing
6fd2f42c2d
Merge pull request #10573 from RCoeurjoly/Rename_SearchPath
Rename SearchPath to LookupPath and searchPath to lookupPath
2024-04-21 17:33:46 +02:00
Robert Hensing
c6526fa33b
Merge pull request #10563 from hercules-ci/doc-glossary-base-directory
doc/glossary: Add base directory
2024-04-21 15:17:16 +02:00
Robert Hensing
73125e46fc doc/glossary: Add base directory 2024-04-21 14:24:33 +02:00
Tom Sydney Kerckhove
ad65a50a94 Remove the 'prev' check 2024-04-20 14:46:23 +02:00
Robert Hensing
e05b58b060 init: Add flag to avoid loading configuration 2024-04-20 01:45:04 +02:00
Robert Hensing
538eb2617a
Merge pull request #10537 from hercules-ci/c-api-nix-value-init-apply
C API: Add nix_init_apply
2024-04-18 20:09:24 +02:00
Robert Hensing
ad643cde58 C API: Add nix_init_apply
Thunks are relevant when initializing attrsets and lists, passing
arguments. This is an important way to produce them.
2024-04-18 19:13:38 +02:00
Eelco Dolstra
40499bca99
Merge pull request #10549 from alyssais/exportReferencesGraph
Fix exportReferencesGraph when given store subpath
2024-04-18 17:23:12 +02:00
John Ericson
fb9f4208ed
Don't include linux/ in #include
The linux dirs are conditionally added to the `-I` path.
2024-04-18 12:20:20 +02:00
Maximilian Bosch
ba68045187
libstore/local-derivation-goal: prohibit creating setuid/setgid binaries
With Linux kernel >=6.6 & glibc 2.39 a `fchmodat2(2)` is available that
isn't filtered away by the libseccomp sandbox.

Being able to use this to bypass that restriction has surprising results
for some builds such as lxc[1]:

> With kernel ≥6.6 and glibc 2.39, lxc's install phase uses fchmodat2,
> which slips through 9b88e52846/src/libstore/build/local-derivation-goal.cc (L1650-L1663).
> The fixupPhase then uses fchmodat, which fails.
> With older kernel or glibc, setting the suid bit fails in the
> install phase, which is not treated as fatal, and then the
> fixup phase does not try to set it again.

Please note that there are still ways to bypass this sandbox[2] and this is
mostly a fix for the breaking builds.

This change works by creating a syscall filter for the `fchmodat2`
syscall (number 452 on most systems). The problem is that glibc 2.39
and seccomp 2.5.5 are needed to have the correct syscall number available
via `__NR_fchmodat2` / `__SNR_fchmodat2`, but this flake is still on
nixpkgs 23.11. To have this change everywhere and not dependent on the
glibc this package is built against, I added a header
"fchmodat2-compat.hh" that sets the syscall number based on the
architecture. On most platforms its 452 according to glibc with a few
exceptions:

    $ rg --pcre2 'define __NR_fchmodat2 (?!452)'
    sysdeps/unix/sysv/linux/x86_64/x32/arch-syscall.h
    58:#define __NR_fchmodat2 1073742276

    sysdeps/unix/sysv/linux/mips/mips64/n32/arch-syscall.h
    67:#define __NR_fchmodat2 6452

    sysdeps/unix/sysv/linux/mips/mips64/n64/arch-syscall.h
    62:#define __NR_fchmodat2 5452

    sysdeps/unix/sysv/linux/mips/mips32/arch-syscall.h
    70:#define __NR_fchmodat2 4452

    sysdeps/unix/sysv/linux/alpha/arch-syscall.h
    59:#define __NR_fchmodat2 562

I tested the change by adding the diff below as patch to
`pkgs/tools/package-management/nix/common.nix` & then built a VM from
the following config using my dirty nixpkgs master:

    {
      vm = { pkgs, ... }: {
        virtualisation.writableStore = true;
        virtualisation.memorySize = 8192;
        virtualisation.diskSize = 12 * 1024;
        nix.package = pkgs.nixVersions.nix_2_21;
      };
    }

The original issue can be triggered via

    nix build -L github:nixos/nixpkgs/d6dc19adbda4fd92fe9a332327a8113eaa843894#lxc \
      --extra-experimental-features 'nix-command flakes'

however the problem disappears with this patch applied.

Closes #10424

[1] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2031073804
[2] https://github.com/NixOS/nixpkgs/issues/300635#issuecomment-2030844251
2024-04-18 12:20:20 +02:00
John Ericson
9c815db366 file-descriptor.hh: Avoid some Cism for better C++isms
- `reinterpret_cast` not C-style cast
- `using` not `typedef`
2024-04-17 19:55:40 -04:00
John Ericson
6fa3656a32 Make a few commands that were Unix-only no longer
Also clean up some more linux-specific (`setPersonality`) code in
alignment with recent best practices.
2024-04-17 16:20:56 -04:00
Alyssa Ross
0774e8ba33
Fix exportReferencesGraph when given store subpath
With Nix 2.3, it was possible to pass a subpath of a store path to
exportReferencesGraph:

	with import <nixpkgs> {};

	let
	  hello = writeShellScriptBin "hello" ''
	    echo ${toString builtins.currentTime}
	  '';
	in

	writeClosure [ "${hello}/bin/hello" ]

This regressed with Nix 2.4, with a very confusing error message, that
presumably indicates it was unintentional:

	error: path '/nix/store/3gl7kgjr4pwf03f0x70dgx9ln3bhl7zc-hello/bin/hello' is not in the Nix store
2024-04-17 21:51:59 +02:00
John Ericson
21d9412ddc Improve local-overlay docs in a few ways
In response to https://discourse.nixos.org/t/super-colliding-nix-stores/28462/24
2024-04-17 15:28:19 -04:00
a-kenji
5fd28eeba4 docs: fix wiki link
Link to the official wiki.
2024-04-17 19:34:22 +02:00
John Ericson
1063aa502a Relax store path canonicalization 2024-04-17 12:26:10 -04:00
John Ericson
b529d91902 Prefix -DNIX_ paths to be windows-complient for windows
This is a hacky solution, but it will do for now.
2024-04-17 12:26:10 -04:00
John Ericson
8433027e35 Build a minimized Nix with MinGW
At this point many features are stripped out, but this works:

- Can run libnix{util,store,expr} unit tests
- Can run some Nix commands

Co-Authored-By volth <volth@volth.com>
Co-Authored-By Brian McKenna <brian@brianmckenna.org>
2024-04-17 12:26:10 -04:00
John Ericson
2248a3f545 Create no-op Window pathlocks implementation
This keeps the call sites simple, eventually this should be filled in.
2024-04-17 11:48:14 -04:00
John Ericson
05b9dac754
Fix friend struct/class mismatch warning 2024-04-17 15:47:07 +00:00
Eelco Dolstra
6a3ecdaa39 Merge remote-tracking branch 'origin/master' into finish-value 2024-04-17 16:02:44 +02:00
Eelco Dolstra
d53ad516e8
Merge pull request #10517 from edolstra/path-display
Add missing setPathDisplay() calls
2024-04-17 15:36:08 +02:00
José Luis Lafuente
c75b143b6c
C API: nix_get_string now accepts a callback to return the value 2024-04-16 22:36:39 +02:00
Eelco Dolstra
74e4bc9b1d
Merge pull request #10486 from tweag/jl/c-api_function-pointer
C API: Safer function pointer casting
2024-04-16 18:56:08 +02:00
Eelco Dolstra
6892c9803c GitInputScheme: Fix path display for workdirs and submodules 2024-04-16 16:42:47 +02:00
Eelco Dolstra
79363b2273 MountedInputAccessor, FilteringInputAccessor: Respect the path display prefix/suffix
This was causing Git paths not to be rendered correctly.
2024-04-16 16:42:41 +02:00
Eelco Dolstra
6df58a0891 MercurialInputScheme: Improve path display 2024-04-16 14:14:50 +02:00