Commit Graph

10576 Commits

Author SHA1 Message Date
Robert Hensing
1c70eb8eee libcmd: Fix #10774 2024-05-29 21:42:53 +02:00
Qyriad
18ac6545fc print type and value in "flake attr is not a derivation" errors
This turns errors like:

error: flake output attribute 'hydraJobs' is not a derivation or path

into errors like:

error: expected flake output attribute 'hydraJobs' to be a derivation or
path but found a set: { binaryTarball = «thunk»; build = «thunk»; etc> }

This change affects all InstallableFlake commands.

Source: 20981461d4
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2024-05-29 20:51:32 +02:00
Jörg Thalheim
5786e1ae7c
docs: mention importNative/exec in allow-unsafe-native-code-during-evaluation (#10803)
* docs: mention importNative/exec in allow-unsafe-native-code-during-evaluation

Both of these still needs their own actual documentation, but they are
at least now mentioned that they exist and what they're enabled by.

Co-authored-by: Qyriad <qyriad@qyriad.me>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
2024-05-29 07:50:51 +00:00
Philipp
2bd66922ee
add empty line to documentation comments after @brief field (#10800)
* add empty line to documentation comments after `@brief` field

Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2024-05-28 23:05:40 +00:00
John Ericson
bcdee80a0d More work on the scheduler for windows
- Get a rump derivation goal: hook instance will come later, local
  derivation goal will come after that.

- Start cleaning up the channel / waiting code with an abstraction.
2024-05-28 11:39:49 -04:00
John Ericson
1d5d748fe4 Fix format
39b2a399ad passed CI but was landed after
the formatting change in 1d6c2316a9.
2024-05-27 22:32:52 -04:00
John Ericson
d0c7da131f
Merge pull request #10678 from nix-windows/windows-substitution-goal
Start building the scheduler for Windows
2024-05-27 17:47:29 -04:00
John Ericson
3e9c3738d3 Create CommonSSHStoreConfig::createSSHMaster
By moving `host` to the config, we can do a lot further cleanups and
dedups. This anticipates a world where we always go `StoreReference` ->
`*StoreConfig` -> `Store*` rather than skipping the middle step too.

Progress on #10766

Progress on https://github.com/NixOS/hydra/issues/1164
2024-05-27 16:12:53 -04:00
John Ericson
17964441d9
Merge pull request #10781 from obsidiansystems/build-mode-parse
Worker proto use proper serialiser for `BuildMode`
2024-05-27 16:07:54 -04:00
John Ericson
263905da4b
Merge pull request #10785 from mjoerg/fix-typos
fix typos
2024-05-27 10:24:53 -04:00
John Ericson
8527f4e7fa
Merge pull request #10754 from qwqawawow/master
nix repl: make runNix() isInteractive is true by default
2024-05-27 10:07:15 -04:00
Martin Joerg
e7ea5591a2 fix typos 2024-05-27 15:56:52 +02:00
John Ericson
aa4a2927a7
Merge pull request #10778 from Mic92/safer-tab-completion
repl: do not crash when tab-completing import errors
2024-05-27 09:54:29 -04:00
John Ericson
f71b4da0b3 Factor our connection code for worker proto like serve proto
This increases test coverage, and gets the worker protocol ready to be
used by Hydra.

Why don't we just try to use the store interface in Hydra? Well, the
problem is that the store interface works on connection pools, with each
opreation getting potentially a different connection, but the way temp
roots work requires that we keep one logical "transaction" (temp root
session) using the same connection.

The longer-term solution probably is making connections themselves
implement the store interface, but that is something that builds on
this, so I feel OK that this is not churn in the wrong direction.

Fixes #9584
2024-05-27 00:43:46 -04:00
John Ericson
8ebd99c74e Back in enum values for BuildMode serializer
We don't want to rely on how C assigns numbers for enums in the wire
format. Sure, this is totally determined by the ABI, but it obscures the
code and makes it harder to safely change the enum definition (should we
need to) without accidentally breaking the wire format.
2024-05-27 00:22:55 -04:00
John Ericson
eeb89c28b0 Worker proto use proper serialiser for BuildMode
Do this instead of an unchecked cast

I redid this to use the serialisation framework (including a unit test),
but I am keeping the reference to credit Jade for spotting the issue.

Change-Id: Icf6af7935e8f139bef36b40ad475e973aa48855c
(adapted from commit 2a7a824d83dc5fb33326b8b89625685f283a743b)

Co-Authored-By: Jade Lovelace <lix@jade.fyi>
2024-05-27 00:22:55 -04:00
Pierre Bourdon
ffe6ba69d6 repl: do not crash when tab-completing import errors
File not found while importing is not currently caught by the tab-completion handler.

Original bug report: https://git.lix.systems/lix-project/lix/issues/340
Fix has been adapted from https://gerrit.lix.systems/c/lix/+/1189

Example crash:

$ cat /tmp/foo.nix
{
  someImport = import ./this_file_does_not_exist;
}
$ ./src/nix/nix repl --file /tmp/foo.nix
warning: unknown experimental feature 'repl-flake'
Nix 2.23.0pre20240517_dirty
Type :? for help.
Loading installable ''...
Added 1 variables.
nix-repl> someImport.<TAB>
2024-05-25 23:15:28 +02:00
Jörg Thalheim
f97da4b11c libutil/source-accessor: custom error if source does not exist
This allows better error handling by catching this error in particular.
2024-05-25 23:06:57 +02:00
eldritch horrors
2de4589e46 libstore: remove unused copyPath function
Source: 47523944c5

Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2024-05-25 22:28:41 +02:00
John Ericson
5cfa75ea16
Merge pull request #10737 from poweredbypie/mingw-stackSize
Implement `setStackSize` on Windows
2024-05-25 09:56:02 -04:00
John Ericson
e0c94b91ee
Merge pull request #10757 from obsidiansystems/fix-4977
Require `drvPath` attribute to end with `.drv`
2024-05-24 12:14:59 -04:00
Eelco Dolstra
eeb4c40867 Typo 2024-05-24 16:35:06 +02:00
Eelco Dolstra
2c88930ef2 AttrCursor: Remove forceErrors
Instead, force evaluation of the original value only if we need to
show the exception to the user.
2024-05-24 16:34:57 +02:00
Robert Hensing
c90a763273
Merge pull request #10767 from hercules-ci/fix-c-api-primop-for-strict-initializers
C API: Fix custom primops
2024-05-24 08:32:02 +02:00
PoweredByPie
0b7da099d1 Commit more stack size in some windows binaries
This way we can commit the same amount of stack size (64 MB) without a conditional.
Includes nix, libnixexpr-tests, libnixfetchers-tests, libnixstore-tests, libnixutil-tests.
2024-05-23 17:42:55 -07:00
Robert Hensing
2497d10351 C API: Add nix_value_call_multi, NIX_VALUE_CALL
_multi can be implemented more efficiently.
NIX_VALUE_CALL is a convenient way to invoke it.
2024-05-23 21:24:06 +02:00
Robert Hensing
4bc4fb40ea C API: builtin -> custom function
Not all primops will be in `builtins`.
2024-05-23 21:23:14 +02:00
Robert Hensing
8ef6efc184 C API: Require non-thunk value from primop definition 2024-05-23 21:22:59 +02:00
Robert Hensing
8884227045 C API: Require initialized value from primop definition 2024-05-23 21:22:21 +02:00
Robert Hensing
a942a34469 C API: Fix nix_c_primop_wrapper for strict initializers
https://github.com/NixOS/nix/pull/10555 added a check requiring
that output parameters always have an uninitialized Value as argument.

Unfortunately the output parameter of the primop callback received
a thunk instead.

See the comment for implementation considerations.
2024-05-23 18:32:49 +02:00
John Ericson
5384ceacc3 Document field being initialized in Machine constructor 2024-05-23 11:28:25 -04:00
PoweredByPie
5f68e6d69f Get max stack size in setStackSize to match Linux 2024-05-23 03:54:35 -07:00
John Ericson
f2bcebc450 Restore exposing machine file parsing
This was accidentally removed in
e989c83b44. I restored it and also did a
few other cleanups:

- Make a static method for namespacing purposes

- Put the test files in the data dir with the other test data

- Avoid mutating globals in the machine config tests

This will be used by Hydra.
2024-05-23 00:03:52 -04:00
John Ericson
4a19f4a866
Merge pull request #9839 from obsidiansystems/more-machine-cleanup
Create `StoreReference` and use it in `Machine`
2024-05-22 17:01:57 -04:00
John Ericson
dc7615dbbb tryUnshareFilesystem: Ignore ENOSYS too
Fixes #10747
2024-05-22 16:07:38 -04:00
John Ericson
d5fdfdc592 unshareFilesystem: Do not assume caller 2024-05-22 16:07:34 -04:00
John Ericson
f923ed6b6a Require drvPath attribute to end with .drv
Fixes #4977
2024-05-22 12:50:24 -04:00
John Ericson
b3ebcc5aad Use the new StoreReference in Machine
This makes the remote builder abstract syntax more robust.
2024-05-22 09:20:15 -04:00
John Ericson
b59a7a14c4 Add StoreReference::render
This will be needed for the next step.

Also allows us to write round trip tests.
2024-05-22 09:20:15 -04:00
John Ericson
c036d75f9e Factor out abstract syntax for Store URIs
Need to decouple parsing from actually opening a store for Machine
configs.

Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-22 09:20:15 -04:00
John Ericson
1d6c2316a9 Slightly change formatting style
For long expressions, one argument or parameter per line is just easier.
2024-05-22 09:20:15 -04:00
Robert Hensing
bd7a074636
Merge pull request #10724 from prednaz/libexpr-c_example
remove redundant and outdated example from `libexpr-c` documentation
2024-05-22 12:59:14 +02:00
John Ericson
470c0501eb Ensure all store types support "real" URIs
In particular `local://<path>` and `unix://` (without any path) now
work, and mean the same things as `local` and `daemon`, respectively. We
thus now have the opportunity to desguar `local` and `daemon` early.

This will allow me to make a change to
https://github.com/NixOS/nix/pull/9839 requested during review to
desugar those earlier.

Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
2024-05-21 11:56:40 -04:00
eihqnh
bb1a4ea21a nix repl: make runNix() isInteractive is true by default 2024-05-21 20:05:43 +08:00
Philipp Zander
142222030c remove redundant and outdated example from libexpr-c documentation 2024-05-21 14:00:34 +02:00
John Ericson
8b369f90fd Query path infos (plural) and handshake version minimum for hydra
1. Hydra currently queries for multiple path infos at once, so let us
   make a connection item for that.

2. The minimum of the two versions should always be used, see #9584.
   (The issue remains open because the daemon protocol needs to be
   likewise updated.)
2024-05-20 17:43:18 -04:00
Robert Hensing
9a2f21711d
Merge pull request #10494 from obsidiansystems/primop-better-method
Slight refactors in preparation for #10480
2024-05-20 16:03:15 +02:00
Robert Hensing
20ed0c02b8
Merge pull request #10688 from hraban/fix/develop-aliases
fix: don’t expand aliases in develop stdenv setup
2024-05-20 15:59:21 +02:00
Robert Hensing
67db9e0c64
Merge pull request #10722 from obsidiansystems/ca-obj-docs
Content addressing store objects
2024-05-20 15:58:29 +02:00
John Ericson
1c75af969a Document store object content addressing & improve JSON format
The JSON format no longer uses the legacy ATerm `r:` prefixing nonsese,
but separate fields.

Progress on #9866

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-20 09:34:35 -04:00
Robert Hensing
e4be8abe42
Merge pull request #10701 from cole-h/nix-command-warn-unknown-settings
Warn on unknown settings when the first positional is an argument
2024-05-20 15:15:13 +02:00
John Ericson
43dc575fd7
Merge pull request #10738 from poweredbypie/mingw-windowSize
Implement `updateWindowSize` for Windows
2024-05-20 08:57:22 -04:00
John Ericson
d48bbda2e7
Update the updateWindowSize documentation 2024-05-20 08:34:49 -04:00
Eelco Dolstra
927034e7ac value.hh: Shut up warning about useless const qualifier 2024-05-20 10:25:04 +02:00
Philipp
e0bfa6c55f
small additions to the documentation of nix_store_open and nix_state_create (#10728) 2024-05-20 08:27:33 +02:00
PoweredByPie
a41f4223de Use setStackSize on Windows 2024-05-18 16:19:01 -07:00
PoweredByPie
6a3f906382 Implement setStackSize for Windows 2024-05-18 16:19:01 -07:00
PoweredByPie
e42d00c961 Change rlim_t to size_t in setStackSize in preparation of Windows impl 2024-05-18 16:18:50 -07:00
PoweredByPie
53f0c44d6c Implement updateWindowSize for Windows 2024-05-18 16:14:20 -07:00
John Ericson
1ab107b0bf
Merge pull request #10734 from Mic92/better-keepgoing
nix3-build: show all FOD errors with `--keep-going`
2024-05-17 18:46:57 -04:00
John Ericson
d25e54d56a
Merge pull request #10733 from alyssais/nix-store-delete
Improve nix-store --delete failure message
2024-05-17 18:41:23 -04:00
Qyriad
05bc889b89 add and fix -Wignored-qualifiers
Change-Id: I4bffa766ae04dd80355f9b8c10e59700e4b406da
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2024-05-17 19:29:50 +02:00
Qyriad
52a16b7e59 add and fix -Wdeprecated-copy
*so* many warnings, from only two definitions

Change-Id: If2561cd500c05a1e33cce984faf9f3e42a8a95ac
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2024-05-17 19:25:35 +02:00
Maximilian Bosch
19720d733f nix3-build: show all FOD errors with --keep-going
Basically I'd expect the same behavior as with `nix-build`, i.e.
with `--keep-going` the hash-mismatch error of each failing
fixed-output derivation is shown.

The approach is derived from `Store::buildPaths` (`entry-point.cc`):
instead of throwing the first build-result, check if there are any build
errors and if so, display all of them and throw after that.

Unfortunately, the BuildResult struct doesn't have an `ErrorInfo`
(there's a FIXME for that at least), so I have to construct my own here.
This is a rather cheap bugfix and I decided against touching too many
parts of libstore for that (also I don't know if that's in line with the
ongoing refactoring work).

Closes https://git.lix.systems/lix-project/lix/issues/302

Change-Id: I378ab984fa271e6808c6897c45e0f070eb4c6fac
Signed-off-by: Jörg Thalheim <joerg@thalheim.io>
2024-05-17 18:48:52 +02:00
Alyssa Ross
979a019014
Improve nix-store --delete failure message
On several occasions I've found myself confused when trying to delete
a store path, because I am told it's still alive, but
nix-store --query --roots doesn't show anything.  Let's save future
users this confusion by mentioning that a path might be alive due to
having referrers, not just roots.
2024-05-17 18:20:22 +02:00
Philipp Zander
f63292462c document nix_external_print's printer parameter to be an out parameter 2024-05-16 02:55:38 +02:00
Philipp Zander
359043ed0d add missing c api parameter names to documentation 2024-05-16 02:55:38 +02:00
Robert Hensing
303268bb71
Merge pull request #10479 from obsidiansystems/ca-fso-docs
Document file system object content addressing
2024-05-15 22:52:53 +02:00
John Ericson
043135a848 Document file system object content addressing
In addition:

- Take the opportunity to add a bunch more missing hyperlinks, too.

- Remove some glossary entries that are now subsumed by dedicated pages.
  We used to not be able to do this without breaking link fragments, but
  now we can, so pick up where we left off.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-15 16:28:48 -04:00
Eelco Dolstra
3026613893
Merge pull request #10675 from edolstra/zip-symlinks
Handle zip files containing symlinks
2024-05-15 22:07:22 +02:00
Cole Helbling
39a269657e libutil/args: warn on unknown settings after parsing all flags 2024-05-15 12:25:03 -07:00
Eelco Dolstra
39f7cbdc7c
Merge pull request #10691 from DeterminateSystems/commit-lock-file-summary
Rename commit-lockfile-summary to commit-lock-file-summary for consistency
2024-05-14 10:06:48 +02:00
Graham Christensen
e1e041ed8f Rename commit-lockfile-summary to commit-lock-file-summary for consistency 2024-05-13 09:23:59 -04:00
siddhantCodes
39e8aad446 Merge branch 'master' of github.com:NixOS/nix 2024-05-13 18:43:12 +05:30
Eelco Dolstra
56abd341bb
Merge pull request #10685 from siddhantk232/fs-cleanup
inline the usage of `nix::renameFile`, `nix::getFileType` and `nix::copyFile`
2024-05-13 14:12:10 +02:00
siddhantCodes
62e1ea2f4b use path for from arg in nix::copyFile 2024-05-13 16:10:21 +05:30
Eelco Dolstra
5534682166 Update src/libutil/util.hh
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
2024-05-13 12:14:43 +02:00
siddhantCodes
4d0777ca69 fix: copy fileName before calling std::distance 2024-05-13 15:36:00 +05:30
Eelco Dolstra
cf3b044b7e Make large path warnings human-readable 2024-05-13 11:52:38 +02:00
Eelco Dolstra
5314430437 Move printSize() into libutil
Also always include the unit (i.e. "MiB" instead of "M").
2024-05-13 11:52:38 +02:00
Eelco Dolstra
dbe1b51580 Add setting to warn about copying/hashing large paths
This is useful for diagnosing whether an evaluation is copying large
paths to the store. Example:

   $ nix build .#packages.x86_64-linux.default --large-path-warning-threshold 1000000
   warning: copied large path '/home/eelco/Dev/nix-master/' to the store (6271792 bytes)
   warning: copied large path '«github:NixOS/nixpkgs/b550fe4b4776908ac2a861124307045f8e717c8e?narHash=sha256-7kkJQd4rZ%2BvFrzWu8sTRtta5D1kBG0LSRYAfhtmMlSo%3D»/' to the store (155263768 bytes)
   warning: copied large path '«github:libgit2/libgit2/45fd9ed7ae1a9b74b957ef4f337bc3c8b3df01b5?narHash=sha256-oX4Z3S9WtJlwvj0uH9HlYcWv%2Bx1hqp8mhXl7HsLu2f0%3D»/' to the store (22175416 bytes)
   warning: copied large path '/nix/store/z985088mcd6w23qwdlirsinnyzayagki-source' to the store (5885872 bytes)
2024-05-13 11:52:27 +02:00
Eelco Dolstra
d8559cad8d
Merge pull request #10686 from DeterminateSystems/long-commit-message
git putFile: support flake maximalists
2024-05-13 10:22:57 +02:00
Hraban Luyat
6bf7edb18b fix: don’t expand aliases in develop stdenv setup
This fixes https://github.com/NixOS/nixpkgs/pull/290775 by not expanding aliases
when sourcing the stdenv setup script. The way bash handles aliases is to expand
them when a function is defined, not when it is used. I.e.:

    $ alias echo="echo bar "
    $ echo foo
    bar foo
    $ xyzzy() { echo foo; }
    $ shopt -u expand_aliases
    $ xyzzy
    bar foo
    $ xyzzy2() { echo foo; }
    $ xyzzy2
    foo

The problem is that ~/.bashrc is sourced before the stdenv setup, and bashrc
commonly sets aliases for ‘cp’, ‘mv’ and ‘rm’ which you don’t want to take
effect in the stdenv derivation builders. The original commit introducing this
feature (5fd8cf7667) even mentioned this very
alias.

The only way to avoid this is to disable aliases entirely while sourcing the
stdenv setup, and reenable them afterwards.
2024-05-12 21:57:55 -04:00
Graham Christensen
8b5e8f4fba git putFile: support flake maximalists
Passing the commit message as an argument causes update failures on repositories with lots of flake inputs. In some cases, the commit message is over 250,000 bytes.
2024-05-12 16:42:43 -04:00
Robert Hensing
c940d11fb0
Merge pull request #10666 from tie/derivation-outputs-drv-path
Forbid drvPath in strictDerivation outputs attribute
2024-05-12 21:14:32 +02:00
siddhantCodes
ccf94545db rename copy -> copyFile and remove old copyFile
the old `copyFile` was just a wrapper that was calling the `copy`
function. This wrapper function is removed and the `copy` function is
renamed to `copyFile`.
2024-05-12 19:20:17 +05:30
siddhantCodes
d3b7367c80 inline usage of nix::getFileType and remove it 2024-05-12 18:58:05 +05:30
siddhantCodes
4537663740 inline the usage of nix::renameFile
use `std::filesystem::rename` everywhere and remove `nix::renameFile`
2024-05-12 18:40:16 +05:30
siddhantCodes
1db7d1b840 inline the usage of nix::readDirectory
`nix::readDirectory` is removed. `std::filesystem::directory_iterator`
is used directly in places that used this util.
2024-05-12 17:42:18 +05:30
John Ericson
39b2a399ad Start building the scheduler for Windows
Building derivations is a lot harder, but the downloading goals is
portable enough.

The "common channel" code is due to Volth. I wonder if there is a way we
can factor it out into separate functions / files to avoid some
within-function CPP.

Co-authored-by: volth <volth@volth.com>
2024-05-10 20:23:59 -04:00
John Ericson
e0ff8da9d5 Build the local store on Windows
Fixes #10558

Co-Authored-By: Eugene Butler <eugene@eugene4.com>
Co-authored-by: Eelco Dolstra <edolstra@gmail.com>
2024-05-10 13:05:23 -04:00
John Ericson
0998a3ac01 Remove LocalStore::OptimiseStats::blocksFreed as it is dead code 2024-05-10 12:55:21 -04:00
Eelco Dolstra
9951e14ae0 Handle zip files containing symlinks
In streaming mode, libarchive doesn't handle symlinks in zip files
correctly. So write the entire file to disk so libarchive can access
it in random-access mode.

Fixes #10649. This was broken in cabee98152.
2024-05-09 19:33:09 +02:00
Eelco Dolstra
de8c3c034c
Merge pull request #10668 from edolstra/unit-prefixes
Support unit prefixes in configuration settings
2024-05-09 19:29:36 +02:00
John Ericson
b5605217ae
Document string context (#8595)
* Document string context

Now what we have enough primops, we can document how string contexts
work.

Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Co-authored-by: Théophane Hufschmitt <7226587+thufschmitt@users.noreply.github.com>
Co-authored-by: Valentin Gagarin <valentin.gagarin@tweag.io>
Co-authored-by: Felix Uhl <iFreilicht@users.noreply.github.com>
2024-05-08 23:14:00 +02:00
Eelco Dolstra
77a406a5a6 Fix warning 2024-05-08 21:16:53 +02:00
Eelco Dolstra
79c7d6205c Support unit prefixes in configuration settings
E.g. you can now say `--min-free 1G`.
2024-05-08 21:11:09 +02:00
Ivan Trubach
081faeda8c Forbid drvPath in strictDerivation outputs attribute
builtins.strictDerivation returns an attribute set with drvPath and
output paths. For some reason, current implementation forbids drv
instead of drvPath.
2024-05-08 18:20:37 +03:00
siddhantCodes
ddea4c6deb rm isLink
isLink util is removed in favour of std::filesystem::is_symlink
2024-05-08 19:59:37 +05:30
Siddhant Kumar
fcbc36cf78
Use std::filesystem::path in more places (#10657)
Progress on #9205

Co-Authored-By: John Ericson <John.Ericson@Obsidian.Systems>

* Get rid of `PathNG`, just use `std::filesystem::path`
2024-05-07 22:28:50 +00:00