Commit Graph

839 Commits

Author SHA1 Message Date
Jacek Galowicz
869545857f
Merge pull request #208354 from bjornfor/nixos-test-driver-quote-shell-args
nixos/test-driver: quote some shell command lines
2022-12-30 22:41:04 +01:00
Bjørn Forsman
c876f0c146 nixos/test-driver: quote some shell command lines
This makes them robust against paths with whitespace.
2022-12-30 13:19:59 +01:00
Bjørn Forsman
4c45c3f8f2 nixos/test-driver: use ASCII single quotes everywhere
A few places used Unicode U+2018/U+2019 left/right single quotes (but
not always correctly balanced). Let's just use plain ASCII single quotes
everywhere.
2022-12-30 13:00:19 +01:00
Bjørn Forsman
ce5644f658 nixos/test-driver: use f-strings instead of .format()
For readability.

Suggested-by: @tfc
2022-12-29 23:51:35 +01:00
Bjørn Forsman
dba49a43a0 nixos/test-driver: add optional address arg to wait_for_{open,closed}_port
This is useful for testing servers configured to listen on specific
addresses.
2022-12-29 13:50:17 +01:00
Naïm Favier
3fc528ff7f
Merge pull request #207095 from ncfavier/linux-custom-kernel 2022-12-27 17:10:43 +01:00
K900
2d3cf010fe
Revert "treewide: use nativeBuildInputs with runCommand instead of inlining" 2022-12-26 21:05:35 +03:00
Jörg Thalheim
ea415d1a38
Merge pull request #207038 from NixOS/make-disk-image-for-uefi
make-disk-image: documentation, UEFI variables recording, improved determinism
2022-12-26 11:02:28 +00:00
Sandro
5af3f865e8
Merge pull request #206775 from SuperSandro2000/runCommand-nativeBuildInputs 2022-12-25 21:42:05 +01:00
Raito Bezarius
22adcaa449 nixos/lib/make-disk-image: docs, UEFI vars recording, more determinism
- Extensive documentation in NixOS manual
- Deterministic mode that fixes various identifiers relative to disk
  partitions and filesystems in ext4 case
- UEFI variable recording
2022-12-24 19:15:29 +01:00
Robert Hensing
7bfb51f9cd
Merge pull request #207441 from hercules-ci/nixos-testing-python-no-more-specialArgs
nixos/testing-python.nix: Help users who need specialArgs
2022-12-24 17:19:32 +01:00
Jacek Galowicz
9938dec682
Merge pull request #170636 from Synthetica9/wait_before_entry
nixos/test-driver: add wait_before_entry
2022-12-23 19:46:23 +01:00
Robert Hensing
d9283c04d5 nixos/testing-python.nix: Help users who need specialArgs 2022-12-23 18:41:52 +01:00
Naïm Favier
84d8b9a809
nixos/systemd: document what scriptArgs is for 2022-12-22 15:17:05 +01:00
Naïm Favier
f677cbabe9
nixos/tests: remove minimal-kernel module
It's not used, doesn't build, and seems like the only reason to have
`manualConfig` take `stdenv` as an argument.
2022-12-22 01:42:15 +01:00
Sandro Jäckel
26f704b545
treewide: use nativeBuildInputs with runCommand instead of inlining 2022-12-18 23:36:40 +01:00
figsoda
6bb0dbf91f nixos: fix typos 2022-12-17 19:31:14 -05:00
Artturin
05a2dfd674 lib.replaceChars: warn about being a deprecated alias
replaceStrings has been in nix since 2015(nix 1.10)

so it is safe to remove the fallback

d6d5885c15
2022-12-15 22:25:51 +02:00
Naïm Favier
109f8b4657 nixos/make-options-doc: remove dead code
The logic for pretty-printing Nix values isn't needed any more because
`optionAttrSetToDocList` returns already rendered values.
2022-12-08 17:52:52 +01:00
Naïm Favier
6a117e2759 nixos/doc: render option values using lib.generators.toPretty
Render un`_type`d defaults and examples as `literalExpression`s using
`lib.generators.toPretty` so that consumers don't have to reinvent Nix
pretty-printing. `renderOptionValue` is kept internal for now intentionally.

Make `toPretty` print floats as valid Nix values (without a tilde).

Get rid of the now-obsolete `substSpecial` function.

Move towards disallowing evaluation of packages in the manual by
raising a warning on `pkgs.foo.{outPath,drvPath}`; later, this should
throw an error. Instead, module authors should use `literalExpression`
and `mkPackageOption`.
2022-12-08 17:52:52 +01:00
sandydoo
3564228a10
nixos/make-options-doc: improve CommonMark formatting
Render the `type` attribute in a code block to match the rest of the
attributes.
2022-12-07 14:39:26 +00:00
sandydoo
926afb6f1c
nixos/make-options-doc: pretty-print literals
Unlike the XML doc renderer, the AsciiDoc and CommonMark renderers don't
pretty-print certain complex types, like literal expressions, DocBook
literals, and derivations. These types are dumped into the documentation
as JSON.

This commit parses and unwraps these types when loading the
JSON-formatted NixOS options. The AsciiDoc and CommonMark renders have
also been combined into a single script to allow code reuse.
2022-12-07 14:06:56 +00:00
Artturi
645b5a2f9f
Revert "nixos/lib: add /home to pathsNeededForBoot" 2022-12-03 15:16:42 +02:00
Lin Jian
0c70035f24
nixos/lib: add /home to pathsNeededForBoot
Before this patch, if we:

  1. add a new user in the config,
  2. mount /home seperately,
  3. not set neededForBoot for /home,
  4. and run `nixos-rebuild boot`,

the newly added user's home will not be created after a reboot. This
is because when nixos/modules/config/update-users-groups.pl is running
in stage 2 to setup users, /home is not mounted.

This patch fixes this issue.
2022-12-01 00:49:46 +08:00
Zhaofeng Li
8b2d34fa5e test-driver: Allow configuring delay for send_{key,chars} 2022-11-19 06:55:25 +01:00
Robert Hensing
833f9d5e1f
Merge pull request #200646 from hercules-ci/options-markdown-and-errors
`nixosOptionsDoc`: add `markdownByDefault`, error handling
2022-11-14 18:58:54 +01:00
Andreas Rammhold
923e404389 nixos/lib/testing-python: remove unused with pkgs;
This with statement has no users anymore and can safely be removed.
2022-11-11 16:05:00 +01:00
Robert Hensing
b106ff14ed nixosOptionsDoc: Report in which option an error occurs 2022-11-11 06:47:30 +01:00
Robert Hensing
429ba6c714 nixosOptionsDoc: Add markdownByDefault parameter 2022-11-11 06:29:44 +01:00
Domen Kožar
e190302018 nixos options markdown: fix html escaping
\<foo\> will often be displayed like \<foo>, for example by mkdocs.

I've tested a number of markdown renderers and they render html escape
sequences fine.
2022-11-05 21:20:01 +00:00
Jörg Thalheim
951f81c0cc
nixos/systemd-unit-options: document correct wantedBy default for user units (#199007)
Until this commit, the documentation suggested `multi-user.target`
as `wantedBy` for all services.
Since `multi-user.target` is not available for user services,
propose a different default for those in the documentation.

Co-authored-by: Naïm Favier <n@monade.li>
Co-authored-by: Florian Warzecha <liketechnik@disroot.org>
2022-11-02 07:03:22 +00:00
Florian Klink
eec71f9dee
Merge pull request #186314 from ck3d/unit-add-map-control
nixos: Add option to influence override strategies in systemd units
2022-10-28 17:56:49 +02:00
Christian Kögler
6d31353099 nixos: Add unit option overrideStrategy 2022-10-27 20:19:18 +02:00
Winter
f540aeda6f nixos/make-options-doc: fix JSON generation on Darwin 2022-10-25 22:58:50 -04:00
Christian Kögler
c4e0d766b7 nixos/unitGenerator: fix generation for nspawn files 2022-10-23 15:57:55 +02:00
github-actions[bot]
c434165354
Merge master into staging-next 2022-10-21 00:05:50 +00:00
Daniel Olsen
3251123a77 nixos/lib.escapeSystemdPath: Implement the correct algorithm for escaping names in systemd units
Co-authored-by: ajs124 <git@ajs124.de>
2022-10-20 20:12:15 +02:00
Jan Tojnar
457f28f6f8 Merge branch 'master' into staging-next
; Conflicts:
;	pkgs/development/tools/codespell/default.nix

codespell 2.2.2 switched to pyproject & setuptools_scm:
https://github.com/codespell-project/codespell/pull/2523
2022-10-19 05:24:28 +02:00
Robert Hensing
14a822f72a
Merge pull request #196281 from hercules-ci/restore-nixos-test-dx
nixos: Restore test DX
2022-10-18 11:38:37 +02:00
Robert Hensing
6259b29f29
Merge pull request #194035 from Ma27/show-option-quoting
lib/options/showOption: fix quoting of attr-names that are not identifiers
2022-10-18 11:31:54 +02:00
Robert Hensing
1d9b913088 nixos/lib/testing: Delay nodes.machine.~config~ migration
Provide a window during which both solutions are valid without
warnings, in order to fight warning fatigue, and not to push 3rd
party repo maintainers to add unnecessary compat code.
2022-10-16 16:30:21 +02:00
Robert Hensing
611f247810 nixos/tests: Generalize nix-build file.nix hack to testing-python.nix 2022-10-16 16:30:20 +02:00
github-actions[bot]
3e675d06f5
Merge staging-next into staging 2022-10-09 12:02:02 +00:00
Maximilian Bosch
2480532bd1
nixos/doc: fix build
Now we even have options like
`services.listmonk.database.settings."app.notify_emails"` shown
correctly (i.e. with quotes).
2022-10-09 10:13:21 +02:00
talyz
4df4d2a8ea genJqSecretsReplacementSnippet: Allow dots in attribute names...
...and escape quotation marks and backslashes.
2022-10-09 08:12:19 +02:00
Artturin
09226fffcf nixosOptionsDoc: buildInputs -> nativeBuildInputs
to make strictDepsByDefault work
2022-10-07 19:26:22 +03:00
Robert Hensing
70ec3b9f54
Merge pull request #193498 from hercules-ci/nixos-doc-disambiguate-test-options
nixos/doc: disambiguate test option ids
2022-10-04 21:35:49 +01:00
Adam Joseph
c46bdcbaf2 nixos/lib/qemu-common.nix: set qemuSerialDevice for isMips64 2022-10-04 07:48:12 +00:00
figsoda
a1d50eecab
Merge pull request #193132 from figsoda/clean-up
treewide: clean up
2022-10-01 17:03:11 -04:00
Robert Hensing
216c5dc10d nixos/doc: Disambiguate test option ids
Changing this later on would break external links into the manual.
2022-09-29 12:41:59 +02:00
Robert Hensing
91879ce160 make-options-doc: Make optionIdPrefix configurable ("opt-") 2022-09-29 12:34:06 +02:00
Robert Hensing
aed1deab05 nixos/testing: Fix release.nix tests evaluation
Fixes the problem introduced by 12b3066aae
which caused nixos/release.nix to return the wrong attributes, while
intending to only affect nixos/lib's runTest.
This also removes callTest from the test options, because callTest is
only ever invoked by all-tests.nix.
2022-09-29 10:49:09 +02:00
Franz Pletz
417b476db7
Merge pull request #187817 from intelfx/nixos-containertarballs-fix-pixz
make-system-tarball: use `pixz -t`
2022-09-29 03:26:20 +02:00
Robert Hensing
12b3066aae lib.nixos.runTest: Return the test itself
(NB: the lib in the flake)

Correcting a mistake I made when removing the test matrix code
from https://github.com/NixOS/nixpkgs/pull/191540
2022-09-28 18:34:36 +02:00
Robert Hensing
7f0d934f9a
Merge pull request #191540 from hercules-ci/nixosTest-modular
nixosTest: make modular
2022-09-28 10:27:45 +01:00
figsoda
dc4b44a2f3 nixos/lib/make-single-disk-zfs-image.nix: remove unused binding 2022-09-26 22:19:06 -04:00
figsoda
84bc9ff334 nixos/lib/make-multi-disk-zfs-image.nix: remove unused binding 2022-09-26 22:18:55 -04:00
Robert Hensing
1c0b9c4a48 nixos/testing/network.nix: Add network config to specialisations 2022-09-24 17:42:52 +01:00
Robert Hensing
7cdc9bc340 nixos/testing: Improve interactive docs 2022-09-24 17:42:52 +01:00
Robert Hensing
666e969da0 nixos/testing: Add nodes.config backcompat to nodes module argument 2022-09-24 17:38:10 +01:00
Robert Hensing
6205d37747 nixos/testing: Improve option docs 2022-09-24 17:38:10 +01:00
Robert Hensing
4ce93fbae8 nixos/testing-python: Add interactive variant support to makeTest 2022-09-24 17:38:10 +01:00
Robert Hensing
52bfa318e8 nixos/testing: Support mypy through regular mechanisms
Rebase / forward port of 2c8bbf33fd
2022-09-24 17:38:10 +01:00
Robert Hensing
0af6e6b0e5 nixos/testing/meta.nix: Add options, some optional 2022-09-24 17:38:09 +01:00
Robert Hensing
f01fec4099 nixos/testing/network.nix: Fix specialisations onlyShorthand 2022-09-24 17:38:09 +01:00
Robert Hensing
9886db059a nixos/testing: Embrace callTest
My conception of its input was wrong. It is quite a useful construct,
even if its name is a bit weird.
2022-09-24 17:38:09 +01:00
Robert Hensing
5297d584bc nixos/lib/eval-config: Document the use of baseModules 2022-09-24 17:38:09 +01:00
Robert Hensing
57ac9dd3ae nixos/lib/build-vms.nix: Remove
It has been replaced by the modular test framework in nixos/lib/testing.

If you are looking for a way to produce a VM-test-like configuration
outside of the test framework, use the nixos/lib/testing/nixos-test-base.nix
NixOS module, possibly in combination with { _module.args.nodes = .....; }.
2022-09-24 17:38:09 +01:00
Robert Hensing
24d1d74e4e nixos/testing: Extract nixos-test-base.nix NixOS module 2022-09-24 17:38:09 +01:00
Robert Hensing
38fb09e427 testing-python.nix: Replace makeTest implementation 2022-09-24 17:38:08 +01:00
Robert Hensing
b0c781cc41 nixos/testing: Move entrypoint to nixos/lib + doc 2022-09-24 17:38:08 +01:00
Robert Hensing
9e4277b970 nixos/testing/network.nix: Fix for specialisations and specialArgs.name
Before this, it relied on being able to `imports` from the `name`
module argument, which wasn't really necessary and required a
potentially quite breaking change. We can revert that now.
2022-09-21 10:55:12 +01:00
Robert Hensing
a958a4aa00 nixos/testing: Add pkgs parameter
This parameter is for packages to use in VMs, unlike hostPkgs.
2022-09-21 10:55:11 +01:00
Robert Hensing
124b0c4abc nixos/testing/network.nix: Avoid deprecated .config 2022-09-21 10:55:11 +01:00
Robert Hensing
b3de22483c nixos/testing-python.nix: Add evalTest
This is a decomposition of the testing-python.nix and build-vms.nix
files into modules.

By refactoring the glue, we accomplish the following:

 - NixOS tests can now use `imports` and other module system features.
    - Network-wide test setup can now be reusable; example:
       - A setup with all VMs configured to use a DNS server
       - Split long, slow tests into multiple tests that import a
         common module that has most of the setup.
    - Type checking for the test arguments
    - (TBD) "generated" options reference docs
 - Aspects that had to be wired through all the glue are now in their
   own files.
    - Chief example: interactive.nix.
    - Also: network.nix

In rewriting this, I've generally stuck as close as possible to the
existing code; copying pieces of logic and rewiring them, without
changing the logic itself.

I've made two exceptions to this rule

 - Introduction of `extraDriverArgs` instead of hardcoded
   interactivity logic.

 - Incorporation of https://github.com/NixOS/nixpkgs/pull/144110
   in testScript.nix.

I might revert the latter and split it into a new commit.
2022-09-21 10:55:11 +01:00
Robert Hensing
5b52d552cb nixos/make-options-doc: Explain docbook to markdown migration 2022-09-17 13:38:23 +01:00
pennae
767485a0de lib/options: deprecate docbook text and literalDocBook
deprecate literalDocBook by adding a warning (that will not fire yet) to
its uses and other docbook literal strings by adding optional warning
message to mergeJSON.
2022-09-10 18:23:13 +02:00
Jacek Galowicz
8ae3e986a7
Merge pull request #189999 from sbruder/test-driver-formatting-closed-port
nixos/test-driver: fix formatting of closed port
2022-09-06 14:48:13 +02:00
Simon Bruder
9caf5f6ba9
nixos/test-driver: fix formatting of closed port
Co-authored-by: a-kenji <aks.kenji@protonmail.com>
2022-09-06 14:32:56 +02:00
Alyssa Ross
488412a1db
nixos/test-driver: add wait_for_unit() timeout arg
For example, the wait_for_unit() call in the Moodle test times out for
myself and others[1], so it would be good to be able to increase it to
something less likely to be hit by a test that would otherwise pass.

[1]: https://github.com/NixOS/nixpkgs/pull/177052#issue-1266336706
2022-09-02 18:06:03 +00:00
pennae
3bddcf5f90
Merge branch 'master' into option-docs-md 2022-09-01 16:10:09 +02:00
pennae
9547123258 nixos/*: convert internal option descriptions to MD
we'll have to do it eventually, may as well be now.
2022-08-31 16:32:54 +02:00
pennae
ef176dcf7e nixos/*: automatically convert option descriptions
conversions were done using https://github.com/pennae/nix-doc-munge
using (probably) rev f34e145 running

    nix-doc-munge nixos/**/*.nix
    nix-doc-munge --import nixos/**/*.nix

the tool ensures that only changes that could affect the generated
manual *but don't* are committed, other changes require manual review
and are discarded.
2022-08-31 16:32:53 +02:00
pennae
5a643387ec nixos/*: normalize more docbook for auto-conversion 2022-08-31 16:32:53 +02:00
pennae
9c3c13b50d nixos/make-options-doc: add inline roles for varname/envar
both of these render distinctly from plain literals in the manpage, and
manpages even semantically distinguish between the two.
2022-08-31 16:21:10 +02:00
Silvan Mosberger
6eb620ace7
Merge pull request #180222 from YorikSar/qemu-vm-darwin-pkgs
nixos/qemu-vm: Allow to build and run VMs on Darwin
2022-08-30 21:54:22 +02:00
pennae
65fd6f0774 nixos/make-options-doc: eat newlines in MD admonitions
leaving some newlines around after an admonition was closed causes the
newline rule to match, which in turn inserts literallayout newlines into
te xml output. that's not what we want.
2022-08-27 19:18:29 +02:00
Yuriy Taraday
e0fd131310 nixos/qemu: Add QEMU arguments for HVF on Darwin
Only support the same architechture for now, no emulation.

Related to #108984.
2022-08-25 17:00:14 +04:00
Yuriy Taraday
8a25d56a6d nixos/qemu: nixpkgs-fmt nixos/lib/qemu-common.nix 2022-08-25 17:00:11 +04:00
Ivan Shapovalov
ac18c55b56 make-system-tarball: use pixz -t
Pass `-t` to pixz to prevent it from appending an index to the end of
the uncompressed stream, confusing tools such as `machinectl import-tar`.

Fixes: #187816
2022-08-22 05:28:36 +03:00
Christian Kögler
5e7662d848
Merge pull request #167692 from peperunas/onsuccess
nixos/systemd: Add onSuccess module option for units
2022-08-19 09:17:14 +02:00
Robert Schütz
243053e521 python310Packages.mistune: 0.8.4 -> 2.0.4
Remove mistune_0_8 because it's insecure.
2022-08-15 06:53:01 +00:00
pennae
af98bacbe0 Revert "nixos/docs: cache mergeJSON md conversion on baseOptionsJSON"
This reverts commit 52b0ad17e3.

we only needed this because mergeJSON was slow, but in the interim we
found a better solution to the slowness.
2022-08-05 17:13:47 +02:00
pennae
645cfa59ac nixos/make-option-docs: add xref support to markdown conversion 2022-08-03 22:01:14 +02:00
pennae
7a091b2686 nixos/make-options-doc: reuse markdown instance in mergeJSON
this doesn't construct a new (expensive) parser for every option, making
rendering about 30x faster.
2022-08-03 22:00:24 +02:00
pennae
6b13dd0e9e
Merge pull request #183491 from pennae/automatic-md-conversions
treewide: automatically md-convert option descriptions
2022-08-02 02:15:30 +02:00
Silvan Mosberger
fe2c9ae814
Merge pull request #177012 from hercules-ci/nixos-modular-system
`flake.lib.nixosSystem`: Allow `nixpkgs.system` to be set modularly; improve error message
2022-08-01 15:05:16 +02:00
Alyssa Ross
5330c0a1af
treewide: use isAarch where appropriate 2022-07-30 16:18:27 +00:00
pennae
52b0ad17e3 nixos/docs: cache mergeJSON md conversion on baseOptionsJSON
with ever more options being markdown rather than docbook the conversion
time is starting to become a significant factor of doc build time.
luckily we can pre-convert all nixos option docs to MD and cache the
result of this conversion, then merge the already-converted json file
with user option docs. we leave options.json unconverted to keep it as
close to the actual nix code as possible.
2022-07-28 23:20:02 +02:00
pennae
18be724a58 nixos/make-options-doc: give MD conversion error locations
during docs conversion it can be very useful to know exactly *where* the
error the script complained about is. the name of the option should be
sufficient since option merging is rather rare, and won't merge doc
attributes anyway.
2022-07-28 20:03:02 +02:00