Commit Graph

336 Commits

Author SHA1 Message Date
Ilan Joselevich
ebafd551d7
nixos/hercules-ci-agent: sync module with upstream 2023-05-22 15:05:16 +03:00
Yaya
2681d4ff12 nixos/gitlab-runner: Add gitlab team members to maintainers 2023-05-20 20:38:49 +00:00
figsoda
701bcdbead nixos: fix typos 2023-05-19 22:31:04 -04:00
Martin Weinelt
1c80c494ca
Merge pull request #228422 from mweinelt/gitea-actions-runner-module
nixos/gitea-actions-runner: init
2023-05-12 17:54:16 +02:00
Martin Weinelt
1c963cea48
nixos/gitea-actions-runner: init 2023-05-06 17:34:15 +02:00
Jörg Thalheim
301d6bb9c2 nixos/hercules-ci-agent: increase stack limit
see https://github.com/hercules-ci/hercules-ci-agent/pull/451 for context
2023-05-02 09:53:41 +02:00
Artturi
b83db86a9e
Merge pull request #222080 from Stunkymonkey/nixos-optionalString 2023-04-20 16:07:30 +03:00
Jörg Thalheim
90b750456e buildbot: move out of python3.pkgs
By moving it out we can start apply overrides to python dependendencies
starting with sqlalchemy.

This fixes the build with the current version.
2023-04-11 12:38:58 +02:00
Felix Buehler
327b0cff7a treewide: use more lib.optionalString 2023-04-07 13:38:33 +02:00
Bruno BELANYI
6048912d8b nixos/woodpecker-*: add myself as maintainer 2023-03-23 21:33:21 +00:00
Bruno BELANYI
67de7d105e nixos/woodpecker-agents: per-agent 'enable' option 2023-03-23 21:33:20 +00:00
Bruno BELANYI
cd116db45e nixos/woodpecker-agents: bind network files
Otherwise the agent might experience trouble with DNS resolution [1].

[1]: https://github.com/woodpecker-ci/plugin-git/issues/65
2023-03-23 14:16:45 +00:00
Bruno BELANYI
eb3bea6359 nixos/woodpecker-agents: simplify 'extraGroups' handling 2023-03-23 14:16:45 +00:00
Bruno BELANYI
c3afdb82db nixos/woodpecker-agents: use list for environment files 2023-03-23 14:16:44 +00:00
Bruno BELANYI
e4f5f1b718 nixos/woodpecker: refactor to multi-agents setup
The module file has been renamed from `agent.nix` to `agents.nix` to
mirror the change.
2023-03-23 12:47:47 +00:00
Janik H
dafedbbba6 nixos/woodpecker: init 2023-03-14 20:31:39 +01:00
Rick van Schijndel
8d8f5ede91
Merge pull request #216451 from Mindavi/hydra/systemd-target
nixos/hydra: wait for network-online before evaluator start
2023-03-09 20:53:21 +01:00
Rick van Schijndel
60446dda1f nixos/hydra: wait for network-online before evaluator start
This prevents issues I'm seeing with the hydra I'm running on my laptop.
Every time I reboot it I see eval errors like this:

```
error fetching latest change from git repo at `https://github.com/nixos/nixpkgs.git':
fatal: unable to access 'https://github.com/nixos/nixpkgs.git/': Could not resolve host: github.com
```

This is because the evaluator already starts before the network is
actually online. It should wait until the network is fully online before
starting evaluation to prevent evaluation errors like above.
2023-03-07 21:58:23 +01:00
Ilan Joselevich
edff57cff4
Merge pull request #218455 from veehaitch/gh-runner-builddotnet
github-runner: switch to `buildDotnetModule`
2023-03-06 19:11:06 +02:00
Stefan Frijters
2af041ab44 nixos/gitlab-runner: do not pull in Docker if gitlab-runner-clear-docker-cache is disabled
Only create the service if the option is enabled and if any docker executors exist.
2023-02-27 22:24:14 +01:00
Vincent Haupert
137db83090 nixos/github-runners: use Runner.Listener directly for registration
Nothing the script `config.sh` does prior to the final call to
`Runner.Listener configure` is relevant for the systemd service.
Particularly, we don't need (nor want) any of the artifacts the `env.sh`
script creates.
2023-02-26 15:39:19 +01:00
Aaron Andersen
d8c0a9204a
Merge pull request #217834 from yaxitech/gh-runner-workdir-clean
nixos/github-runners: clean `workDir` as root
2023-02-26 08:05:05 -05:00
Profpatsch
d316c28a30 modules/github-runner: Improve tokenFile docs
We’ve been having trouble figuring out which kind of token to use and
why our setup would break every few system updates.

This should clarify which options there are, and which ones lead to
better results.

Ideally there would be a manual section that has a step-by-step guide
on how to set up the github runner, with screenshots and everything.
2023-02-23 12:37:30 +01:00
Vincent Haupert
36949b9718 nixos/github-runners: clean workDir as root
Purge contents of `workDir` as root to also allow the removal of files
marked as read-only. It is easy to create read-only files in `workDir`,
e.g., by copying files from the Nix store.
2023-02-23 09:05:27 +01:00
Karel Kočí
116872aedd
nixos/gitlab-runner: fix shell syntax preventing build
The build fails when configFile is provided because of shellcheck
detects missing quotes here.
2023-02-13 13:20:26 +01:00
Vincent Haupert
57a517db35 nixos/github-runner{,s}: add @veehaitch and @newam as maintainers 2023-01-09 09:37:00 +01:00
Vincent Haupert
b634dbe576 nixos/github-runners: fix format of service file 2023-01-09 09:36:59 +01:00
Vincent Haupert
66dbf9b199 nixos/github-runners: use mkMerge for serviceConfig
The `serviceOverrides` module option is commonly used to loosen the
systemd unit's hardening. This commit merges the `serviceConfig` with
`mkMerge` instead of using the update operator `//` which discards all
existing values on conflict. To avoid a breaking change which requires
defining each option with a higher priority (e.g., through `mkForce`),
this commit prefixes hardening values with `mkDefault`.

Notable exceptions are list hardening options which use `mkBefore`
instead of `mkDefault`. This allows for easy extension of the existing
settings. Resetting redefinitions are still possible through `mkForce`.
2023-01-09 09:36:04 +01:00
Aaron Andersen
4769274f53
Merge pull request #205526 from newAM/github-runner-oom
nixos/github-runner: add workDir option
2023-01-08 19:01:44 -05:00
Maciej Pasternacki
c3ec444a61 nixos/gitlab-runner: fix style issues flagged by statix 2023-01-08 17:37:43 +01:00
Maciej Pasternacki
d5cb89b024 nixos/gitlab-runner: fix problems introduced by last #209716 2023-01-08 17:36:45 +01:00
Matthieu Coudron
c61f554c1a modules.gitlab-runner: accept space in names
when you register a runner with spaces in its name (possible if you use 'description' option) then the runners never get unregistered because our bash scripts assume no space in names.

This solves the issue

Retreiving the fullname of the runner via `gitlab-runner list` got surprisingly hard between lazy-capture issues and `gitlab-runner list` displaying invisible (CSI) characters that break the regex etc.
Which is why I fell back on the pseudo-json format.

This PR adds the hash in the name, which allows to keep both the
stateless aspect of the module while allowing for a freeform name.

I found using bash associative arrays easier to use/debug than the current
approach.
2023-01-05 11:43:32 +01:00
Alex Martens
307730fb72 nixos/github-runner: add workDir option 2023-01-03 19:54:25 -08:00
figsoda
6bb0dbf91f nixos: fix typos 2022-12-17 19:31:14 -05:00
Daniel Nagy
ad866e565d
treewide: switch to port type for nixos modules 2022-12-08 00:00:00 +01:00
Daniel Nagy
dbe8182e74
treewide: switch to port type for nixos modules 2022-12-01 22:30:00 +01:00
figsoda
1b9b057d92 nixos/buildbot: syntax clean up 2022-11-30 17:31:28 -05:00
Vincent Haupert
847774470e nixos/github-runners: restart service if exit code signals a retryable error
On some occasions, the GitHub runner service encounters errors which are
deemed retryable but result in the runner's termination. To signal a
retryable error, the runner exits with status code 2:

https://github.com/actions/runner/blob/40ed7f8/src/Runner.Common/Constants.cs#L146

To account for that behavior, this commit sets
`RestartForceExitStatus=2` which results in a service restart regardless
of using an ephemeral runner or not.
2022-11-10 11:06:04 -05:00
Bjørn Forsman
fcf81f91a3 nixos/jenkins-job-builder: better defaults for accessUser/accessTokenFile
The new defaults allows jenkins-job-builder to reload the configuration
out-of-the-box, whereas the previous defaults required users to manually
reload/restart jenkins, or configure accessUser/accessTokenFile
themselves.

(If `extraJavaOptions = [ "-Djenkins.install.runSetupWizard=false" ]`
then the initial admin user is *not* created and you have to use JCasC
or something else to bootstrap.)
2022-10-30 19:01:48 +01:00
Sandro
b05729432a
Merge pull request #196060 from risicle/ris-gocd-22.2.0 2022-10-27 01:07:34 +02:00
ajs124
3dd1098f45
Merge pull request #195567 from helsinki-systems/feat/jenkins-jdk17
nixos/jenkins: jdk11 -> jdk17
2022-10-21 22:59:03 +02:00
Vincent Haupert
ea8cf2e486 nixos/github-runners: support fine-grained personal access tokens
Add support for GitHub's new fine-grained personal access tokens [1]. As
opposed to the classic PATs, those start with `github_pat_` instead of
`ghp_`.

Make sure to use a token which has read and write access to the
"Administration" resource group [2] to allow for registrations of new
runners.

[1] https://github.blog/2022-10-18-introducing-fine-grained-personal-access-tokens-for-github/

[2] https://docs.github.com/en/rest/overview/permissions-required-for-github-apps#administration
2022-10-19 13:50:34 +02:00
Domen Kožar
ff27dc3a09
Merge pull request #176691 from codedownio/multiple-github-runners
GitHub runners: configurable user, environment, and service overrides + multiple runners
2022-10-19 13:38:23 +02:00
Tom McLaughlin
c2cc9aeafd Use config name by default, falling back to attr name 2022-10-19 03:33:30 -07:00
Tom McLaughlin
5221e7af04 Add comments to explain about the name defaults 2022-10-17 00:04:52 -07:00
Tom McLaughlin
2c099d1a14 Set runner name to attr name for github-runners.${name} 2022-10-17 00:01:04 -07:00
Robert Scott
4f54ef201d gocd-server: 19.3.0 -> 22.2.0 2022-10-15 13:34:50 +01:00
Tom McLaughlin
fc8fdb03a0 Try simpler github-runner.nix 2022-10-13 19:54:36 -06:00
Tom McLaughlin
69d9538b34
Update nixos/modules/services/continuous-integration/github-runners.nix
Co-authored-by: Vincent Haupert <mail@vincent-haupert.de>
2022-10-13 18:53:25 -06:00
Tom McLaughlin
cf1b952988
Update nixos/modules/services/continuous-integration/github-runner.nix
Co-authored-by: Vincent Haupert <mail@vincent-haupert.de>
2022-10-13 18:49:02 -06:00
Pascal Bach
1ca4c178dd
Merge pull request #195488 from KoviRobi/gitlab-runner-clear-docker-cache
Gitlab runner clear docker cache
2022-10-12 07:56:46 +02:00
ajs124
18a17d11ff nixos/jenkins: jdk11 -> jdk17
see https://www.jenkins.io/doc/upgrade-guide/2.361/
we'll need to do this eventually, not sure when the best point in time is
2022-10-11 21:16:37 +02:00
Tom McLaughlin
0b67081ad8 Cherry-pick 499748b 2022-10-11 06:10:11 -06:00
Tom McLaughlin
9a7f38040b Fix user type 2022-10-11 06:04:25 -06:00
Tom McLaughlin
b744fee880 Re-add DynamicUser = true per review discussion 2022-10-11 06:04:25 -06:00
Tom McLaughlin
b3de807a6a Update descriptions to use lib.mdDoc 2022-10-11 06:04:25 -06:00
Tom McLaughlin
327e05c382 Get rid of DynamicUser flag 2022-10-11 06:04:25 -06:00
Tom McLaughlin
f13759e21f Fix a deprecated types.string -> types.str 2022-10-11 06:04:25 -06:00
Tom McLaughlin
998083f2ad github-runner: configurable user, environment, service overrides + multiple runners 2022-10-11 06:04:21 -06:00
Martin Weinelt
294201004f Merge remote-tracking branch 'origin/master' into staging-next 2022-10-10 21:45:18 +02:00
Robert Kovacsics
c8eae7a526 nixos/gitlab-runner: Add gitlab-runner.clear-docker-cache service 2022-10-10 18:17:19 +01:00
Vincent Haupert
941c79b620 nixos/github-runner: fix bugs related to InaccessiblePaths=
This commit fixes two bugs:

1) When starting a github-runner for the very first time, the
   unconfigure script did not copy the `tokenFile` to the state
   directory. This case just was not handled so far. As a result, the
   runner could not configure. The unit did, however, fail even before
   as the state token file is configured as inaccessible for the service
   through `InaccessiblePaths=`. As the given path did not exist in the
   described case, setting up the unit's namespacing failed.

2) Similarly, the `tokenFile` is also marked as not accessible to the
   service user. There are, however, cases where other namespacing
   options make the files inaccessible even before `InaccessiblePaths=`
   kicks in; thus, they appear as non existing and cause the namespacing
   to fail yet again. Prefixing the entry with a `-` causes Systemd to
   ignore the entry if it cannot find it. This is the behavior we want.

I also took fixing those bugs as a chance to refactor the unconfigure
script to make it easier to follow.
2022-10-08 01:32:55 +02:00
Artturin
6910a4eea0 treewide: makeWrapper to nativeBuildInputs
this should be all of them other than the failed splices
found with nixpkgs-lint
2022-09-26 17:53:26 +03:00
pennae
d98322834b nixos/*: fix docbook deprecation notices
mostly no rendering changes except in buildkite, which used markdown
where docbook was expected without marking up its markdown.
2022-09-10 18:23:13 +02:00
pennae
9547123258 nixos/*: convert internal option descriptions to MD
we'll have to do it eventually, may as well be now.
2022-08-31 16:32:54 +02:00
pennae
ef176dcf7e nixos/*: automatically convert option descriptions
conversions were done using https://github.com/pennae/nix-doc-munge
using (probably) rev f34e145 running

    nix-doc-munge nixos/**/*.nix
    nix-doc-munge --import nixos/**/*.nix

the tool ensures that only changes that could affect the generated
manual *but don't* are committed, other changes require manual review
and are discarded.
2022-08-31 16:32:53 +02:00
pennae
51a11254a7 nixos/*: literalDocBook -> literalMD
no change to rendered output
2022-08-27 19:18:29 +02:00
pennae
6039648c50 nixos/*: automatically convert option docs 2022-08-19 22:40:58 +02:00
pennae
7e7d68a250 nixos/*: mark pre-existing markdown descriptions as mdDoc 2022-08-19 22:40:58 +02:00
pennae
b51f8036c2 nixos/*: use properly indented strings for option docs
using regular strings works well for docbook because docbook is not as
whitespace-sensitive as markdown. markdown would render all of these as
code blocks when given the chance.
2022-08-19 22:40:58 +02:00
pennae
b7327e966b nixos/*: normalize links with #TEXT=href
the xslt pass will re-add this text, and not having href and text be
equal is required to automatically migrate these links to markdown.
2022-08-19 22:40:58 +02:00
Vincent Haupert
786f72c32e nixos/github-runner: start Runner.Listener directly in ExecStart=
The `runsvc.sh` script wraps a JavaScript script which starts
`Runner.Listener` and also handles failures. This has the downside that
the service _always_ exits with status code 0, i.e., success. This
causes frequent service restarts when running in ephemeral mode with a
faulty config as Systemd always sees a success exit status. To prevent
this, this commit changes the service config to call `Runner.Listener`
directly. The JavaScript wrapper stops the process with a SIGINT, hence,
the Systemd unit now sends a SIGINT to stop the service.
2022-08-15 13:13:58 -04:00
Vincent Haupert
987a4b4231 nixos/github-runner: add support for ephemeral runners
Adds the module option `ephemeral`. If set to true, configures the
runner registration with the `--ephemeral` option. This causes the
runner to exit after processing a single job, to de-register itself, and
to delete its configuration. Afterward, systemd restarts the service
which triggers a new ephemeral registration with a clean state.
2022-08-15 13:13:58 -04:00
Vincent Haupert
3f075e5bb1 nixos/github-runner: add PAT support
This commit introduces support for runner registrations through a
personal access token (PAT). To use a PAT instead of a registration
token, place an appropriately scoped PAT in `tokenFile`. If the file
contains a PAT, the configuration script queries a new runner
registration token. Using a runner registration token directly continues
to work as before.
2022-08-15 13:13:58 -04:00
Vincent Haupert
65542a6348 nixos/github-runner: use state instead of runtime dir as RUNNER_ROOT
Using the runtime directory as `RUNNER_ROOT` is wrong. We should always
use the state directory like we already do when invoking the runner
configure script. Otherwise, the runner constructs the wrong path for
some files (.credentials, .runner, ...).
2022-08-15 13:13:58 -04:00
pennae
93c57a9884
Merge pull request #185056 from pennae/option-docs-md
nixos/*: more option docs conversions
2022-08-05 17:36:49 +02:00
Sandro
3125eef570
Merge pull request #182713 from SuperSandro2000/hydra-runuser-2 2022-08-04 12:08:44 +02:00
pennae
61e93df189 nixos/*: automatically convert option docs to MD
once again using nix-doc-munge (69d080323a)
2022-08-03 22:46:41 +02:00
pennae
3aebb4a2be nixos/*: normalize link format
make (almost) all links appear on only a single line, with no
unnecessary whitespace, using double quotes for attributes. this lets us
automatically convert them to markdown easily.

the few remaining links are extremely long link in a gnome module, we'll
come back to those at a later date.
2022-08-03 21:57:46 +02:00
pennae
6b13dd0e9e
Merge pull request #183491 from pennae/automatic-md-conversions
treewide: automatically md-convert option descriptions
2022-08-02 02:15:30 +02:00
pennae
2e751c0772 treewide: automatically md-convert option descriptions
the conversion procedure is simple:

 - find all things that look like options, ie calls to either `mkOption`
   or `lib.mkOption` that take an attrset. remember the attrset as the
   option
 - for all options, find a `description` attribute who's value is not a
   call to `mdDoc` or `lib.mdDoc`
 - textually convert the entire value of the attribute to MD with a few
   simple regexes (the set from mdize-module.sh)
 - if the change produced a change in the manual output, discard
 - if the change kept the manual unchanged, add some text to the
   description to make sure we've actually found an option. if the
   manual changes this time, keep the converted description

this procedure converts 80% of nixos options to markdown. around 2000
options remain to be inspected, but most of those fail the "does not
change the manual output check": currently the MD conversion process
does not faithfully convert docbook tags like <code> and <package>, so
any option using such tags will not be converted at all.
2022-07-30 15:16:34 +02:00
misuzu
9b1db3db0e nixos/gitlab-runner: undeprecate configFile option 2022-07-28 17:16:55 +02:00
misuzu
b4028126f1 nixos/gitlab-runner: add settings option 2022-07-28 17:16:55 +02:00
Sandro
87c66cbb56
nixos/hydra: fix runuser in init 2022-07-24 18:05:10 +02:00
Bjørn Forsman
16108ff74a nixos/jenkins-job-builder: set serviceConfig.Type = "oneshot"
This change allows detecting configuration errors during
switch-to-configuration instead of them being reported asynchronously
*after* switch-to-configuration has exited.

(And update the NixOS test accordingly.)
2022-07-23 13:30:53 +02:00
Vincent Haupert
539b61ea37 nixos/github-runner: fix capset syscall filtering
capset(2) is a single system call, not a set of multiple system calls.
2022-07-21 16:08:15 +02:00
Alex Martens
c34749dd63 nixos/github-runner: fix systemd defaults for common workflows 2022-07-17 22:02:57 -07:00
Bjørn Forsman
0080a93cdf nixos/jenkins-job-builder: create secret file with umask 0077
IOW, don't make it world readable.
2022-07-17 15:24:48 +02:00
Bjørn Forsman
50eaf82b6f nixos/jenkins-job-builder: fix jenkins authentication
The current authentication code is broken against newer jenkins:

  jenkins-job-builder-start[1257]: Asking Jenkins to reload config
  jenkins-start[789]: 2022-07-12 14:34:31.148+0000 [id=17]        WARNING hudson.security.csrf.CrumbFilter#doFilter: Found invalid crumb 31e96e52938b51f099a61df9505a4427cb9dca7e35192216755659032a4151df. If you are calling this URL with a script, please use the API Token instead. More information: https://www.jenkins.io/redirect/crumb-cannot-be-used-for-script
  jenkins-start[789]: 2022-07-12 14:34:31.160+0000 [id=17]        WARNING hudson.security.csrf.CrumbFilter#doFilter: No valid crumb was included in request for /reload by admin. Returning 403.
  jenkins-job-builder-start[1357]: curl: (22) The requested URL returned error: 403

Fix it by using `jenkins-cli` instead of messing with `curl`.

This rewrite also prevents leaking the password in process listings. (We
could probably do it without `replace-secret`, assuming `printf` is a
shell built-in, but this implementation should be safe even with shells
not having a built-in `printf`.)

Ref https://github.com/NixOS/nixpkgs/issues/156400.
2022-07-16 12:30:41 +02:00
Sandro Jäckel
90761632ae
nixos/hydra: use runuser like hydra flake 2022-07-07 22:37:51 +02:00
Sebastien Bourdeauducq
6cecab9cd8 hydra: create runcommand-logs directory 2022-06-06 11:36:58 -07:00
Rick van Schijndel
3a7f7c1153 hydra-unstable -> hydra_unstable 2022-05-10 23:32:02 +02:00
Ivan Kozik
59a76614f3 treewide: chown user:group instead of user.group to fix warnings from coreutils 9.1 2022-05-05 22:05:18 +00:00
Matthieu Coudron
c1e318d0c5 services.gitlab-runner: support runner description
The description for the runner in the UI is by default sthg like
"npm_nixos_d0544ed48909" i.e., the name of the attribute.

I wanted to have a more user-friendly description and added a
description to the service.
Seems like gitlab-runner doesn't like having both fields set:
"Cannot use two forms of the same flag: description name"
so I used one or the other.
2022-04-26 11:43:46 +02:00
Lassulus
c29e523824
Merge pull request #163030 from newAM/github-runner-harden
nixos/github-runner: systemd service hardening
2022-04-20 16:30:23 +01:00
P. R. d. O
eeb3206a29 nixos/jenkinsSlave: install java
Installs Java into the Jenkins agent and allows specifying the JDK/JRE package to use. This is necessary as Jenkins verifies if the agent contains Java installed through the java -fullversion command, which if not, the connection will fail.
2022-04-01 22:11:18 +02:00
Alex Martens
334b30c464 nixos/github-runner: systemd service hardening 2022-04-01 12:22:10 -07:00
Lucas Hoffmann
f3bda2f2c0 docs: replace defunct git://github.com urls with https:// 2022-03-25 11:21:42 +01:00
Profpatsch
75b08e3624 modules/github-runner: Improve description of url
We spent a whole afternoon debugging this, because upstream has very
bad software quality and the error messages were incredibly
misleading.

So let’s document it for the sanity of other people.

Btw, I think the implementation of our module is pretty brittle,
especially the part about diffing tokens to check whether they
changed. We should rather just request a new builder registration
every time, it’s not that much overhead, and always set `replace` so
it is idempotent.
2022-03-11 10:26:39 +01:00
Vincent Haupert
7c2319905d nixos/github-runner: use --disableupdate flag
Use the new `--disableupdate` configure flag instead of our patch.
2022-01-28 09:46:12 +01:00