This change adds a flag --slow to hydra-report.sh get-report which
causes it to fetch the cheap evaluation overview endpoint (which only
contains build ids and meta data). The gathered information is then used
to request each build's status individually instead of in bulk which is
very slow, but useful as a last resort if the bulk endpoint times out.
Since every failure in the jobset means one request to get the log when
generating the list of newly broken packages, we need to add an option
to disable log requesting in case a lot of new breakage needs to be
entered.
If we want to push only one branch, we'll have to specify branch and
remote explicitly. Pushing to origin doesn't work for everyone, since
some of us have a origin remote that can't be pushed to. Using plain
`git push` has the problem that it'll try pushing all checked out
branchs which fails e.g. if some branches (staging, staging-next, …) are
behind their remote counterparts.
The solution is to require everyone to configure a per branch pushRemote
for haskell-updates which will then be used by merge-and-open-pr.sh.
* update.py: introduce subparsers for plugin updaters
This is preliminary work to help create more powerful plugin updaters.
Namely I would like to be able to "just add" plugins without refreshing
the older ones (helpful when github temporarily removes a user from
github due to automated bot detection).
Also concerning the lua updater, we pin some of the dependencies, and I
would like to be able to unpin the package without editing the csv
(coming in later PRs).
* doc/updaters: update command to update editor plugins
including vim, kakoune and lua packages
Co-authored-by: figsoda
cabal-install 3.10 has some quirky new logic for config, cache, …
directory discovery. We reimplement this in this simple bash script,
additionally respecting the CABAL_DIR environment variable.
This update adds support for $CABAL_DIR as well as the new
$XDG_CACHE_HOME location of the hackage db.
Since we maintain hackage-db, having the latest version always is nice
even though it has more reverse dependencies than the other libraries we
maintain.
When DEBUG is defined, the script just prints the URL's without actually
checking whether they're already cached or downloading/uploading anything.
That got broken because connecting to S3 now fails fast. This PR makes sure
we skip connecting to s3 in DEBUG mode.
Add a newtype for a package name and a package set. This is less for
correctness, and more just to make the code a little easier to read
through without having to keep in mind what each Text refers to.
This script is heavily based on the script used to update all python
libraries at
pkgs/development/interpreters/python/update-python-libraries/update-python-libraries.py
The Octave Packages' website uses YAML as their basis, so we must
reformat to use YAML instead of JSON.
the script can output a list of sed commands to create the order it
expects to find. this was mainly useful for initially sorting the list,
but we'll keep it here for later reference.
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
nixpkgs:trunk also builds aarch64-darwin these days, so this forces our
hand a little bit. We can still refuse to care about failures _too_
much, but at least we will stop merging as big a rebuilds as we are
currently.
Previously when packages that required the git fetcher were updated, we
would wrongly rely on `nix-prefetch-url`, which would reliable break the
hash.
Instead we need to use `nix-prefetch-git` to determine the proper hash,
when the relevant attributes are present.
`ghc-pkg list` tells us everything hackage2nix needs to know. In the
past the core-packages list and compiler setting in hackage2nix was
maintained manually which inevitably leads to it being forgot once in a
while – this will then mess with flag resolution when generating the
package set in some cases. Luckily, we can just let a simple derivation
do this for us.
Resolves#202621.
Since we now have a versioned configuration-ghc-*.nix file for GHC HEAD,
we don't need to add a super special case to the package set logic in
test-configurations.nix anymore. We can just create a versioned
attribute for the ghcHEAD package set (which is not exposed) and keep
using the normal discovery logic.
The only tricky bit is that GHC HEAD's configuration file is named after
the GHC release that will be branched off from it, so a little bit of
arithmetic is involved.
This will allow tests.pkg-config.defaultPkgConfigPackages to run on
hydra without breaking the tarball job.
Regarding the use of null, I'll quote 473ac96 which does lib.hydraJob.
By allowing null, we allow code to avoid filterAttrs, improving
laziness in real world use cases.
Specifically, this strategy prevents infinite recursion errors,
performance issues and possibly other errors that are unrelated to
the user's code.
The Haskell Hydra report generator
(`maintainers/scripts/haskell/hydra-report.hs`) uses this
`maintainer-handles.nix` script for generating a mapping of email
addresses to GitHub handles.
This `maintainer-handles.nix` script is necessary because the Haskell
Hydra report generator gets Hydra job status info as input, but needs to
ping users on GitHub. Hydra job status info only contains user emails (not
GitHub handles). So the `maintainer-handles.nix` script is necessary
for looking up GitHub handles from email addresses.
This commit fixes the `maintainers-handles.nix` code to ignore
maintainers that don't have email addresses. The code was originally
assuming that all maintainers have email addresses, but there was
recently a maintainer added without an email address.
Move the manpage-to-URL mapping to `doc/manpage-urls.json` so that we can
reuse that file elsewhere, and generate the `link-manpages.lua` filter from
that file.
Also modify the Pandoc filter so that it doesn't wrap manpages that are
already inside a link.
Keeping a Lua filter is essential for speed: a Python filter would
increase the runtime `md-to-db.sh` from ~20s to ~30s (but Python is not
to blame; marshalling Pandoc types to and from JSON is a costly operation).
Parsing in Lua seems tedious, so I went with the Nix way.
- use `restrict-eval` so that we're not affected by the user's environment
- use jq instead of the horrible echo+sed hack
The second point also fixes the indentation before each line to be two
spaces instead of one, so I set it back to one space to avoid a diff.
instead of running luarocks with the requested interpreter, we can ask any interpreter to search for a specific version via --lua-version. It avoids building/running different luarocks-nix just for that.
use tagged release of nvim-cmp
Transform exit handlers of the form
trap cleanup EXIT [INT] [TERM] [QUIT] [HUP] [ERR]
(where cleanup is idempotent)
to
trap cleanup EXIT
This fixes a common bash antipattern.
Each of the above signals causes the script to exit. For each signal,
bash first handles the signal by running `cleanup` and then runs
`cleanup` again when handling EXIT.
(Exception: `vscode/*` prevents the second run of `cleanup` by removing
the trap in cleanup`).
Simplify the cleanup logic by just trapping exit, which is always run
when the script exits due to any of the above signals.
Note: In case of borgbackup, the exit handler is not idempotent, but just
trapping EXIT guarantees that it's only run once.
this script can be used to attempt an automatic conversion of option
docs for most modules. it'll also show a diff of options.json before and
after the changes, which should be a good form for checking for unwanted
changes. we specifically show a json diff rather than an xml diff
because newline changes in json are "\n" added are removed, and those
are easier for diff tools to pick out and show in a meaningful way for
this process.
it does *not* check for incorrectly applied changes though, those aren't
easy enough to do automatically for this script.
These are new dependencies for SILE, currently needed by the flake in the upstream repo and will be needed by the stable package in this repo on the next release
I've been working for a long time towards automatic nix dependencies for
neovim plugins (using luarocks rockspecs to discover the said
dependencies).
This is an initial commit to help me complete the missing bits.
buildNeovimPluginFrom2Nix is right now a placeholder which helps me test
in my fork a version that does a flat install of luarocks.
the vim updater will now check for attributes with the same name in the lua package set,
if that's the case the script will generate buildNeovimPluginFrom2Nix.
Required for Prosody 0.12.0.
I first tried to run the bare
./maintainers/scripts/update-luarocks-packages, however it did break a
fair share of Lua packages.
In the end, I:
1. Added the luaunbound entry to luarocks-packages.csv
2. Run the update-luprocks-packages.
3. Cherry picked the luaunbound entry
4. Reverted the rest of lua-modules/generated-packages.nix to what we
currently have in Nixpkgs.
5. Injected the native unbound library via the lua-modules overrides.
Accept either Nightly or LTS as the solver configuration variable in the
script. The Stackage version is now considered a tuple of solver and
version, allowing the script to handle updates and switches between
solvers gracefully.
Tested updating Nightly and updating from Nightly to LTS.
We have generally shipped the latest cabal-install version. Stackage has
re-added cabal-install recently which caused cabal-install to get
downgraded to 3.4 to match the Cabal version shipped by GHC 9.0.2. This
commit reverts that change.
usage
'./maintainers/scripts/remove-old-aliases.py --year 2018 --file ./pkgs/top-level/aliases.nix'
first the matched lines are converted to throws with the comment
'Converted to throw $CURRENT_DATE'
then the throws older than the passed date are removed.
Some updaters like `unstableGitUpdater` rely solely `UPDATE_NIX_ATTR_PATH`
environment variable to find out package attribute so they all have
the same `passthru.updateScript`. This means we cannot rely
on that attribute to filter out aliased packages because it would
consider all packages using `unstableGitUpdater` the same.
We need to use different criterion to distinguish them. `meta.position`
is a reasonable candidate but we cannot always utilize it since
it might not be available or it might be shared among multiple packages
(e.g. when using custom mkDerivation wrapper).
Combination of the two attributes should distinguish most cases.
Fixes: https://github.com/NixOS/nixpkgs/issues/112583
I find it more practical (can run update.py --github-token=$(pass gihtub-token)" rather than putting token in ENV). Also makes it more discoverable. I introduced a FetchConfig variable to pass editor agnostic configuration.
Before 46c68ad both "@" and " as " could be used in the same line like
the following:
vimwiki/vimwiki@dev as vimwiki-dev
After 46c68ad this gives an error due to the split URI still erroneously
including the " as [name]" at the end, due to splitting from the wrong
variable.
configuration-nix.nix uses builtins.intersectAttrs to not any overrides
for packages not present in `super` (presumably for use outside of
nixpkgs?). To accomodate it, we pass an attribute set with every
attribute of haskellPackages, but set to `null` as `super`, and — while
we're at it — a fix point as `self`.
While being able to test them is neat (on x86_64-linux they work very
well, actually), we usually don't want to do this, since the set is
only (recommended to be) used to bootstrap GHC. Consequently there is
almost no binary cache and testing them mostly leads to unenlightening
and seemingly endless compilation.
The added nix expression allows maintainers to check for regressions in
the configuration overlays employed by haskellPackages and friends. The
reasoning behind this is that, if we add an override for something, it
should also build. To test this fact, we extract all attributes touched
by a configuration and obtain all relevant derivations corresponding to
it which can then be thrown into nix-build --keep-going.
I've been using this expression to verify configuration-ghc-9.2.x.nix
for a week or so which works quite well. The amount of stale overrides
in other configuration makes it a bit more painful for other use cases
at the moment.
The "0.5.0-1" rockspec on luarocks has a bug, resulting in it pulling
the current git master version, which is what we have effectively been
using.
Given that 0.5.0-1 is the latest release, is 6 years old, and that there
have been some bug fixes since then, we do actually want to be using the
git master version, but we also want to be using the correct rockspec
(particularly as alt-getopt has been replaced by argparse in the `moon`
binary).
Avoiding passthru itself was the right direction, but not a
complete solution. Passthru attributes typically do not contain
dependencies, but rather extra paths that are not relevant to
the build itself.
Whether we want to include all (passthru) test dependencies this
way can be debated, but removing them makes this expression more
robust.
When updating a package using nix-update-script with `--argstr commit true`,
update.nix would not detect the changes because nix-update would stage them
and `git diff` would be empty.
We now detect both staged and unstaged changes to handle this use case.
- luarocks-nix: bumped to pass args too, `package_X = callPackage ...`.
It allows to remove the annoying `with self`.
This new version disables tests (because broken) and now emits derivation
with a callPackage in front.
- replaced X.override with lib.overrideLuarocks, it should be used
whenever buildInputs/extraVariables is changed since it impacts the
generated luarocks config.
Once structured attributes are in, it will be easier to have the
luarocks config generated by a hook and we probably will be able to
replace all overrideLuarocks by overrideAttrs.
* maintainers/scripts/rebuild-amount.sh: report parallelism setting
This slightly helps discovering parallelism support.
* maintainers/scripts/rebuild-amount.sh: add basic example
It's not clear if script should be used against HEAD or HEAD^.
Let's have a copyable example.
* maintainers/scripts/rebuild-amount.sh: convert tabs to four spaces
One notable change is here-document conversion.
Before:
<TAB>cat <<-FOO
<TAB>...
<TAB>FOO
After:
<sp><sp><sp><sp>cat <<FOO
<sp><sp><sp><sp>...
FOO
Note seemingly misaligned 'FOO': '-FOO' understands leading tabs,
but not spaces.
I noticed this minor grammar mistake when running update.nix, and then
while grepping to find the source I noticed we had it a few times in
Nixpkgs. Just as easy to fix treewide as it was to fix the one
occurrence I noticed.
Upstream is working on some major deprecations, being on a HEAD commit
will not be a good plan going forward.
Remove double `busted` in `checkInputs` - keep only 1.
If the script is executed in a CI (like in github actions, with the
install-nix-action) it fails. This changes fixes that use case. In that
scenario, you should provide the --no-commit flag.
