This script needs to support being run both as part of a `fetchgit`
derivation and as a standalone, command-line tool.
The use of `$NIX_BUILD_TOP` only works when used in `fetchgit` but not when
invoked as a standalone tool. Instead we try to respect `$NETRC` so that
the command-line invocation behaves more like standard tools and the
`fetchgit` derivation can explicitly set `$NETRC` when `netrcPhase` is used
to avoid all ambiguity.
This fixes a regression introduced in https://github.com/NixOS/nixpkgs/pull/256628
which broke fetching with private = true through a netrc file.
Tested locally with a really special github enterprise.
When specifying the `builder` attribute in `stdenv.mkDerivation`, this
will be effectively transformed into
builtins.derivation {
builder = stdenv.shell;
args = [ "-e" builder ];
}
This also means that `default-builder.sh` is never sourced and as a
result it's not guaranteed that `$NIX_ATTRS_SH_FILE` is set to a correct
location[1].
Also, we need to source `.attrs.sh` to source `$stdenv`. So, the
following is done now:
* If `$NIX_ATTRS_SH_FILE` points to a correct location, then use it.
Directly using `.attrs.sh` is problematic for `nix-shell(1)` usage
(see previous commit for more context), so prefer the environment
variable if possible.
* Otherwise, if `.attrs.sh` exists, then use it. See [1] for when this
can happen.
* If neither applies, it can be assumed that `__structuredAttrs` is
turned off and thus nothing needs to be done.
[1] It's possible that it doesn't exist at all - in case of Nix 2.3 or
it can point to a wrong location on older Nix versions with a bug in
`__structuredAttrs`.
nix-prefetch-git is either run as part of a build, usually sandboxed,
or outside a build, unsandboxed, to prefetch something that will later
be used in a build. It's important that the latter use produces
hashes that can be reproduced by the former.
One way that they can differ is if the user's git config does
something that changes the result of git clone. I ran into this,
because my global git config automatically enables git-lfs, whereas
nix-prefetch-git otherwise only uses git-lfs if specifically
requested. This led to very confusing hash mismatches.
Passing a (multi-line) string was deprecated in #200082 in favour of
list of strings, but still supported (with warning). Now, enforce use of
list of strings.
The `sparseCheckout` argument allows the user to specify directories or
patterns of files, which Git uses to filter files it should check-out.
Git expects a multi-line string on stdin ("newline-delimited list", see
`git-sparse-checkout(1)`), but within nixpkgs it is more consistent to
use a list of strings instead. The list elements are joined to a
multi-line string only before passing it to the builder script.
A deprecation warning is emitted if a (multi-line) string is passed to
`sparseCheckout`, but for the time being it is still accepted.
The fetchgit function in nixpkgs sets the leaveDotGit argument to true
if deepClone is set to true. nix-prefetch-git did behave differently. It
would not assume --leave-dotGit if --deepClone is specified. With this
change the inconsistency is addressed by assuming --leave-dotGit if
--deepClone is specified.
When maintainers override stages of `fetchgit' (e.g. `postPatch`) it
is very easy for them to accidentally leak the outpath-hash of their
current `stdenv` into `fetchgit''s output, and therefore into the
value they paste into `sha256`.
This is a problem, because the resulting expression will break
whenever any change is made to `stdenv` or when anybody attempts to
build the expression on a different platform than the one used by the
original maintainer.
Almost as much of a problem is the fact that CI **does not catch**
these problems. The `fetchgit` is run only once, then its output goes
into cachix, and all future builds (hydra, CI, ofborg) pull from
cachix.
Let's offer maintainers the option to check that they aren't making
this mistake, by passing through `allowedRequisites`. The default
value is `null`, but it might be worth changing that at some point in
the future.
It is also sometimes difficult to communicate to package maintainers
why their expression is problematic. Having `allowedRequisites`
passed through makes it easier to do this: "look, when I switch on
`allowedRequisites` your package breaks; are you sure you meant to
hardcode the hash today's `x86_64-linux.stdenv` into your expression?`
For an example use case, see https://github.com/NixOS/nixpkgs/pull/171223
The issue above is part of a larger problem with nixpkgs infra: there
large parts of cachix cannot be reproduced easily if they are lost.
Once something ends goes into cachix, we never ever again reverify the
procedure by which it was placed into cachix.
a67950f20b added `url` attribute
from `fetchurl` and therefore also from `fetchzip`.
We previously relied on `url` from fetchgit-based fetchers
to find the repo URL but now it will just return tarballs
in the case of `fetchFrom{GitHub,GitLab}`.
Let’s add an attribute to `fetch{git,FromGitHub,FromGitLab}`
to expose a repo URL consistently.
