Some programs (e.g. nvfetcher) parse the output of nix-prefetch-git as
JSON. However, there is a pushd/popd command in nix-prefetch-git that
outputs the directory name, breaking the structure of the result JSON.
We suppressed it with `>/dev/null`.
Passingly fixes a mixuse of tab and spaces.
This script needs to support being run both as part of a `fetchgit`
derivation and as a standalone, command-line tool.
The use of `$NIX_BUILD_TOP` only works when used in `fetchgit` but not when
invoked as a standalone tool. Instead we try to respect `$NETRC` so that
the command-line invocation behaves more like standard tools and the
`fetchgit` derivation can explicitly set `$NETRC` when `netrcPhase` is used
to avoid all ambiguity.
This fixes a regression introduced in https://github.com/NixOS/nixpkgs/pull/256628
which broke fetching with private = true through a netrc file.
Tested locally with a really special github enterprise.
nix-prefetch-git is either run as part of a build, usually sandboxed,
or outside a build, unsandboxed, to prefetch something that will later
be used in a build. It's important that the latter use produces
hashes that can be reproduced by the former.
One way that they can differ is if the user's git config does
something that changes the result of git clone. I ran into this,
because my global git config automatically enables git-lfs, whereas
nix-prefetch-git otherwise only uses git-lfs if specifically
requested. This led to very confusing hash mismatches.
The fetchgit function in nixpkgs sets the leaveDotGit argument to true
if deepClone is set to true. nix-prefetch-git did behave differently. It
would not assume --leave-dotGit if --deepClone is specified. With this
change the inconsistency is addressed by assuming --leave-dotGit if
--deepClone is specified.
It's hugely inefficient as we can't use shallow cloning (--depth=1).
This has been tested and adapted for quite a few hosts fetchgit is used on in
Nixpkgs. For those where fetching the hash directly doesn't work (most notably
git.savannah.gnu.org), we simply fall back to the old method.
The reason for this change is simply to avoid the following messages
that are unnecessary and can be confusing (and these messages will be
repeated for each submodule):
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
With this change the behaviour remains unchanged (apart from the
suppressed "warning" in the console output of course) and therefore this
doesn't cause any hashes to change and by default nix-prefetch-git uses
the "fetchgit" branch anyway (branchName can be set to override the
default):
Switched to a new branch 'fetchgit'
For that reason the initial branch name doesn't matter anyway and since
we're not relying on / hardcoding "master" we could simply switch to
"main" (which seems most common nowadays). See [0] for more details on
why this wouldn't break anything.
However, since the initial branch name doesn't matter and to avoid any
additional risks it was "decided" to keep using "master" (s. #113313).
[0]: https://github.com/NixOS/nixpkgs/pull/113313#issuecomment-780589516
`git repack` and `git gc` sometimes print “Nothing new to pack.”
to stdout, which breaks redirecting output to JSON file.
Let’s move the stdout of all git calls where it is not used to stderr
so that we still receive the info but it does not pollute our output.
Especially as a new user it is a much better experience to receive a
proper help response to `-h`. Currently passing `-h` will cause some
runtime error with the `git remote` error help being shown. Not very
helpful.
It doesn't hurt to be a bit more user friendly in this case.
The script would parse the output of `git submodule status` but
didn't handle paths with spaces in them. This would result in the
following error when trying to determine the URL of the submodule:
error: key does not contain a section: .url
Summary:
According to git-submodule manpage,
"git submodule status" prefixes the hash with a '-' if it is not
initialized, and other chars in other circumstances.
(this is consistent on the various git versions tested)
nix-prefetch-git runs "git submodule init" which does you'd think,
but apparently despite this earlier versions of git before 2.16
would still give the hash the '-' suffix.
In particular this is the behavior when using 2.15 and 2.14.1
from the nixos-17.09 and nixos-17.03 channels respectively.
The script then used awk to drop the first char of the first field
which does the wrong thing when there is no prefix emitted:
while there is a space character before the hash, this is not
part of the field and so we ended up eating the first character
of the hash.
To fix this in a way that also works with the previous behavior,
this commit instead uses awk to grab the hash field
and uses tr to delete any '-' chars should they be present.
This seems to work in my testing, and for example can now
successfully fetch the source for "nginxModules.brotli"
where previously it would generate an error:
fatal: '22564a95d9ab58865a096b8d9f7324ea5f2e03e' is not a commit and a branch 'fetchgit' cannot be created from it
(we dropped a '2' from the beginning of the hash)
