Commit Graph

145 Commits

Author SHA1 Message Date
Jörg Thalheim
954d995394
nix-prefetch-git: escape string fields properly
json requires certain characters to be escaped in strings.
2016-12-05 12:32:14 +01:00
Eelco Dolstra
e3a873479e
Remove fetchMD5warn
Deprecation warnings should not be used in Nixpkgs because they spam
innocent "nix-env -qa" users with (in this case) dozens of messages
that they can't do anything about.

This also reverts commit 2ca8833383.
2016-11-30 15:14:30 +01:00
Alexey Shmalko
a80cacd3ec
fetchFromGitHub: add fetchSubmodules option
This commit extends fetchFromGitHub with ability to fetch GitHub
repositories with submodules, so we can use the function consistently
with all GitHub repositories.

Note it doesn't change the previous behavior.
2016-11-18 13:04:26 +02:00
Michael Raskin
f603dc11a6 fetch*: print a trace warning about md5 deprecation 2016-10-09 16:19:04 +02:00
Profpatsch
61462c94e6 lib/fetchers.nix: factor out impure proxy vars (#18702)
Apparently everyone just copied those variables, instead of creating a
library constant for them. Some even removed the comment. -.-
2016-09-17 21:50:01 +02:00
Bjørn Forsman
68205d0ade nix-prefetch-git: unbreak JSON output
This fixes a regression caused by commit f56ab9e
("nix-prefetch-git: Include the date in the machine-readable [...]")
where a couple of directory paths printed by pushd/popd appeared before
the JSON output on stdout (thus breaking it). Fix it by redirecting the
extraneous output to /dev/null.

Reported by Michael Alan Dorman <mdorman@ironicdesign.com>.
2016-07-04 14:39:47 +02:00
David Grayson
f56ab9e5e4 nix-prefetch-git: Include the date in the machine-readable output on
stdout, in strict ISO 8601 format.

This will be helpful for automatically updating fetchgit expressions
and the dates in version numbers associated with them.
2016-07-03 11:19:13 +01:00
Benno Fünfstück
9ff91371d0 nix-prefetch-git: fix bash evaluation order dependency 2016-06-15 22:54:35 +02:00
zimbatm
a42b7faaec nix-prefetch-git: shellcheck fixes
Used shellcheck (https://github.com/koalaman/shellcheck) to validate
the script and fixed any resulting escaping and ambiguity issues.
2016-06-12 13:45:20 +01:00
Eelco Dolstra
a5fa7c25cb Merge pull request #15469 from NixOS/fetchgit
fetchgit: remove only .git folder
2016-05-16 16:44:55 +02:00
Domen Kožar
64a072e357 fetchgit: remove only .git
Source of this change goes back to 2009 and original version of
fetchgit at 205fb0c87e.

The nondeterminism is really caused by changing .git so leave other
files alone as they might be interesting.

Note: this causes a hash mismatch with Hydra's version of Git Plugin
which we should fix to comply.
2016-05-15 00:24:04 +01:00
Guido Zgraggen
6ea0ae58af nix-prefetch-git: create parent directories 2016-04-22 16:51:49 -07:00
zimbatm
40e9dff04a nix-prefetch-git: fix url_to_name heuristic
The function wasn't checking that *all* of the characters where
[a-z0-9]. Fixes #13921
2016-03-23 11:22:51 +00:00
Ryan Trinkle
be30ba8e0e nix-prefetch-scripts: make nix-prefetch-git report fetchSubmodules in its JSON output
Previously, nix-prefetch-git would report the same JSON whether submodules were being fetched or not; with this change, the --fetch-submodules option will cause the JSON output to include "fetchSubmodules": true, so that fetchgit (builtins.fromJSON (builtins.readFile ./path/to/output.json)) will work.
2016-03-21 23:26:18 -04:00
zimbatm
90de261f33 nix-prefetch-git: change the default output to JSON
As discussed on the mailing list. The nix output was short-lived so it's
probably okay to change it.
2016-02-29 22:47:16 +00:00
Tim Cuthbertson
21547a61ba nix-prefetch-git: print out valid nix expression; make --quiet very quiet 2016-02-27 21:26:35 +11:00
Tim Cuthbertson
456cbb29d9 nix-prefetch-git: add --quiet flag and minor cleanup 2016-02-27 16:56:38 +11:00
Graham Christensen
60f354dfb8 nix-prefetch-git: output base32 hash so output matches nix-build errors
It turns out hashFormat has never been set.
2016-02-22 10:50:27 -06:00
zimbatm
02f5a01c19 nix-prefetch-git: use fetchgit's naming heuristic
This commit fixes #6651.

Before this change the `nix-prefetch-git` script would use a different store
name than nix's `fetchgit` function. Because of that it was not possible to
use `nix-prefetch-git` as a way to pre-populate the store (for example when
the user it using private git dependencies that needs access to the ssh agent)
2016-02-13 14:39:44 +00:00
Arnaud Spiwack
07f7ecdcce Fix usage message in nix-prefetch-git
The comment related to the `deepClone` and `no-deepClone` options was
misleading as these options have no relation with submodules, but on the
the depth in `git clone --depth n`.
2016-01-26 16:09:52 +01:00
Vladimír Čunát
0957359568 Merge branch 'staging' 2016-01-22 13:48:35 +01:00
Rok Garbas
36fc03edb6 fetchgit: import impure GIT_PROXY_COMMAND and SOCKS_SERVER, fixes #8605
These environment variables allow using fetchgit with git:// URLs using
the SOCKS proxy technique described in 'Using Git with a SOCKS proxy':

  http://www.patthoyts.tk/blog/using-git-with-socks-proxy.html

Briefly, GIT_PROXY_COMMAND is set to a script which invokes connect[1],
which reads SOCKS_PROXY, which might be pointing to a local instance of
'ssh -D'.

[1] pkgs/tools/networking/connect
2016-01-20 02:18:22 +01:00
Domen Kožar
9c571a98ed Merge pull request #10998 from andrewrynhard/master
Add help flag to nix-prefetch-git
2016-01-19 12:01:55 +01:00
Peter Simons
b89514eced fetchgit: follow up to 2cf7069b7d
If "fetcher" is a string, then Nix will execute it with bash already, so
the additional bash argument in that string was redundant and apparently
causes trouble on non-Linux platforms.

Hopefully fixes https://github.com/NixOS/nixpkgs/issues/11496.
2015-12-06 15:06:02 +01:00
Vladimír Čunát
263fd55d4b Merge recent staging built on Hydra
http://hydra.nixos.org/eval/1231884
Only Darwin jobs seem to be queued now,
but we can't afford to wait for that single build slave.
2015-12-05 11:11:51 +01:00
Peter Simons
2cf7069b7d fetchgit: call in-repository script with bash explicitly
The script's shebang depends on /usr/bin/env, which we don't have in chroot
environments. This patch remedies the fallout from ade9f7167d, which
fixed https://github.com/NixOS/nixpkgs/issues/11284.
2015-11-29 15:43:56 +01:00
Peter Simons
ade9f7167d nix-prefetch-git: make sure the script is interpreted by bash
Fixes https://github.com/NixOS/nixpkgs/issues/11284.
2015-11-27 11:17:50 +01:00
Danny Wilson
d50d14d55f Use the nix provided shell.
This is guaranteed to be bash, SmartOS has ksh by
 default and doesn't work.
2015-11-16 17:20:11 +01:00
Andrew Rynhard
42ff8ad780 Add help flag to nix-prefetch-git 2015-11-12 15:10:56 -08:00
Nikolay Amiantov
f5deff5137 nix-prefetch-git: pull all tags from remote when checkouting by revision
Close #9790.
This fixes checkouting for a nasty combination:

1. To be checkouted is a revision which corresponds to tag in a form "<tag>^{}".
2. This revision is not fetched by default.
2015-09-20 18:24:34 +02:00
William A. Kennington III
ffd0539eba cacert: store ca-bundle.crt in $out/etc/ssl/certs instead of $out 2015-06-05 13:00:52 -07:00
William A. Kennington III
14c1e0fa1f Fix ca-bundle paths 2015-05-29 14:03:34 -07:00
Georges Dubus
7d67efa3f2 Add support for building cargo'ed Rust programs 2015-04-21 19:46:29 +02:00
Ricardo M. Correia
c55c7e1c1e fetchgit: Add support for specifying branch name
This is useful when `leaveDotGit = true` and some other derivation
expects some branch name to exist.

Previously, `nix-prefetch-git` always created a branch with a
hard-coded name (`fetchgit`).
2015-04-21 19:46:29 +02:00
Peter Simons
5d02f0e854 fetchgit: add 'deepClone' argument to disable shallow fetching
This patch resolves https://github.com/NixOS/nixpkgs/issues/6395. Deep
cloning is useful in combination with 'leaveDotGit' for builds that want
to run "git describe" to obtain a proper version string, etc., like the
'haskellngPackages.cabal2nix' package does.
2015-03-10 13:14:00 +01:00
Bjørn Forsman
06f5651ef5 nix-prefetch-git: print commit date (close #6522)
The commit date can be used as a version number in packages that don't
have proper releases.
2015-02-24 23:02:59 +01:00
Vladimír Čunát
77d66ed834 fetchgit: improve name detection, discard nix-1.8 check
The name detection didn't work for e.g. http://git.suckless.org/sinit/.
I tested the tarball builds now.

@shlevy claimed nixpkgs requires nix-1.8 features anyway,
so the additional check with message were superfluous.
2015-01-13 19:45:31 +01:00
Shea Levy
a8603605aa fetchgit: give output a nicer name
Instead of git-export, we get the basename of the repo, plus the
shortrev if the commit-ish is a rev.
2015-01-01 10:03:45 -05:00
William A. Kennington III
4fcbe096b6 fetchgit: Simplify submodule url discovery 2014-12-04 15:17:58 -08:00
Vincent Laporte
cded306f82 nix-prefetch-git: allow dots in submodule names 2014-11-15 18:37:58 +00:00
Edward Tjörnhammar
4030ab3ba4 add parens to de ambiguify 2014-11-10 06:53:24 +01:00
Edward Tjörnhammar
91c7ffc498 Revert "Revert "initialized git repo should use the set http_proxy""
This reverts commit f8a833cfb1.
This makes sure the check returns a zero code.

See discussion on https://github.com/NixOS/nixpkgs/commit/5af576f
2014-11-09 21:27:26 +01:00
Luca Bruno
f8a833cfb1 Revert "initialized git repo should use the set http_proxy"
This reverts commit 5af576ff7f.

See discussion on https://github.com/NixOS/nixpkgs/commit/5af576f
2014-11-09 13:50:48 +01:00
Edward Tjörnhammar
5af576ff7f initialized git repo should use the set http_proxy 2014-11-07 11:50:30 +01:00
Bjørn Forsman
96cacf01bf nix-prefetch-git: run single-threaded 'git repack'
Without this, the generated pack files are non-deterministic.

I didn't notice this issue in my earlier testing, because my test repo
had too few commits for the thread scheduling to take effect. (Test repo
had about 10 commits.)
2014-11-03 22:52:57 +01:00
Bjørn Forsman
53614cf1a7 nix-prefetch-git: fix determinism with leaveDotGit
Add more files to the delete list:

 * .git/FETCH_HEAD
 * .git/ORIG_HEAD
 * .git/refs/remotes/origin/HEAD
 * .git/config

Further, remove all remote branches, remove tags not reachable from the
given 'rev', do a full repack and then garbage collect unreferenced
objects.

According to my testing, the result is fully deterministic. As in "any
change done to the upstream repo, ahead of 'rev', will not affect the
hash of the resulting 'clone'". Even changing the clone URL will not
change the output hash, because .git/config is removed.

A new version of git can of course change store format, but that's
unavoidable.

For big repositories, the repack operation may be a bit heavy. But as
far as I can see there is no cheaper way to determinism.
2014-11-02 13:15:33 +01:00
Bjørn Forsman
faaa8a6acf nix-prefetch-git: remove unneeded semicolons
Shell isn't Perl ;-)
2014-11-01 15:17:11 +01:00
Bjørn Forsman
8ec8358724 nix-prefetch-git: whitespace cleanup
Use 4 spaces per indent level instead of a mix of 2 spaces, 4 spaces and
hardtab. (According to nixpkgs coding style for shell scripts.)
2014-11-01 13:47:37 +01:00
Michael Raskin
46cdc2e017 Allow git checkouts to have custom name 2014-09-03 21:54:25 +04:00
aszlig
5e7a1cf955
build-support: Fix nix-prefetch-* on OS X.
Fixes a regression on OS X introduced by f83af95.

Don't use --tmpdir for mktemp, because that flag doesn't exist on OS X.
However, using -t is deprecated in GNU coreutils, so as suggested by
@ip1981 we're now using parameter expansion on ${TMPDIR:-/tmp} to
provide /tmp as a fallback if TMPDIR is not set and use it instead.

Also use this approach for nix-prefetch-cvs now in order to stay
consistent.

Reported-by: Vladimir Kirillov <proger@wilab.org.ua>
Tested-by: Igor Pashev <pashev.igor@gmail.com>
Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-08-27 11:22:34 +02:00
aszlig
f83af95f8a
build-support: Use mktemp -d in nix-prefetch-*.
Instead of relying on $$ to not collide with an existing path.

Quoting the Bash manual about $$:

> Expands to the process ID of the shell. In a () subshell, it expands
> to the process ID of the current shell, not the subshell.

So, this is different from $BASHPID:

> Expands to the process ID of the current bash process. This differs
> from $$ under certain circumstances, such as subshells that do not
> require bash to be re-initialized.

But even $BASHPID is prone to race conditions if the process IDs wrap
around, so to be on the safe side, we're using mktemp here.

Closes #3784.

Signed-off-by: aszlig <aszlig@redmoonstudios.org>
2014-08-25 15:28:55 +02:00
Benno Fünfstück
ce3e86702f prefetch-git: output human-readable rev to stderr
that way, the stdout stays compatible with nix-prefetch-{bzr,svn,hg}
2014-08-17 23:00:03 +02:00
Georges Dubus
ca0b0a68e8 Fixed deterministicness of fetchgit with leaveDotGit
The shebang of .git/hooks depended the git's bash, which made the result
depend of that bash's path.
2014-08-10 16:25:29 +02:00
Georges Dubus
f4fbcddd44 fetchgit: make deterministic with leaveDotGit (close #3392)
There was a few files containing timestamp, so we now remove them.

It shouldn't be a problem for logs. However, index might be. Anyway,
that's better than nothing.
2014-08-08 21:51:29 +02:00
Shea Levy
b969e9340b Separate fetchgitPrivate into a separate file 2014-07-16 18:05:56 -04:00
Paul Colomiets
c1e24abfef Print output of git describe in nix-prefetch-git
I hope it will help make git-packages' versions much nicer

It's usually only useful in --deepClone is also specified.
2014-06-25 01:12:10 +03:00
Benno Fünfstück
be25ce8f58 nix-prefetch-git: fix printing of revision
If the user explictly gives a ref such as "refs/heads/master", `git
rev-parse` failed because we only checked out the `fetchgit`
branch. Now, we also try `git rev-parse fetchgit` if the first call
fails, which fixes the issue.
2014-06-11 14:09:58 +02:00
Simon Hengel
26fb26e331 nix-prefetch-git: Print git revision 2014-05-16 15:22:28 +08:00
ambrop7@gmail.com
3a765a7309 fetchgit: Implement option to not check out submodules. 2014-03-25 20:03:55 +01:00
Eelco Dolstra
a8ded9d5d3 fetchgit: Require a content hash
Without this, the result will not be a fixed-output derivation and
won't work in general.
2014-02-18 19:13:07 +01:00
Vladimír Čunát
161d029d7d preferLocalBuild: set to true for wrappers and fetchers 2014-02-10 21:04:17 +01:00
William A. Kennington III
42229f1040 nix-prefetch-git: Convert relative submodule URLS to absolute URLS
nix-prefetch-git does not convert relative submodule urls into absolute
urls based on the parent's origin. This patch adds support for
repositories which are using the relative url syntax.
2014-01-10 17:28:35 -06:00
William A. Kennington III
49b36a3a77 Fix nix-prefetch-git
The nix-prefect git script was broken when trying to parse certain
groups of submodules. This patch fixes the url detection for submodule
repositories to use the more reliable `git config` commands.
2013-11-24 14:03:34 +01:00
Kirill Elagin
1be62b87fe Handle invalid references 2012-06-20 23:28:53 -04:00
Kirill Elagin
ba766ebcf4 Do not use old way of setting url and rev
This was breaking `nix-prefetch-git --url smth --ref smth` (when
expected hash is not specified)
2012-06-20 23:28:52 -04:00
Nicolas Pierron
9cd0f7ed09 nix-prefetch-git: remove .git in the cloned directory. (Fix issue 116)
svn path=/nixpkgs/trunk/; revision=29077
2011-09-07 10:02:17 +00:00
Nicolas Pierron
0608f1a9b5 nix-prefetch-git: Display usage message if missing builder arguments are
missing.

svn path=/nixpkgs/trunk/; revision=29076
2011-09-07 10:02:14 +00:00
Nicolas Pierron
5f5a47fc76 nix-prefetch-git: Remove missleading option name.
svn path=/nixpkgs/trunk/; revision=29075
2011-09-07 10:02:12 +00:00
Michael Raskin
f9728ee0db Fix a typo
svn path=/nixpkgs/trunk/; revision=28871
2011-08-29 10:06:12 +00:00
Nicolas Pierron
d528cba5b8 fetchgit: Handle https.
svn path=/nixpkgs/trunk/; revision=28857
2011-08-28 16:03:14 +00:00
Nicolas Pierron
9e53a20f50 Replace fetchgit default builder by a wrapper around nix-prefect-git.
svn path=/nixpkgs/trunk/; revision=28709
2011-08-20 14:29:57 +00:00
Peter Simons
c78c1288c3 fetchgit: strip trailing whitespace
svn path=/nixpkgs/trunk/; revision=27862
2011-07-20 22:56:44 +00:00
Peter Simons
0b1bf668cd fetchgit: added support for fetching sub-modules
svn path=/nixpkgs/trunk/; revision=27860
2011-07-20 22:56:35 +00:00
Florian Friesdorf
8e606466b3 fetchgit builder - correctly quote "$rev"
svn path=/nixpkgs/trunk/; revision=27531
2011-06-22 23:33:41 +00:00
Florian Friesdorf
c8f12bd8be fetchgit builder comment: can handle remotes without HEAD
thx civodul for bringing it up and testing

svn path=/nixpkgs/trunk/; revision=27523
2011-06-22 21:01:36 +00:00
Florian Friesdorf
e50e84daa1 fetchgit report fetch progress
svn path=/nixpkgs/trunk/; revision=27522
2011-06-22 19:34:32 +00:00
Florian Friesdorf
ee731974e9 fetchgit builder to handle broken remotes without a HEAD
svn path=/nixpkgs/trunk/; revision=27521
2011-06-22 19:34:27 +00:00
Florian Friesdorf
a8c3649dd4 fetchgit builder rewrite to handle more cases
The old builder failed for:
- remote HEAD != master
- local repositories

svn path=/nixpkgs/trunk/; revision=26984
2011-04-27 03:00:47 +00:00
Ludovic Courtès
3369508fa9 nix-prefetch-git: Add rudimentary TopGit support.
svn path=/nixpkgs/trunk/; revision=26305
2011-03-14 22:00:20 +00:00
Ludovic Courtès
8ccb0fda33 fetchgit: Track all remote branches when `$rev' is non-empty.
svn path=/nixpkgs/trunk/; revision=21892
2010-05-19 21:27:01 +00:00
Rob Vermaas
13417770a3 applying patches provided by griswold
svn path=/nixpkgs/trunk/; revision=19707
2010-01-27 12:12:35 +00:00
Ludovic Courtès
45aa4e2d7f nix-prefetch-git: Add support for $NIX_PREFETCH_GIT_DEEP_CLONE.
svn path=/nixpkgs/trunk/; revision=19648
2010-01-25 13:23:08 +00:00
Nicolas Pierron
ba79b38503 nix-prefetch-git:
* Optimized download size in case of references.
* Add support for submodules.

svn path=/nixpkgs/trunk/; revision=19513
2010-01-18 08:28:38 +00:00
Rob Vermaas
4dca2c8b12 echo to stderr in stead of stdout. else hydra cannot read hash/path
svn path=/nixpkgs/trunk/; revision=18448
2009-11-18 19:10:24 +00:00
Ludovic Courtès
4f20056bca nix-prefetch-git: Default to SHA256.
svn path=/nixpkgs/trunk/; revision=18443
2009-11-18 16:39:09 +00:00
Ludovic Courtès
39476057a2 nix-prefetch-git: Support $NIX_PREFETCH_GIT_LEAVE_DOT_GIT.
svn path=/nixpkgs/trunk/; revision=18442
2009-11-18 16:39:06 +00:00
Ludovic Courtès
3124661215 nix-prefetch-git: Remove `--depth' argument since it's risky.
The `--depth' argument asks Git to fetch the last revisions of the given
repo on *any* branch, which is often useless.

Thanks to Lluís Battle for clarifying this.

svn path=/nixpkgs/trunk/; revision=18438
2009-11-18 16:24:31 +00:00
Rob Vermaas
66a537f774 redirect stdout of git calls to stderr to allow git to work in hydra as input
svn path=/nixpkgs/trunk/; revision=18393
2009-11-17 15:50:20 +00:00
Marc Weber
edf78fcb6a some fetchgit documentation
svn path=/nixpkgs/trunk/; revision=18283
2009-11-08 03:02:10 +00:00
Lluís Batlle i Rossell
896dc42662 Forgot some 'depth' regarding fetchgit still.
svn path=/nixpkgs/trunk/; revision=18279
2009-11-08 01:59:50 +00:00
Lluís Batlle i Rossell
f1ee14bfaf Removing fetchgit --depth and its usage. It does not provide much advantages...
I think it takes the recent N commits into the repository, which says very little,
even for wanting master/HEAD.

svn path=/nixpkgs/trunk/; revision=18277
2009-11-08 01:51:20 +00:00
Michael Raskin
cd04c58fe2 Allow not-that-shallow git fetches; enable it for uzbl
svn path=/nixpkgs/trunk/; revision=17980
2009-10-27 19:52:01 +00:00
Michael Raskin
edd56d707a Update source updater to support fetchgit; add aufs2 using that method. Unfortunately, AUFS2 seems to require some strict conditions on kernel options and version..
svn path=/nixpkgs/trunk/; revision=16983
2009-09-08 09:35:21 +00:00
Marc Weber
c59fb90f62 make fetchgit and nix-prefetch-git use --depth 1 to receive only a shallow clone cutting history
svn path=/nixpkgs/trunk/; revision=16047
2009-06-26 00:56:54 +00:00
Rob Vermaas
205fb0c87e * fetchgit and nix-prefetch-git
svn path=/nixpkgs/trunk/; revision=16035
2009-06-24 12:48:01 +00:00