3 ms·
I'm the developer of [git-annex](http://git-annex.branchable.com/ http://git-annex.branchable.com/) which is AFAIK the closest eqivalant for git. I only learned
by joeyh 15y ago
I'm the developer of [git-annex](http://git-annex.branchable.com/ http://git-annex.branchable.com/) which is AFAIK the closest eqivalant for git. I only learned about the mercurial bfiles extension (which became the large files extension) after designing git-annex.
The designs are obviously similar at a high level, but one important difference is that git-annex tracks, in a fully distributed manner, which git repositories currently contain the content of a particular large file. The mercurial extension is, AFAIK, rather more centralized; while it can transfer large file content from multiple stores it can't, for example, transfer a large file from a nearby client that happens to currently have a copy, which git-annex can do (if a remote is set up). This location tracking also allows me to have offline archival disks whose content is tracked with git-annex. If I ask for an archived file, git-annex knows which disks I can put online to retrieve it.
Another difference is that the mercurial extension always makes available all the large files for the currently checked out tree. git-annex allows a tree to be checked out with large files not present (they appear as broken symlinks); you can ask it to populate the tree and it retrieves the files as a separate step. This is both more complex and more flexible. For example, I have a git repository containing a few terabytes of data. It's checked out on my laptop's 30 gb SSD. Only the files I'm currently using are present on my laptop, but I can still manage all the other files, reorganizing them, requesting ones I need, etc.
git-annex also has support for special remotes, which are not git repositories, but in which large files are stored. So large files can be stored in Amazon S3 (or the Internet Archive S3), in a bup repository, or downloaded from arbitrary urls on the web.
Content in special remotes are tracked the same as other remotes. This lets me do things like this (the first file is one of my Grandfather's engineering drawings of Panama Canal locks):
joey@gnu:~/lib/big/raw/eckberg_panama>git annex whereis img-0124.png
whereis img-0124.png (5 copies)
5863d8c0-d9a9-11df-adb2-af51e6559a49 -- turtle (turtle internal drive)
7e55d8d0-81ab-11e0-acc9-bfb671110037 -- archive-panama (internet archive http://www.archive.org/details/panama-canal-lock-design-papers)
905a3a64-4149-11e0-8b3f-97b9501cdcd3 -- passport (passport usb drive 1 terabyte)
9b22e786-dff4-11df-8b4c-731a6178061c -- archive-leech (archive-6 sata drive)
f4c185e2-da3e-11df-a198-e70f2c123f40 -- archive (archive-5 sata drive)
ok
joey@gnu:~/lib/big/raw/eckberg_panama>git annex get img-0124.png --from archive-panama
get img-0124.png (from archive-panama...) ok
I'm hopeful that git will grow some internal hooks for managing large files that will improve git-annex and also allow others to develop extensions that, perhaps, behave more like the mercurial largefiles extension. I recently attended the GitTogether and this stuff was a major topic of discussion.
- baq 15y agodoes git-annex work on windows? if not, do you have plans to port it? it's an important problem and it'd suck to have a solution which doesn't work on all major platforms.
- joeyh 15y agoNot so far. Discussion is here: http://git-annex.branchable.com/install/#comment-4637ce9b32abecf6ebf94c75f907f351 http://git-annex.branchable.com/install/#comment-4637ce9b32a... It's fine on OSX though.
- naveen99 15y agohave you seen: https://github.com/apenwarr/bup https://github.com/apenwarr/bup "file backup system based on the git packfile format. Capable of doing fast incremental backups of virtual machine images. "
- joeyh 15y agoSure, git-annex can use bup as a special remote. This way you get bup's nice properties of storing big stuff in git with binary deltas, with git-annex's nice properties of a normal-looking file in a git clone.