4 ms·

git-annex. It supports multiple external remotes, including USB drives, Glacier[1], bup and ddar[2]. You can keep files on any combination of the external remot

by rlpb 12y ago

git-annex. It supports multiple external remotes, including USB drives, Glacier[1], bup and ddar[2]. You can keep files on any combination of the external remotes for redundancy and cost management. It keeps track of what is where, so you don't have to. The only catch is that you have to make sure to back up the (metadata-only) git repository itself also, and maintaining this backup on anything that is not a direct filesystem is painful. [1] Through my tool, glacier-cli, for which git-annex has native support. [2] I wrote ddar and the git-annex support for it.

icebraining 12y ago
+1 for git annex. For the git repo, if you don't have privacy concerns, you can just push to a private bitbucket repository. But I just had a script with: git clone /my/repo /tmp/backup-git-annex tar cvf /tmp/backup-git-annex.tar /tmp/backup-git-annex That way, you're left with a single tar file to backup.
rcthompson 12y ago
So git-annex now has support for efficiently deduplicating backends? Might be time for me to give it another look.
- rlpb 12y ago
  For "local" storage as supported by bup and ddar, yes. Generally for external remotes: not yet, AFAIK. http://git-annex.branchable.com/design/assistant/deltas/ http://git-annex.branchable.com/design/assistant/deltas/ has some design thoughts, so I think it is planned.
sytse 12y ago
GitLab CEO here, as you mention backing up the git repository is very important. We just announced free hosting for git-annex files and repositories on GitLab.com that might be good for your use case, see https://about.gitlab.com/2015/02/17/gitlab-annex-solves-the-problem-of-versioning-large-binaries-with-git/ https://about.gitlab.com/2015/02/17/gitlab-annex-solves-the-... for an overview.
n3t 12y ago
git-annex website says[1] it is not supposed to be a backup solution: > git-annex is not a backup system. It may be a useful component of an archival system, or a way to deliver files to a backup system With this in mind, can we treat it as a backup tool? Assuming backing up the git repository itself as you mentioned. [1] http://git-annex.branchable.com/not/ http://git-annex.branchable.com/not/

by rlpb 12y ago

icebraining 12y ago

+1 for git annex. For the git repo, if you don't have privacy concerns, you can just push to a private bitbucket repository. But I just had a script with: git clone /my/repo /tmp/backup-git-annex tar cvf /tmp/backup-git-annex.tar /tmp/backup-git-annex That way, you're left with a single tar file to backup.

rcthompson 12y ago

So git-annex now has support for efficiently deduplicating backends? Might be time for me to give it another look.

rlpb 12y ago
For "local" storage as supported by bup and ddar, yes. Generally for external remotes: not yet, AFAIK. http://git-annex.branchable.com/design/assistant/deltas/ http://git-annex.branchable.com/design/assistant/deltas/ has some design thoughts, so I think it is planned.

sytse 12y ago

GitLab CEO here, as you mention backing up the git repository is very important. We just announced free hosting for git-annex files and repositories on GitLab.com that might be good for your use case, see https://about.gitlab.com/2015/02/17/gitlab-annex-solves-the-problem-of-versioning-large-binaries-with-git/ https://about.gitlab.com/2015/02/17/gitlab-annex-solves-the-... for an overview.

n3t 12y ago

git-annex website says[1] it is not supposed to be a backup solution: > git-annex is not a backup system. It may be a useful component of an archival system, or a way to deliver files to a backup system With this in mind, can we treat it as a backup tool? Assuming backing up the git repository itself as you mentioned. [1] http://git-annex.branchable.com/not/ http://git-annex.branchable.com/not/