7 ms·

Sustaining git-annex development

187 points by zrail 13y ago

jlgreco 13y ago
His personal finances are really impressive. Selling his fulltime dedication for 12k/year...
- jsnell 13y ago
  His arrangements are interesting, to say the least, see http://joey.hess.usesthis.com/ http://joey.hess.usesthis.com/ . The whole thing is worth a read, but here's a sample: > So the whole house runs on 12 volt DC power to avoid the overhead of an inverter; my laptop is powered through a succession of cheap vehicle power adapters, and my home server runs on 5 volt power provided by a USB adapter.
  - icebraining 13y ago
    There's also his notes to a caretaker: http://joeyh.name/blog/entry/notes_for_a_caretaker/ http://joeyh.name/blog/entry/notes_for_a_caretaker/
- aspensmonster 13y ago
  I don't recall where I read about it, but my understanding is the developer lives a very frugal lifestyle in an off-grid environment, something akin to homesteading but with an internet connection. His costs are as minimal as he can make them, and in turn he can both spend more time on his work, and be fully commissioned with campaigns like this. I think he was also involved with the Debian GNU/Linux installer.
  - mpyne 13y ago
    > I think he was also involved with the Debian GNU/Linux installer. Yes he was (might still be, too). That where I'd first heard of Joey.
- zoul 13y ago
  Not that it’s his case, but $12k/year is quite a reasonable amount of money in a lot of parts of the world. As an example, here in the Czech Republic you can live a decent life for that amount with all the usual comforts.
zellyn 13y ago
For those of you wondering whether this is worth supporting, the answer is Yes. I work with a lot of smart engineers, and periodically ask them what they do for personal backup of images, video, etc. I don't think I've yet talked to someone who was satisfied with their system. Git annex (with the assistant) brings me closer to my ideal system than I've been before: a drive at home, one in my desk drawer at work for fire-proof backups, easy automatic shuttling of data between them using my phone or USB sticks, and pluggable cloud storage if I decide I want it. Oh, and a friendly, responsive and excellent developer behind it, developing software in a cabin, in the woods. :-)
- SoftwareMaven 13y ago
  Time Machine to a local drive for immediate restores. Backblaze for disaster recovery. Time Machine could easily be replaced by cron and tar or some other local backup option. Other backup providers are easily found. This is not to say anything about git-annex's usefulness or the value of supporting the developer. It just surprises me when engineers are still struggling with backups. I can kind of understand "regular folk", but that's not really the git-annex market, either.
  - asdfs 13y ago
    Can you use Time Machine to shuttle data via a USB stick? How well does it work when you have more than two drives that you want to keep mostly synchronized?
    - semanticist 13y ago
      Time Machine does the one thing it does really well, which is incremental, local (or network-local) backups and easy restores. It's not for snapshotting or mirroring. I used to use CarbonCopyCloner to clone my boot drive to an external USB drive every morning at ~4am, so if the internal drive on my iMac died I had an immediate bootable replacement. (And when the internal drive DID die, I just rebooted the machine and took nearly a year to get around to replacing the internal drive!)
  - 13y ago
zeckalpha 13y ago
How does git-annex compare to bittorrent-sync? (I'm a big fan of Joey's other work, so I bet this is top notch, there just seems to be more momentum behind bittorrent-sync.)
- computer 13y ago
  bittorrent-sync is closed source.
  - zeckalpha 13y ago
    I can't believe I missed that. Thanks!
- muuck 13y ago
  I like git-annex better. Because git-annex doesn't automatic keep copies of the files in all the directories you've shared. When you need a file that's not in your local directory you can just pull it from a different place, git-annex keeps track where the file is stored.
- joeyh 13y ago
  Use cases overlap partially, but are generally quite distinct. I think they could potentially complement one another, especially if bittorrent-sync used an open protocol, so I could adapt one of the haskell bittorrent clients (which seem to make excellent use of haskell's concurrency from what I've read) to use it.
navs 13y ago
I donated and I don't often donate (though I really should). It's not just how awesome the software is that compels me but his dedication to his work and his frugal lifestyle that leaves me wanting to support this guy with all his projects.
malandrew 13y ago
"Git-annex now auto-syncing photos from my android phone to a Tor hidden SSH service I control (via @guardianproject's Orbot) #prismbreak" Interesting. I love to see how that is all set up. I'm wondering if that can be done with a Drobo. If not, what is required to get a mini-server set up that connects to Drobo over iscsi.
rsync 13y ago
We (rsync.net) are supporting git-annex by offering all git-annex users a heavily discounted rsync.net account. This was announced a few days ago, but not on HN in any way: http://rsync.net/products/git-annex-pricing.html http://rsync.net/products/git-annex-pricing.html We've been explicitly supporting git-annex on our platform since our friend Jason Scott first showed it to us[1] and we will be contributing to the new campaign. [1] http://ascii.textfiles.com/archives/3625 http://ascii.textfiles.com/archives/3625
- carterschonwald 13y ago
  awesome!
- rb2k_ 13y ago
  That's nice :) And just so people know without having to check the site: > git-annex users may sign up for our full featured offsite filesystem at a rate of 10 cents per GB, per month. An annual payment is required, and the minimum account size is 50 GB. There are no usage/bandwidth charges, no signup fee, and no contract to sign.
  - mjn 13y ago
    Wow, that looks great. With the discount it matches S3's storage pricing, but without bandwidth charges, with a more convenient interface (an ssh-reachable regular filesystem), and full-service support. Plus an interesting attempt at a "warrant canary": http://www.rsync.net/resources/notices/canary.txt http://www.rsync.net/resources/notices/canary.txt
    - zoul 13y ago
      That canary thing is an interesting trick, previous discussion here: https://news.ycombinator.com/item?id=702247 https://news.ycombinator.com/item?id=702247. (Four years ago!)
      - rsync 13y ago
        We've been operating the warrant canary since early 2006.
M4v3R 13y ago
The best part of git-annex assistant is that the developer managed to deliver, or even over-deliver what he was pitching in Kickstarter campaign. There wasn't a single week in the whole year where he didn't do some real work on the project. You can check his development blog [1] for details. [1] http://git-annex.branchable.com/design/assistant/blog/ http://git-annex.branchable.com/design/assistant/blog/
aw3c2 13y ago
Blimey, https://campaign.joeyh.name/blog/initial_goal_reached_in_seven_hours/ https://campaign.joeyh.name/blog/initial_goal_reached_in_sev... Congratulations!
XorNot 13y ago
I can't remember why I didn't switch to git-annex from Unison, although it did put me onto bup (which still doesn't quite do what I want either). I think my biggest problem was by and large I want a versioning sync tool, and worry a lot less about managing what's on my devices.
- warp 13y ago
  I think you can automate most or all of that with the git annex assistant now (and I think most of its functionality is exposed as commandline commands as well, so you don't have to use the GUI if you don't want to).
mkilling 13y ago
Is there a German cloud storage provider that I can use as a git annex remote?
gnuvince 13y ago
Can anyone give me an example of their usage of git-annex? The project looks interesting, but the most immediate usage I see would be to add big files (e.g. a database dump, PDF documentation, etc.) to a git repository. What else do people use it for?
- warp 13y ago
  I use it to manage my "archive" folder, which contains large binary files which rarely change. This includes music, some tv series, operating system .iso images, backups of retired computers, etc.. The great thing about git annex is that each clone of the repository has the entire tree structure of the repository, but by default has none of the data. So if I'm going on a trip I can just cd into the right folder on my laptop and type "git annex get ." or "git annex get Windows*.iso". Being able to tab-complete all the files in the annex even though you don't have a copy of most of them makes it very convenient. The numcopies constraints also help enforce redundancy on the data. I could use raid to have local redundancy, but that only protects against a harddisk crashing. If I have four repositories in different physical locations and set numcopies to three, then git annex helps make sure there are always enough copies of a file (in three different physical locations), so I won't lose data even if my house burns down.

187 points by zrail 13y ago

jlgreco 13y ago

His personal finances are really impressive. Selling his fulltime dedication for 12k/year...

jsnell 13y ago
His arrangements are interesting, to say the least, see http://joey.hess.usesthis.com/ http://joey.hess.usesthis.com/ . The whole thing is worth a read, but here's a sample: > So the whole house runs on 12 volt DC power to avoid the overhead of an inverter; my laptop is powered through a succession of cheap vehicle power adapters, and my home server runs on 5 volt power provided by a USB adapter.
- icebraining 13y ago
  There's also his notes to a caretaker: http://joeyh.name/blog/entry/notes_for_a_caretaker/ http://joeyh.name/blog/entry/notes_for_a_caretaker/
aspensmonster 13y ago
I don't recall where I read about it, but my understanding is the developer lives a very frugal lifestyle in an off-grid environment, something akin to homesteading but with an internet connection. His costs are as minimal as he can make them, and in turn he can both spend more time on his work, and be fully commissioned with campaigns like this. I think he was also involved with the Debian GNU/Linux installer.
- mpyne 13y ago
  > I think he was also involved with the Debian GNU/Linux installer. Yes he was (might still be, too). That where I'd first heard of Joey.
zoul 13y ago
Not that it’s his case, but $12k/year is quite a reasonable amount of money in a lot of parts of the world. As an example, here in the Czech Republic you can live a decent life for that amount with all the usual comforts.

zellyn 13y ago

For those of you wondering whether this is worth supporting, the answer is Yes. I work with a lot of smart engineers, and periodically ask them what they do for personal backup of images, video, etc. I don't think I've yet talked to someone who was satisfied with their system. Git annex (with the assistant) brings me closer to my ideal system than I've been before: a drive at home, one in my desk drawer at work for fire-proof backups, easy automatic shuttling of data between them using my phone or USB sticks, and pluggable cloud storage if I decide I want it. Oh, and a friendly, responsive and excellent developer behind it, developing software in a cabin, in the woods. :-)

SoftwareMaven 13y ago
Time Machine to a local drive for immediate restores. Backblaze for disaster recovery. Time Machine could easily be replaced by cron and tar or some other local backup option. Other backup providers are easily found. This is not to say anything about git-annex's usefulness or the value of supporting the developer. It just surprises me when engineers are still struggling with backups. I can kind of understand "regular folk", but that's not really the git-annex market, either.
- asdfs 13y ago
  Can you use Time Machine to shuttle data via a USB stick? How well does it work when you have more than two drives that you want to keep mostly synchronized?
  - semanticist 13y ago
    Time Machine does the one thing it does really well, which is incremental, local (or network-local) backups and easy restores. It's not for snapshotting or mirroring. I used to use CarbonCopyCloner to clone my boot drive to an external USB drive every morning at ~4am, so if the internal drive on my iMac died I had an immediate bootable replacement. (And when the internal drive DID die, I just rebooted the machine and took nearly a year to get around to replacing the internal drive!)
- 13y ago

zeckalpha 13y ago

How does git-annex compare to bittorrent-sync? (I'm a big fan of Joey's other work, so I bet this is top notch, there just seems to be more momentum behind bittorrent-sync.)

computer 13y ago
bittorrent-sync is closed source.
- zeckalpha 13y ago
  I can't believe I missed that. Thanks!
muuck 13y ago
I like git-annex better. Because git-annex doesn't automatic keep copies of the files in all the directories you've shared. When you need a file that's not in your local directory you can just pull it from a different place, git-annex keeps track where the file is stored.
joeyh 13y ago
Use cases overlap partially, but are generally quite distinct. I think they could potentially complement one another, especially if bittorrent-sync used an open protocol, so I could adapt one of the haskell bittorrent clients (which seem to make excellent use of haskell's concurrency from what I've read) to use it.

navs 13y ago

I donated and I don't often donate (though I really should). It's not just how awesome the software is that compels me but his dedication to his work and his frugal lifestyle that leaves me wanting to support this guy with all his projects.

malandrew 13y ago

"Git-annex now auto-syncing photos from my android phone to a Tor hidden SSH service I control (via @guardianproject's Orbot) #prismbreak" Interesting. I love to see how that is all set up. I'm wondering if that can be done with a Drobo. If not, what is required to get a mini-server set up that connects to Drobo over iscsi.

rsync 13y ago

We (rsync.net) are supporting git-annex by offering all git-annex users a heavily discounted rsync.net account. This was announced a few days ago, but not on HN in any way: http://rsync.net/products/git-annex-pricing.html http://rsync.net/products/git-annex-pricing.html We've been explicitly supporting git-annex on our platform since our friend Jason Scott first showed it to us[1] and we will be contributing to the new campaign. [1] http://ascii.textfiles.com/archives/3625 http://ascii.textfiles.com/archives/3625

carterschonwald 13y ago
awesome!
rb2k_ 13y ago
That's nice :) And just so people know without having to check the site: > git-annex users may sign up for our full featured offsite filesystem at a rate of 10 cents per GB, per month. An annual payment is required, and the minimum account size is 50 GB. There are no usage/bandwidth charges, no signup fee, and no contract to sign.
- mjn 13y ago
  Wow, that looks great. With the discount it matches S3's storage pricing, but without bandwidth charges, with a more convenient interface (an ssh-reachable regular filesystem), and full-service support. Plus an interesting attempt at a "warrant canary": http://www.rsync.net/resources/notices/canary.txt http://www.rsync.net/resources/notices/canary.txt
  - zoul 13y ago
    That canary thing is an interesting trick, previous discussion here: https://news.ycombinator.com/item?id=702247 https://news.ycombinator.com/item?id=702247. (Four years ago!)
    - rsync 13y ago
      We've been operating the warrant canary since early 2006.

M4v3R 13y ago

The best part of git-annex assistant is that the developer managed to deliver, or even over-deliver what he was pitching in Kickstarter campaign. There wasn't a single week in the whole year where he didn't do some real work on the project. You can check his development blog [1] for details. [1] http://git-annex.branchable.com/design/assistant/blog/ http://git-annex.branchable.com/design/assistant/blog/

aw3c2 13y ago

Blimey, https://campaign.joeyh.name/blog/initial_goal_reached_in_seven_hours/ https://campaign.joeyh.name/blog/initial_goal_reached_in_sev... Congratulations!

XorNot 13y ago

I can't remember why I didn't switch to git-annex from Unison, although it did put me onto bup (which still doesn't quite do what I want either). I think my biggest problem was by and large I want a versioning sync tool, and worry a lot less about managing what's on my devices.

warp 13y ago
I think you can automate most or all of that with the git annex assistant now (and I think most of its functionality is exposed as commandline commands as well, so you don't have to use the GUI if you don't want to).

mkilling 13y ago

Is there a German cloud storage provider that I can use as a git annex remote?

gnuvince 13y ago

Can anyone give me an example of their usage of git-annex? The project looks interesting, but the most immediate usage I see would be to add big files (e.g. a database dump, PDF documentation, etc.) to a git repository. What else do people use it for?

warp 13y ago
I use it to manage my "archive" folder, which contains large binary files which rarely change. This includes music, some tv series, operating system .iso images, backups of retired computers, etc.. The great thing about git annex is that each clone of the repository has the entire tree structure of the repository, but by default has none of the data. So if I'm going on a trip I can just cd into the right folder on my laptop and type "git annex get ." or "git annex get Windows*.iso". Being able to tab-complete all the files in the annex even though you don't have a copy of most of them makes it very convenient. The numcopies constraints also help enforce redundancy on the data. I could use raid to have local redundancy, but that only protects against a harddisk crashing. If I have four repositories in different physical locations and set numcopies to three, then git annex helps make sure there are always enough copies of a file (in three different physical locations), so I won't lose data even if my house burns down.