4 ms·

Probably based on JGit/Gerrit. It is much easier to build "cloud" git (i.e. reliable, redundant git) that way than it is to make your disk reliable. Github we

Probably based on JGit/Gerrit. It is much easier to build "cloud" git (i.e. reliable, redundant git) that way than it is to make your disk reliable. Github went the reliable-disk route with DRBD, AFAIK, which is why Github is always going down ;-) I wrote a blog post about this: http://blog.justinsb.com/blog/2013/12/14/cloudata-day-8/ http://blog.justinsb.com/blog/2013/12/14/cloudata-day-8/

btown 12y ago
Intriguing post! For those who hadn't heard about DRBD, it's http://en.wikipedia.org/wiki/Distributed_Replicated_Block_Device http://en.wikipedia.org/wiki/Distributed_Replicated_Block_De... - it makes failover incredibly simple to integrate by pushing a lot of distributed-systems responsibility down to the filesystem level, but in doing so it forces you to have a single-master setup, and you can't take advantage of parts of your domain model that are well-suited to eventual consistency. Thinking about it, now that Riak supports strong (quorum-based) consistency on a per-bucket basis, Riak-backed scalable Git hosting would probably be relatively easy to implement. Looks like you were building your own system - did you ever release the code?
- justinsb 12y ago
  Ooops... looks like I didn't link to the code in that post. The whole series was open source; here's the git code itself: https://github.com/justinsb/cloudata/tree/master/cloudata-git https://github.com/justinsb/cloudata/tree/master/cloudata-gi... The git object data itself goes into a blob-store (like S3); it can be stored without strong consistency. It turns out you only need to keep track of a very small amount of metadata consistently (the refs). Riak, etcd, DynamoDB or the Google Cloud DataStore would all be good choices, I think. I was working on an open-source implementation of Raft as part of this (called barge), but it isn't as reliable as the alternatives above - yet!
wmf 12y ago
As a counterpoint, here's the rationale for why GitHub uses regular git repo storage instead of webscale object storage: http://devslovebacon.com/conferences/bacon-2013/talks/my-mom-told-me-that-git-doesnt-scale http://devslovebacon.com/conferences/bacon-2013/talks/my-mom... (maybe skip to 16m) Given Amazon's completely different philosophy, it's natural for them to implement git on S3.
- justinsb 12y ago
  Very interesting. I hope that's not the real rationale inside Github, as it totally ignores the fact that the blobs are immutable and thus trivially cacheable.

btown 12y ago

Intriguing post! For those who hadn't heard about DRBD, it's http://en.wikipedia.org/wiki/Distributed_Replicated_Block_Device http://en.wikipedia.org/wiki/Distributed_Replicated_Block_De... - it makes failover incredibly simple to integrate by pushing a lot of distributed-systems responsibility down to the filesystem level, but in doing so it forces you to have a single-master setup, and you can't take advantage of parts of your domain model that are well-suited to eventual consistency. Thinking about it, now that Riak supports strong (quorum-based) consistency on a per-bucket basis, Riak-backed scalable Git hosting would probably be relatively easy to implement. Looks like you were building your own system - did you ever release the code?

justinsb 12y ago
Ooops... looks like I didn't link to the code in that post. The whole series was open source; here's the git code itself: https://github.com/justinsb/cloudata/tree/master/cloudata-git https://github.com/justinsb/cloudata/tree/master/cloudata-gi... The git object data itself goes into a blob-store (like S3); it can be stored without strong consistency. It turns out you only need to keep track of a very small amount of metadata consistently (the refs). Riak, etcd, DynamoDB or the Google Cloud DataStore would all be good choices, I think. I was working on an open-source implementation of Raft as part of this (called barge), but it isn't as reliable as the alternatives above - yet!

wmf 12y ago

As a counterpoint, here's the rationale for why GitHub uses regular git repo storage instead of webscale object storage: http://devslovebacon.com/conferences/bacon-2013/talks/my-mom-told-me-that-git-doesnt-scale http://devslovebacon.com/conferences/bacon-2013/talks/my-mom... (maybe skip to 16m) Given Amazon's completely different philosophy, it's natural for them to implement git on S3.

justinsb 12y ago
Very interesting. I hope that's not the real rationale inside Github, as it totally ignores the fact that the blobs are immutable and thus trivially cacheable.