3 ms·
Like what? I'm going to tentatively suggest this is one of those 'hard' problems that throwing buzz words like 'cloud technologies' at doesn't solve. What rep
by shadowmint 11y ago
Like what?
I'm going to tentatively suggest this is one of those 'hard' problems that throwing buzz words like 'cloud technologies' at doesn't solve.
What replication tech would you imagine solves this issue of distributing hundreds of thousands of constantly updated repositories?
- justinsb 11y agoIt's actually a relatively easy problem (compared to say a full POSIX filesystem) - I mentioned elsewhere in these comments a blog post where I implemented a fairly good solution. The objects are essentially immutable and content-addressable so you can get away with very relaxed semantics here. You need a reliable mapping of names to SHAs, but this is also comparatively easy (a key-value store). For example, you can easily satisfy this with S3 and DynamoDB - I think the latest version of the cloudata project I was blogging about actually does that now.
- summner 11y agoThey've addressed that in the post. To make git work comfortably especially on bigger repositories you need to have fast local access to all blobs. IIRC with something like libgit2 its relatively easy to implement what you describe but to make that perform while doing git log or diffs is completely different story.
- justinsb 11y agoIt isn't clear that they wouldn't get that same performance by caching the blobs locally. In my experience, you would, with much higher reliability and scalability.