3 ms·
It seems like every large service which started with a vanilla MySQL setup is now utilizing Vitess for scaling/sharding. Does anyone know how a service like Gi
by paxys 5y ago
It seems like every large service which started with a vanilla MySQL setup is now utilizing Vitess for scaling/sharding.
Does anyone know how a service like Github would store & backup the Git repo itself? Not in MySQL, I imagine?
- iudqnolq 5y agoThey use the regular filesystem (via a library version of git: libgit2). It seems crazy that they use the standard off the shelf option but they've contributed a massive amount of work to improving it. https://github.blog/2009-10-20-how-we-made-github-fast/ https://github.blog/2009-10-20-how-we-made-github-fast/
- sofixa 5y ago>It seems crazy that they use the standard off the shelf option but they've contributed a massive amount of work to improving it. Especially considering Gitlab developed Gitaly due to scale and redundancy issues with a regular filesystem.
- iudqnolq 5y agoSkimming search results for that I think GitHub does something similar. It seems Gitaly is an rpc layer between the frontend and git c code running on file servers, no?
- vtbassmatt 5y agoHow we store them: https://github.blog/2016-04-05-introducing-dgit/ https://github.blog/2016-04-05-introducing-dgit/ (now called Spokes) I don't see anywhere that we've talked publicly about how backups work. If there's interest, I can see if someone wants to write a blog post. Disclosure: I'm the GitHub product manager for Git systems including storage and protocols.
- pas 5y ago> If there's interest There is! Storage, backup & restore at that scale are always interesting :)
- vtbassmatt 5y agoSounds good! I'll ask around.
- vtbassmatt 5y agoWhen I was asking around, someone pointed me to a talk given by one of our engineers a few years ago at Git Merge: https://www.youtube.com/watch?v=jcN6UgPN7zA https://www.youtube.com/watch?v=jcN6UgPN7zA
- pas 5y agoThank you!