22 ms·

Sweet. It’s kinda wild to me that this hasn’t been done. Would be so much nicer if gitlab/forgejo/gitea/… supported object storage for everything. Currently you

by mgrandl 13d ago

Sweet. It’s kinda wild to me that this hasn’t been done. Would be so much nicer if gitlab/forgejo/gitea/… supported object storage for everything. Currently you always still need a filesystem for git repos.

throwaway7356 13d ago
It has been done. It is no longer done because it is not efficient. Yes, it was not "object storage", but Git can be served from a dumb http server as static files. Now someone figured out that object storage can also serve static files via http. Wow!
eru 13d ago
On Linux, Fuse is your friend. See eg https://github.com/matthiasgoergens/git-snap-fs https://github.com/matthiasgoergens/git-snap-fs
baalimago 13d ago
> It’s kinda wild to me that this hasn’t been done https://github.com/awslabs/git-remote-s3 https://github.com/awslabs/git-remote-s3 This works seamlessly so you can run `git remote add origin s3://my-git-bucket/my-repo`
everforward 13d ago
Doesn’t Git store data in sparse files? My impression is that git’s on-disk layout is exactly the kind of thing you’d avoid on object storage. Interactions involve a ton of small reads, resulting in poor performance and enormous bills. It only really makes sense to tarball repos into cold storage on S3 or Glacier, but short of GitLab cloud I suspect there aren’t enough repos cold enough to be worth the dev costs. S3 that doesn’t have wild costs for git objects sounds like it’s just NFS
- masklinn 13d ago
  Git does not use sparse files no, however in packs objects are delta-ified: any object can be either literal or a combination of literal segments and range copies from a base object, and that base object can be the same, recursively. So to resolve any object you need to jump around a fair bit in the pack file. The normal GC creates delta chain of length 10, aggressive defaults to 50, the maximum is 4095.
shados 12d ago
There's a bunch. I got pissed during one of the last GitHub outages and had an AI agent one shot a clone with PR, stacks, actions, and full compatibility with the git client running on Cloudflare workers and R2. Only limitations is it was https only because CF workers don't have tcp to implement and without an enterprise account. Works great to this day. And I was far from the first.

by mgrandl 13d ago

throwaway7356 13d ago

It has been done. It is no longer done because it is not efficient. Yes, it was not "object storage", but Git can be served from a dumb http server as static files. Now someone figured out that object storage can also serve static files via http. Wow!

eru 13d ago

On Linux, Fuse is your friend. See eg https://github.com/matthiasgoergens/git-snap-fs https://github.com/matthiasgoergens/git-snap-fs

baalimago 13d ago

> It’s kinda wild to me that this hasn’t been done https://github.com/awslabs/git-remote-s3 https://github.com/awslabs/git-remote-s3 This works seamlessly so you can run `git remote add origin s3://my-git-bucket/my-repo`

everforward 13d ago

Doesn’t Git store data in sparse files? My impression is that git’s on-disk layout is exactly the kind of thing you’d avoid on object storage. Interactions involve a ton of small reads, resulting in poor performance and enormous bills. It only really makes sense to tarball repos into cold storage on S3 or Glacier, but short of GitLab cloud I suspect there aren’t enough repos cold enough to be worth the dev costs. S3 that doesn’t have wild costs for git objects sounds like it’s just NFS

masklinn 13d ago
Git does not use sparse files no, however in packs objects are delta-ified: any object can be either literal or a combination of literal segments and range copies from a base object, and that base object can be the same, recursively. So to resolve any object you need to jump around a fair bit in the pack file. The normal GC creates delta chain of length 10, aggressive defaults to 50, the maximum is 4095.

shados 12d ago

There's a bunch. I got pissed during one of the last GitHub outages and had an AI agent one shot a clone with PR, stacks, actions, and full compatibility with the git client running on Cloudflare workers and R2. Only limitations is it was https only because CF workers don't have tcp to implement and without an enterprise account. Works great to this day. And I was far from the first.