4 ms·

underlying idea of storing git repositories in object stores is identical to https://github.com/tigrisdata/objgit https://github.com/tigrisdata/objgit see: htt

underlying idea of storing git repositories in object stores is identical to https://github.com/tigrisdata/objgit https://github.com/tigrisdata/objgit see: https://news.ycombinator.com/item?id=48661938 https://news.ycombinator.com/item?id=48661938

inigyou 1mo ago
That one is upfront that it's vibecoded as an experiment. I can respect that.
- xena 1mo ago
  I've been thinking about circling back to improve it. Right now one of the major limitations is that it uses a filesystem abstraction as the main bridge from git to object storage. I really wanted to see how far I could go by kitbashing parts together from software I'd already made. Turns out you can go pretty damn far! Turns out this is kind of a horrible idea in practice in ways that only really turn up when you try to square peg -> round hole as hard as I did (pushing gcc.git took multiple hours due to filesystem iops latency, turns out a thing designed for nanoseconds of latency copes poorly with a setup that has milliseconds of latency). I'm probably going to end up using the filesystem as a staging area and then have the git server asynchronously unpack git objects into individual S3 objects. I'm still not sure how to best implement merging or other git operations, but this is just kind of a hard problem in practice.
  - inigyou 1mo ago
    Could you modify Git itself to access several objects at the same time? Possibly not in C, but how about the Go reimplementation since concurrency is quite easy in Go?
    - xena 1mo ago
      I don't know enough about how the relevant libraries and primitives work to give your question the meaningful response it deserves. A lot of the problem here is that most of this implementation requires me to really deeply understand the plumbing of Git in ways I currently don't. This is part of why my prototype was done with heavy assistance from agentic coding tools. I was hoping that those hard parts could be abstracted away for later with load-bearing hacks. It seems that the chickens came home to roost much faster than I expected.
      - inigyou 1mo ago
        And you didn't vibe ask AI either. Interesting.
        xena 1mo ago
        It's rude to be a meat proxy. I practice what I preach.

inigyou 1mo ago

That one is upfront that it's vibecoded as an experiment. I can respect that.

xena 1mo ago
I've been thinking about circling back to improve it. Right now one of the major limitations is that it uses a filesystem abstraction as the main bridge from git to object storage. I really wanted to see how far I could go by kitbashing parts together from software I'd already made. Turns out you can go pretty damn far! Turns out this is kind of a horrible idea in practice in ways that only really turn up when you try to square peg -> round hole as hard as I did (pushing gcc.git took multiple hours due to filesystem iops latency, turns out a thing designed for nanoseconds of latency copes poorly with a setup that has milliseconds of latency). I'm probably going to end up using the filesystem as a staging area and then have the git server asynchronously unpack git objects into individual S3 objects. I'm still not sure how to best implement merging or other git operations, but this is just kind of a hard problem in practice.
- inigyou 1mo ago
  Could you modify Git itself to access several objects at the same time? Possibly not in C, but how about the Go reimplementation since concurrency is quite easy in Go?
  - xena 1mo ago
    I don't know enough about how the relevant libraries and primitives work to give your question the meaningful response it deserves. A lot of the problem here is that most of this implementation requires me to really deeply understand the plumbing of Git in ways I currently don't. This is part of why my prototype was done with heavy assistance from agentic coding tools. I was hoping that those hard parts could be abstracted away for later with load-bearing hacks. It seems that the chickens came home to roost much faster than I expected.
    - inigyou 1mo ago
      And you didn't vibe ask AI either. Interesting.
      - xena 1mo ago
        It's rude to be a meat proxy. I practice what I preach.