5 ms·

Something that I really like in Git is how its data structures are easy to understand and how transparent it is. It's possible to write your own "Git" compatibl

Something that I really like in Git is how its data structures are easy to understand and how transparent it is. It's possible to write your own "Git" compatible with existing Git directories only by reading how it works under the hood

shivasaxena 1y ago
I agree, but only in theory. Projects like gitoxide have been in development for years now.
- fiddlerwoaroof 1y ago
  I wrote a nearly complete implementation of git file format parsers in Common Lisp over like a month of evenings and weekends. I’m sure there are hard parts between where I am and a full git implementation but you can get quite a bit of utility out of a relatively small amount of effort.
  - lucasoshiro 1y ago
    Yeah, I wrote mine in Haskell. It's a good exercise for understanding how Git works
  - MrJohz 1y ago
    It's a case of Pareto. Parsing the git file format is relatively simple, but handling all the weird states a Git repo can be in and doing the correct things to those files in each state is a lot harder. And then adding the network protocol on top of that makes directly reproducing Git quite difficult. I know JJ used to use Git2 for a lot of network operations like pushing and pulling, but ran into too many issues with SSH handling that they've since switched to directly invoking the Git binary for those operations.
    - fiddlerwoaroof 1y ago
      There aren’t that many weird states a git repository can be in: the on-disk format of the repository is too simple for that. The hard part has to do with the various protocols for transferring objects around.
      - deathanatos 1y ago
        I think there's more corners out there than most people would give credit to? Just off the top of my head: files in the index (but maybe this isn't "weird enough"), rebasing but paused, rebasing with conflicts, merge with conflicts, cherry-picking but conflicts, middle of a bisect with all the state that implies, alternate objects dirs, alternate working dirs, submodules and all of their weirdness, and a "bare" repo. Heck, had my PS1 return an error this week after I created a separate working dir for a repo and cd'd into it. Did you know .git can be a normal file? I didn't when I wrote my PS1.
        fiddlerwoaroof 1y ago
        I knew .git can be a normal file because of worktrees. But most of the weird states have to do with the working tree not the repository. Even rebasing isn’t weird as far as the file formats go: it just is replaying commits on top of a new base commit. Since my goal was basically to implement enough of git to serve files from a git repository as a website, the actual task was fairly small.
- chubot 1y ago
  Not sure what gitoxide is, but libgit already exists, and it seems to be an independent implementation - https://github.com/libgit2/libgit2 https://github.com/libgit2/libgit2 I think Github and most big Git hosts use it
  - steveklabnik 1y ago
    libgit2 has a ton of compatibility issues, especially around authentication, that make it only useful in some circumstances. (gitoxide is a similar project but in Rust, it's not ready for the big time either, though it keeps on getting better!)
    - chubot 1y ago
      If it's about auth, then I'd say that's more about the "edges" / porcelain than the core. The claim about git is that you can re-implement the core data structures pretty easily, and they are stable While new functionality can be implemented on top of this stable base. It doesn't surprise me that it's hard to re-implement all of git, including the network protocols, but that's a different thing than the core This thread has some good info about the core, including the tree encoding issue brought up by bradfitz, but it still seems to be approximately true ... e.g. perhaps compared to hg I also remember that JGit is a very complete re-implementation of git, done by a git core dev, and used in Android infrastructure, etc. This was well over 10 years ago at this point, maybe 15 https://github.com/eclipse-jgit/jgit https://github.com/eclipse-jgit/jgit
      - steveklabnik 1y ago
        Yes, I think that the core is much better, all I mean to say is that 100% compat is difficult.
  - 3eb7988a1663 1y ago
    Jujitsu threw in the towel and is shelling out to the git CLI because of minor variations in libgit vs the binary. Failing to find a write-up, but there was this lobster thread[0] where someone from GitLab reported they had to do the same owing to some discrepancies vs the binary -where all of the real development happens. [0] https://lobste.rs/s/vmdggh/jujutsu_v0_30_0_released https://lobste.rs/s/vmdggh/jujutsu_v0_30_0_released
- ratmice 1y ago
  If I'm not mistaken, gitoxide is attempting to not only be feature complete, but also the fastest. Both of which reduce development velocity

shivasaxena 1y ago

I agree, but only in theory. Projects like gitoxide have been in development for years now.

fiddlerwoaroof 1y ago
I wrote a nearly complete implementation of git file format parsers in Common Lisp over like a month of evenings and weekends. I’m sure there are hard parts between where I am and a full git implementation but you can get quite a bit of utility out of a relatively small amount of effort.
- lucasoshiro 1y ago
  Yeah, I wrote mine in Haskell. It's a good exercise for understanding how Git works
- MrJohz 1y ago
  It's a case of Pareto. Parsing the git file format is relatively simple, but handling all the weird states a Git repo can be in and doing the correct things to those files in each state is a lot harder. And then adding the network protocol on top of that makes directly reproducing Git quite difficult. I know JJ used to use Git2 for a lot of network operations like pushing and pulling, but ran into too many issues with SSH handling that they've since switched to directly invoking the Git binary for those operations.
  - fiddlerwoaroof 1y ago
    There aren’t that many weird states a git repository can be in: the on-disk format of the repository is too simple for that. The hard part has to do with the various protocols for transferring objects around.
    - deathanatos 1y ago
      I think there's more corners out there than most people would give credit to? Just off the top of my head: files in the index (but maybe this isn't "weird enough"), rebasing but paused, rebasing with conflicts, merge with conflicts, cherry-picking but conflicts, middle of a bisect with all the state that implies, alternate objects dirs, alternate working dirs, submodules and all of their weirdness, and a "bare" repo. Heck, had my PS1 return an error this week after I created a separate working dir for a repo and cd'd into it. Did you know .git can be a normal file? I didn't when I wrote my PS1.
      - fiddlerwoaroof 1y ago
        I knew .git can be a normal file because of worktrees. But most of the weird states have to do with the working tree not the repository. Even rebasing isn’t weird as far as the file formats go: it just is replaying commits on top of a new base commit. Since my goal was basically to implement enough of git to serve files from a git repository as a website, the actual task was fairly small.
chubot 1y ago
Not sure what gitoxide is, but libgit already exists, and it seems to be an independent implementation - https://github.com/libgit2/libgit2 https://github.com/libgit2/libgit2 I think Github and most big Git hosts use it
- steveklabnik 1y ago
  libgit2 has a ton of compatibility issues, especially around authentication, that make it only useful in some circumstances. (gitoxide is a similar project but in Rust, it's not ready for the big time either, though it keeps on getting better!)
  - chubot 1y ago
    If it's about auth, then I'd say that's more about the "edges" / porcelain than the core. The claim about git is that you can re-implement the core data structures pretty easily, and they are stable While new functionality can be implemented on top of this stable base. It doesn't surprise me that it's hard to re-implement all of git, including the network protocols, but that's a different thing than the core This thread has some good info about the core, including the tree encoding issue brought up by bradfitz, but it still seems to be approximately true ... e.g. perhaps compared to hg I also remember that JGit is a very complete re-implementation of git, done by a git core dev, and used in Android infrastructure, etc. This was well over 10 years ago at this point, maybe 15 https://github.com/eclipse-jgit/jgit https://github.com/eclipse-jgit/jgit
    - steveklabnik 1y ago
      Yes, I think that the core is much better, all I mean to say is that 100% compat is difficult.
- 3eb7988a1663 1y ago
  Jujitsu threw in the towel and is shelling out to the git CLI because of minor variations in libgit vs the binary. Failing to find a write-up, but there was this lobster thread[0] where someone from GitLab reported they had to do the same owing to some discrepancies vs the binary -where all of the real development happens. [0] https://lobste.rs/s/vmdggh/jujutsu_v0_30_0_released https://lobste.rs/s/vmdggh/jujutsu_v0_30_0_released
ratmice 1y ago
If I'm not mistaken, gitoxide is attempting to not only be feature complete, but also the fastest. Both of which reduce development velocity