5 ms·

The other day I was trying to work with git LFS. I was very surprised to find out git-lfs, as in the binary, CLI application is the only (open) implementation i

The other day I was trying to work with git LFS. I was very surprised to find out git-lfs, as in the binary, CLI application is the only (open) implementation in existence. There is nothing else. And even it itself does not offer itself up as a library; so even native Go code (the implementation language) has to fall back to shelling out to the CLI git extension! Not even bindings are possible. Such a painful loss of interoperability: IPC via return codes and parsing stdout/stderr. It seems a similar story with the rest of git. I have hopes for gitoxide aka gix, and think the approach of library-first is correct going into the future. A CLI is then simply a thin wrapper around it, mapping argv to library operations basically.

38 3y ago
> it itself does not offer itself up as a library yeah, it does: https://godocs.io/github.com/git-lfs/git-lfs/v3 https://godocs.io/github.com/git-lfs/git-lfs/v3
- diarrhea 3y ago
  I don’t know what that is, but their docs very prominently and strongly say this: > However, we do not maintain a stable Go language API or ABI, as Git LFS is intended to be used solely as a compiled binary utility. Please do not import the git-lfs module into other Go code and do not rely on it as a source code dependency. https://github.com/git-lfs/git-lfs#limitations https://github.com/git-lfs/git-lfs#limitations
  - 38 3y ago
    [flagged]
    - yjftsjthsd-h 3y ago
      That you technically have the ability to call into an internal module does not in any way constitute it "offering itself up as a library", and doesn't make it effectively useful in that way.
    - IggleSniggle 3y ago
      It is a module. It is available. It does not offer itself up as a library.
      - 38 3y ago
        > It is a module. > It does not offer itself up as a library. in Go code, those are the same thing.
        IggleSniggle 3y ago
        A library and a module are the same. Having an open and available module does not make it "offered up as a library," was the point I was trying (and failing, evidently), to make.
        Brian_K_White 3y ago
        You made the point perfectly. So did the docs in the first place. It's not on you that someone decides not to agree the sky is blue because they use their own definition of blue.
    - dang 3y ago
      Please don't cross into personal attack. It destroys what this site is for, and you can easily make your substantive points without it. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
  - dmoy 3y ago
    Anything's a library if you're sufficiently motivated - Hyrum Wright, probably
    - jrockway 3y ago
      I made Kubernetes into a library once. Like the apiserver and default controllers, running in-process. Copying and pasting were involved. I was not expecting support. If code is out there under a compatible license, you can do whatever you want with it. If it breaks, you get to keep both pieces.
TkTech 3y ago
> It seems a similar story with the rest of git. Dulwich[1] is a pure-python Git implementation that's been around for many years, meant to be used as a library. I used it a long time ago to make a git-backed wiki. There's also libgit2 which is exactly what it sounds like and it has mature Go bindings[2]. I'm sure there are more implementations. [1]: https://github.com/jelmer/dulwich https://github.com/jelmer/dulwich [2]: https://github.com/libgit2/git2go https://github.com/libgit2/git2go
coryrc 3y ago
> IPC via return codes and parsing stdout/stderr That's wildly different from Go's method of != nil and error strings.
- benhoyt 3y ago
  I'm guessing you're being sarcastic. If so, it's not really a fair criticism. Go has properly-typed return values (not just parsing text from stdout) and any type can implement the error interface, allowing for error types with any custom, properly-typed attributes. You use Go's type assertion or errors.Is / errors.As to handle the latter.
  - coryrc 3y ago
    Maybe it's my job, but I don't see a lot of Go code treating errors as anything but strings. Errors I get from Go programs are similarly vague and redundant and lacking context. ("error from RPC: failure making from call: couldn't complete operation: 7 is not a valid flag" ah thanks, so much better than stack trace) Also it's the weekend and I'm just bitching.
    - benhoyt 3y ago
      I think that's a fair critique. You have to really put thought into good error messages and error handling. When you do, they're nicer and more informative than an ugly stack trace. But a poor error message chain like the one you've shown is significantly less helpful than a stack trace.
    - stouset 3y ago
      Yes, but don’t you value the fact that this error message was painstakingly crafted by hand under the care of skilled golang artisans instead of being mass-produced by an automated exception handler?
    - ttymck 3y ago
      Yeah because that's the error interface? It's literally a method that returns string. Unfortunately, I guess after the fact they decided pattern matching was useful, so they did it via reflection (errors.As). But they don't warn you that errors.Is might return false for a true errors.As, it is certainly a mess, but the pedagogy could be improved.
CuriousCosmic 3y ago
> It seems a similar story with the rest of git. I have hopes for gitoxide aka gix, and think the approach of library-first is correct going into the future. A CLI is then simply a thin wrapper around it, mapping argv to library operations basically. It's worth noting that there is currently a push to "lib-ify` git internals and it's a gradual process. I'm not actually sure how much of this work has actually made it into the tree yet but I've been seeing patchsets towards that goal on the mailing list since at least January.
strogonoff 3y ago
Isomorphic Git is a Git implementation purely in JS (no WASM). I wrote a minimal library to handle LFS with it, it’s not that hard, the spec is pretty small.
adrianmsmith 3y ago
I always respected the fact that the authors of Subversion, right from the start, structured their software as a library, with the CLI being a user of that library. The way IDEs and GUIs interacted with CVS was to shell out to the CLI, which inevitably had problems with filenames with spaces, parsing of error messages, etc. Subversion understood in 2000 that the things were changing, and that the CLI was only one way you'd use a VCS. People were more and more interacting with the VCS via IDEs, or via right-click menus in Windows Explorer, etc. I felt happy knowing I'd never again have to deal VCSs via tools just shelling out to their CLI ever again. How wrong I was...
- diamondo25 3y ago
  JetBrains forgot this memo, as it requires you to configure svn.exe in order for it to work. And usually TortoiseSVN is the way to go for subversion...
alexhornby 3y ago
> git-lfs, as in the binary, CLI application is the only (open) implementation in existence. There is nothing else. There’s at lease one in sapling and Mononoke. https://github.com/facebook/sapling/tree/main/eden/mononoke/lfs_protocol/src https://github.com/facebook/sapling/tree/main/eden/mononoke/...

38 3y ago

> it itself does not offer itself up as a library yeah, it does: https://godocs.io/github.com/git-lfs/git-lfs/v3 https://godocs.io/github.com/git-lfs/git-lfs/v3

diarrhea 3y ago
I don’t know what that is, but their docs very prominently and strongly say this: > However, we do not maintain a stable Go language API or ABI, as Git LFS is intended to be used solely as a compiled binary utility. Please do not import the git-lfs module into other Go code and do not rely on it as a source code dependency. https://github.com/git-lfs/git-lfs#limitations https://github.com/git-lfs/git-lfs#limitations
- 38 3y ago
  [flagged]
  - yjftsjthsd-h 3y ago
    That you technically have the ability to call into an internal module does not in any way constitute it "offering itself up as a library", and doesn't make it effectively useful in that way.
  - IggleSniggle 3y ago
    It is a module. It is available. It does not offer itself up as a library.
    - 38 3y ago
      > It is a module. > It does not offer itself up as a library. in Go code, those are the same thing.
      - IggleSniggle 3y ago
        A library and a module are the same. Having an open and available module does not make it "offered up as a library," was the point I was trying (and failing, evidently), to make.
        Brian_K_White 3y ago
        You made the point perfectly. So did the docs in the first place. It's not on you that someone decides not to agree the sky is blue because they use their own definition of blue.
  - dang 3y ago
    Please don't cross into personal attack. It destroys what this site is for, and you can easily make your substantive points without it. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
- dmoy 3y ago
  Anything's a library if you're sufficiently motivated - Hyrum Wright, probably
  - jrockway 3y ago
    I made Kubernetes into a library once. Like the apiserver and default controllers, running in-process. Copying and pasting were involved. I was not expecting support. If code is out there under a compatible license, you can do whatever you want with it. If it breaks, you get to keep both pieces.

TkTech 3y ago

> It seems a similar story with the rest of git. Dulwich[1] is a pure-python Git implementation that's been around for many years, meant to be used as a library. I used it a long time ago to make a git-backed wiki. There's also libgit2 which is exactly what it sounds like and it has mature Go bindings[2]. I'm sure there are more implementations. [1]: https://github.com/jelmer/dulwich https://github.com/jelmer/dulwich [2]: https://github.com/libgit2/git2go https://github.com/libgit2/git2go

coryrc 3y ago

> IPC via return codes and parsing stdout/stderr That's wildly different from Go's method of != nil and error strings.

benhoyt 3y ago
I'm guessing you're being sarcastic. If so, it's not really a fair criticism. Go has properly-typed return values (not just parsing text from stdout) and any type can implement the error interface, allowing for error types with any custom, properly-typed attributes. You use Go's type assertion or errors.Is / errors.As to handle the latter.
- coryrc 3y ago
  Maybe it's my job, but I don't see a lot of Go code treating errors as anything but strings. Errors I get from Go programs are similarly vague and redundant and lacking context. ("error from RPC: failure making from call: couldn't complete operation: 7 is not a valid flag" ah thanks, so much better than stack trace) Also it's the weekend and I'm just bitching.
  - benhoyt 3y ago
    I think that's a fair critique. You have to really put thought into good error messages and error handling. When you do, they're nicer and more informative than an ugly stack trace. But a poor error message chain like the one you've shown is significantly less helpful than a stack trace.
  - stouset 3y ago
    Yes, but don’t you value the fact that this error message was painstakingly crafted by hand under the care of skilled golang artisans instead of being mass-produced by an automated exception handler?
  - ttymck 3y ago
    Yeah because that's the error interface? It's literally a method that returns string. Unfortunately, I guess after the fact they decided pattern matching was useful, so they did it via reflection (errors.As). But they don't warn you that errors.Is might return false for a true errors.As, it is certainly a mess, but the pedagogy could be improved.

CuriousCosmic 3y ago

> It seems a similar story with the rest of git. I have hopes for gitoxide aka gix, and think the approach of library-first is correct going into the future. A CLI is then simply a thin wrapper around it, mapping argv to library operations basically. It's worth noting that there is currently a push to "lib-ify` git internals and it's a gradual process. I'm not actually sure how much of this work has actually made it into the tree yet but I've been seeing patchsets towards that goal on the mailing list since at least January.

strogonoff 3y ago

Isomorphic Git is a Git implementation purely in JS (no WASM). I wrote a minimal library to handle LFS with it, it’s not that hard, the spec is pretty small.

adrianmsmith 3y ago

I always respected the fact that the authors of Subversion, right from the start, structured their software as a library, with the CLI being a user of that library. The way IDEs and GUIs interacted with CVS was to shell out to the CLI, which inevitably had problems with filenames with spaces, parsing of error messages, etc. Subversion understood in 2000 that the things were changing, and that the CLI was only one way you'd use a VCS. People were more and more interacting with the VCS via IDEs, or via right-click menus in Windows Explorer, etc. I felt happy knowing I'd never again have to deal VCSs via tools just shelling out to their CLI ever again. How wrong I was...

diamondo25 3y ago
JetBrains forgot this memo, as it requires you to configure svn.exe in order for it to work. And usually TortoiseSVN is the way to go for subversion...

alexhornby 3y ago

> git-lfs, as in the binary, CLI application is the only (open) implementation in existence. There is nothing else. There’s at lease one in sapling and Mononoke. https://github.com/facebook/sapling/tree/main/eden/mononoke/lfs_protocol/src https://github.com/facebook/sapling/tree/main/eden/mononoke/...