5 ms·

What’s the challenge with LFS?

Aeolun 4y ago
That it’s not baked right into git? Half of my challenges revolve around getting either everything out of LFS, or putting everything in LFS.
penjelly 4y ago
non-devs (designers) groan when they need to worry about LFS in my exp. Though i never understood the pain either, just sharing what I've observed
- raincole 4y ago
  Non-devs complain about any version control tho. Perforce? Plastic SCM? Does anyone really like them?
- Cthulhu_ 4y ago
  What version control systems are there for non-code? I never looked into it, I can imagine there's a whole world out there for things like written articles, books, but also video game assets like artwork, sprites, 3D models, textures, animations, etc. edit: Looks like e.g. the Unreal engine has support for Perforce and SVN (https://docs.unrealengine.com/5.0/en-US/collaboration-and-version-control-in-unreal-engine/ https://docs.unrealengine.com/5.0/en-US/collaboration-and-ve...), where they make use of locking a file when editing it to avoid merge issues with binary files. I can imagine that frequently goes wrong.
  - grrdotcloud 4y ago
    I am of the opinion the mind of the designer or creative does not think in terms of "versions". There are two states, in woodworking for example, the raw material and the final product. The mental model of "cutting longer" or rolling back does not exist. The idea of a filesystem level understanding of "undo' just has not been communicated or explored. While not Oxen related - TimeMachine seems to be the closest conceptual understanding. Snapshotting the drive, deltas, etc are generally not considered in my experience. I would be curious how larger or mature groups manage versioning.
remram 4y ago
LFS is slow (copies files during checkout, downloads/uploads files independently), inefficient (no de-duplication or delta encoding, contrary to base Git), error-prone (needs manual setup, and rewriting history if anyone gets it wrong), and otherwise not well integrated (it relies on smudge filter and other hooks, so some Git commands are aware of it and others not. You will often see the `version ... oid ...` content of the actual blob rather than the large file).
- unshavedyak 4y ago
  > and otherwise not well integrated (it relies on smudge filter and other hooks, so some Git commands are aware of it and others not. You will often see the `version ... oid ...` content of the actual blob rather than the large file). Curious, do you know the solutions here? I've looked into integrating Git LFS-like things into Git and it felt like Git doesn't give you many tools. Smudging, as hacky as it is, feels one of a very small pool of options you have. I want to say Git Annex does something slightly different. Regardless Smudging felt fine due to how difficult Git made it. Are there better ways in your mind?
  - remram 4y ago
    I don't think you can do better if you don't want to change Git, no.
    - unshavedyak 4y ago
      Seems like an odd critique of LFS then, no? Your other points are super valid, i've wanted to replace LFS with my own impl for the same reasons _(also it was originally designed for HTTP servers, ugh)_. However the integration is roughly as good as it can be, no? Or rather, Oxen would suffer the same limitations no?
      - remram 4y ago
        I'm not trying to target this critique at LFS specifically and absolve Git. It might be that LFS is a bad product because of Git's limited extension points, it remains a bad product.
olvy0 4y ago
In addition to what was said above: In later versions, LFS doesn't play nice with misconfigured internal networks based on Microsoft's Azure / Active Directory. The LFS project dropped support for NTLM (for good reasons) but it's difficult to convince IT to switch to Kerberos. Our team have been trying to do that for several weeks now. So the result is that if we want to use LFS we are stuck with old versions of LFS which still supported NTLM, which necessitate using old versions of GIT which supported this old version of LFS, else we run into all sorts of weird error messages. Also the same combination of old GIT + old LFS works perfectly on some machines, and fails with some weird error messages on others. Possibly due to another IT misconfiguration or something. I've been wanting for a while now to check if I can fork LFS and fix this, unfortunately my knowledge of golang is very sketchy. Hopefully we'll manage to convince IT to drop NTLM support from the on-premise Azure.
- e12e 4y ago
  > Hopefully we'll manage to convince IT to drop NTLM support from the on-premise Azure. What year is this? ;) On a more serious note - ntlm, really? And with azure ad, not a regular (old) on-premise domain? I know it can be hard to drag windows setups into the future (hello WINS!) - But why keep ntlm auth in a post windows 7 world? Legacy systems without ldap/radius support? Genuinely curious (and a little horrified).
  - olvy0 4y ago
    Tell me about it ;) Apparently, it's related to the few win 7 machines that are still present somewhere in the organization for various reasons. All of them are airgapped (I hope) so there's no real reason to keep NTLM, but nobody has yet made the decision. Working on getting to the guy in charge to authorize it. Sigh.
sam_goody 4y ago
I seem to always be running into git-lfs related issues. The latest include.. If one file doesn't resolve, for any reason, git-lfs will block the entire pull. No way to say "get whatever you could, and the fact that aws is not serving that file is not a blocker" Git clone does not pull the entire lfs history of an image. Lacking the entire history, lfs will fail consistently on future actions.

That it’s not baked right into git? Half of my challenges revolve around getting either everything out of LFS, or putting everything in LFS.

penjelly 4y ago

non-devs (designers) groan when they need to worry about LFS in my exp. Though i never understood the pain either, just sharing what I've observed

raincole 4y ago
Non-devs complain about any version control tho. Perforce? Plastic SCM? Does anyone really like them?
Cthulhu_ 4y ago
What version control systems are there for non-code? I never looked into it, I can imagine there's a whole world out there for things like written articles, books, but also video game assets like artwork, sprites, 3D models, textures, animations, etc. edit: Looks like e.g. the Unreal engine has support for Perforce and SVN (https://docs.unrealengine.com/5.0/en-US/collaboration-and-version-control-in-unreal-engine/ https://docs.unrealengine.com/5.0/en-US/collaboration-and-ve...), where they make use of locking a file when editing it to avoid merge issues with binary files. I can imagine that frequently goes wrong.
- grrdotcloud 4y ago
  I am of the opinion the mind of the designer or creative does not think in terms of "versions". There are two states, in woodworking for example, the raw material and the final product. The mental model of "cutting longer" or rolling back does not exist. The idea of a filesystem level understanding of "undo' just has not been communicated or explored. While not Oxen related - TimeMachine seems to be the closest conceptual understanding. Snapshotting the drive, deltas, etc are generally not considered in my experience. I would be curious how larger or mature groups manage versioning.

remram 4y ago

LFS is slow (copies files during checkout, downloads/uploads files independently), inefficient (no de-duplication or delta encoding, contrary to base Git), error-prone (needs manual setup, and rewriting history if anyone gets it wrong), and otherwise not well integrated (it relies on smudge filter and other hooks, so some Git commands are aware of it and others not. You will often see the `version ... oid ...` content of the actual blob rather than the large file).

unshavedyak 4y ago
> and otherwise not well integrated (it relies on smudge filter and other hooks, so some Git commands are aware of it and others not. You will often see the `version ... oid ...` content of the actual blob rather than the large file). Curious, do you know the solutions here? I've looked into integrating Git LFS-like things into Git and it felt like Git doesn't give you many tools. Smudging, as hacky as it is, feels one of a very small pool of options you have. I want to say Git Annex does something slightly different. Regardless Smudging felt fine due to how difficult Git made it. Are there better ways in your mind?
- remram 4y ago
  I don't think you can do better if you don't want to change Git, no.
  - unshavedyak 4y ago
    Seems like an odd critique of LFS then, no? Your other points are super valid, i've wanted to replace LFS with my own impl for the same reasons _(also it was originally designed for HTTP servers, ugh)_. However the integration is roughly as good as it can be, no? Or rather, Oxen would suffer the same limitations no?
    - remram 4y ago
      I'm not trying to target this critique at LFS specifically and absolve Git. It might be that LFS is a bad product because of Git's limited extension points, it remains a bad product.

olvy0 4y ago

In addition to what was said above: In later versions, LFS doesn't play nice with misconfigured internal networks based on Microsoft's Azure / Active Directory. The LFS project dropped support for NTLM (for good reasons) but it's difficult to convince IT to switch to Kerberos. Our team have been trying to do that for several weeks now. So the result is that if we want to use LFS we are stuck with old versions of LFS which still supported NTLM, which necessitate using old versions of GIT which supported this old version of LFS, else we run into all sorts of weird error messages. Also the same combination of old GIT + old LFS works perfectly on some machines, and fails with some weird error messages on others. Possibly due to another IT misconfiguration or something. I've been wanting for a while now to check if I can fork LFS and fix this, unfortunately my knowledge of golang is very sketchy. Hopefully we'll manage to convince IT to drop NTLM support from the on-premise Azure.

e12e 4y ago
> Hopefully we'll manage to convince IT to drop NTLM support from the on-premise Azure. What year is this? ;) On a more serious note - ntlm, really? And with azure ad, not a regular (old) on-premise domain? I know it can be hard to drag windows setups into the future (hello WINS!) - But why keep ntlm auth in a post windows 7 world? Legacy systems without ldap/radius support? Genuinely curious (and a little horrified).
- olvy0 4y ago
  Tell me about it ;) Apparently, it's related to the few win 7 machines that are still present somewhere in the organization for various reasons. All of them are airgapped (I hope) so there's no real reason to keep NTLM, but nobody has yet made the decision. Working on getting to the guy in charge to authorize it. Sigh.

sam_goody 4y ago

I seem to always be running into git-lfs related issues. The latest include.. If one file doesn't resolve, for any reason, git-lfs will block the entire pull. No way to say "get whatever you could, and the fact that aws is not serving that file is not a blocker" Git clone does not pull the entire lfs history of an image. Lacking the entire history, lfs will fail consistently on future actions.