3 ms·

Where is the inefficiency, Git LFS spec or some particular server? I think space is not a spec issue at all, an LFS server can use any deduplicating storage un

Where is the inefficiency, Git LFS spec or some particular server? I think space is not a spec issue at all, an LFS server can use any deduplicating storage under the hood The part where filters are used is not elegant, porcelain for LFS could be better.

unshavedyak 4y ago
For me it's just that Git LFS defaults to a server. I'm rewriting it largely because i loathe Git-LFS needing to hit some HTTP server. Running one locally because they defaulted to a Github Webserver mindset frustrates me.
- throwaway290 4y ago
  Personally if files are large I prefer to store all of them and the entire history of their changes somewhere else, so a server makes sense to me...
  - unshavedyak 4y ago
    Yea, i just don't want to be forced to. Ie to me storing them in another folder, a network filesystem, some SSH filesystem, a dumb KV store like S3, etc are all viable storage locations to it. So that's what i'm writing because i'm just picky, i guess heh.
    - throwaway290 4y ago
      The spec says pointer is just an URL so it could probably be a file:... or s3:... URL if you have the right resolver logic
remram 4y ago
> I think space is not a spec issue at all, an LFS server can use any deduplicating storage under the hood The client doesn't deduplicate, so you have to download whole objects and store them. Of course you can clean up the older version of objects, but that will force you to re-download if you change branch or go back in the history.
- throwaway290 4y ago
  How would you deduplicate on the client at all? If I have multiple copies of the same file in different subdirectories, and I need all of them for work, there's no way around storing all of them as multiple copies on my machine, right? Unless you do some symlinking or hardlinking magic, but that will likely break platform and software compatibility so I wouldn't want it to happen either... Similar with history, how can I have my cake (not have previous versions occupy space locally) and eat it too (have them readily available locally without download)? Or do you mean something like chunking binary files and then deduping chunks? Is it an LFS limitation, or simply that no Git LFS client/server implementation does it yet? It's still fairly new after all.

unshavedyak 4y ago

For me it's just that Git LFS defaults to a server. I'm rewriting it largely because i loathe Git-LFS needing to hit some HTTP server. Running one locally because they defaulted to a Github Webserver mindset frustrates me.

throwaway290 4y ago
Personally if files are large I prefer to store all of them and the entire history of their changes somewhere else, so a server makes sense to me...
- unshavedyak 4y ago
  Yea, i just don't want to be forced to. Ie to me storing them in another folder, a network filesystem, some SSH filesystem, a dumb KV store like S3, etc are all viable storage locations to it. So that's what i'm writing because i'm just picky, i guess heh.
  - throwaway290 4y ago
    The spec says pointer is just an URL so it could probably be a file:... or s3:... URL if you have the right resolver logic

remram 4y ago

> I think space is not a spec issue at all, an LFS server can use any deduplicating storage under the hood The client doesn't deduplicate, so you have to download whole objects and store them. Of course you can clean up the older version of objects, but that will force you to re-download if you change branch or go back in the history.

throwaway290 4y ago
How would you deduplicate on the client at all? If I have multiple copies of the same file in different subdirectories, and I need all of them for work, there's no way around storing all of them as multiple copies on my machine, right? Unless you do some symlinking or hardlinking magic, but that will likely break platform and software compatibility so I wouldn't want it to happen either... Similar with history, how can I have my cake (not have previous versions occupy space locally) and eat it too (have them readily available locally without download)? Or do you mean something like chunking binary files and then deduping chunks? Is it an LFS limitation, or simply that no Git LFS client/server implementation does it yet? It's still fairly new after all.