4 ms·

From the README: > Why not Git/Git-LFS, libgit2, or SVN? > Disadvantages: > (Without Git-LFS): Heavy cost with zipping, packing, and delta-compression for la

From the README: > Why not Git/Git-LFS, libgit2, or SVN? > Disadvantages: > (Without Git-LFS): Heavy cost with zipping, packing, and delta-compression for larger files Given the caveat (without Git-LFS) it seems odd to include this in the list > If not properly tracked, binaries become accidentally part of "base" history That's a big "if", and not an inherent problem. This could easily be resolved by any good design-focused UI (e.g. SnowTrack), so this seems a poor argument against using Git as a backend. > Removing older commits is cumbersome due to Gits commit hashing integrity This (like the first bullet point) does not apply to Git-LFS. > Complicated rewriting history procedure What? > Issues with binaries >4GB on Windows A known bug in Git-LFS that they're working to fix. There are workarounds provided in the linked tickets (that could be leveraged by a UI / abstraction layer like SnowTrack). This is the first item in the bullet list that is a real disadvantage of Git LFS, but the workaround for it seems much less effort than developing a new VCS backend from scratch. > Slow in binary modification detection I'm not sure if this applies to Git or Git LFS; there's little detail provided. But if it's significant, this is probably the only really compelling disadvantage listed. > Git uses a restrictive license And finally we see the real reason for not using Git. --- NOTE: I don't mean to make out that building an alternative VCS to Git is not worth pursuing. Nor that it needs any specific justification. Just that listing a justification that seems (to me) mostly disingenuous is worth pointing out.

sebastian_io 6y ago
Thanks for listing out the points you disagree with! The project is still in early alpha, so is the Readme. Therefore anything that is ambiguous or not clear, is worth to address. The main requirement is performance (a missing point in your list). If Git would be a good candidate as a versioning system for DCC software packages, it would have been picked up by now, but it didn't happen, among others because of the reasons listed above. Git addresses a completely different target audience and lifecycle than SnowFS. The commit hash integrity is a problem in CG/VFX productions, so is the 4GB limitation, as well as the I/O performance for large binary files. The fact that these issues are still there are fully understandable, given the responsibility and dependencies of this project. That's why SnowFS tries to address the niche requirements with its light implementation. In terms of the license, this is intentionally the weakest argument of all. It doesn't prevent anyone under the GPL to ship Git as an external program with a commercial software, same counts for libgit2 with its linking exception. So there is not even a real benefit here. But the chosen MIT license is an open invitation for everyone. P.S. Certain features and technical solutions will be feature-proposed to libgit2
- unqueued 6y ago
  Have you looked into git-annex? Git annex lets you track references to binary files, only using git for storing references to file hashes. And you can use custom back ends to efficiently store differential data. For example, I have an annex repo that stores about 150G of text files, but it uses bup to compress it down to about 20G, while I can still have access to different versions via git. 1: https://git-annex.branchable.com/special_remotes/bup/ https://git-annex.branchable.com/special_remotes/bup/
  - sebastian_io 6y ago
    Impressive numbers! Unfortunately I know git-annex only on paper. I gave it a try a while ago, but it was a bumpy start, admittedly most likely user-error. Would you mind sharing some details about it (e.g. file numbers, etc)? Can I invite you for a chat? Doesn't need to be long, but might be more suitable for a chat
    - unqueued 6y ago
      Sure, how would you like to get in touch? You have a discord, right? I actually was looking at your project and was thinking of opening a simple PR. (same username) I have some more examples git-annex repos: This is an annex repo I made of this popular abandonware website: https://github.com/unqueued/repo.macintoshgarden.org-fileset https://github.com/unqueued/repo.macintoshgarden.org-fileset And some podcasts https://github.com/unqueued/radiolab-fileset https://github.com/unqueued/radiolab-fileset https://github.com/unqueued/ratholeradio-archive https://github.com/unqueued/ratholeradio-archive What's cool is that people can use standard pull requests to add files to the repo. And the repo itself is small, but it can represent huge filesets. Datalad has some really fascinating medical imaging data repos that are massive (https://www.datalad.org/datasets.html https://www.datalad.org/datasets.html). If you wanna see a really good example of a repo with versioned binary files, check this out the git annex repo of previous git-annex binary releases: https://downloads.kitenet.net/.git/ https://downloads.kitenet.net/.git/ You can just use standard git workflows to see previous revisions of a file (well, previous hashes) but it is really easy to hook into.

sebastian_io 6y ago

Thanks for listing out the points you disagree with! The project is still in early alpha, so is the Readme. Therefore anything that is ambiguous or not clear, is worth to address. The main requirement is performance (a missing point in your list). If Git would be a good candidate as a versioning system for DCC software packages, it would have been picked up by now, but it didn't happen, among others because of the reasons listed above. Git addresses a completely different target audience and lifecycle than SnowFS. The commit hash integrity is a problem in CG/VFX productions, so is the 4GB limitation, as well as the I/O performance for large binary files. The fact that these issues are still there are fully understandable, given the responsibility and dependencies of this project. That's why SnowFS tries to address the niche requirements with its light implementation. In terms of the license, this is intentionally the weakest argument of all. It doesn't prevent anyone under the GPL to ship Git as an external program with a commercial software, same counts for libgit2 with its linking exception. So there is not even a real benefit here. But the chosen MIT license is an open invitation for everyone. P.S. Certain features and technical solutions will be feature-proposed to libgit2

unqueued 6y ago
Have you looked into git-annex? Git annex lets you track references to binary files, only using git for storing references to file hashes. And you can use custom back ends to efficiently store differential data. For example, I have an annex repo that stores about 150G of text files, but it uses bup to compress it down to about 20G, while I can still have access to different versions via git. 1: https://git-annex.branchable.com/special_remotes/bup/ https://git-annex.branchable.com/special_remotes/bup/
- sebastian_io 6y ago
  Impressive numbers! Unfortunately I know git-annex only on paper. I gave it a try a while ago, but it was a bumpy start, admittedly most likely user-error. Would you mind sharing some details about it (e.g. file numbers, etc)? Can I invite you for a chat? Doesn't need to be long, but might be more suitable for a chat
  - unqueued 6y ago
    Sure, how would you like to get in touch? You have a discord, right? I actually was looking at your project and was thinking of opening a simple PR. (same username) I have some more examples git-annex repos: This is an annex repo I made of this popular abandonware website: https://github.com/unqueued/repo.macintoshgarden.org-fileset https://github.com/unqueued/repo.macintoshgarden.org-fileset And some podcasts https://github.com/unqueued/radiolab-fileset https://github.com/unqueued/radiolab-fileset https://github.com/unqueued/ratholeradio-archive https://github.com/unqueued/ratholeradio-archive What's cool is that people can use standard pull requests to add files to the repo. And the repo itself is small, but it can represent huge filesets. Datalad has some really fascinating medical imaging data repos that are massive (https://www.datalad.org/datasets.html https://www.datalad.org/datasets.html). If you wanna see a really good example of a repo with versioned binary files, check this out the git annex repo of previous git-annex binary releases: https://downloads.kitenet.net/.git/ https://downloads.kitenet.net/.git/ You can just use standard git workflows to see previous revisions of a file (well, previous hashes) but it is really easy to hook into.