42 ms·

So how do largish binaries work in git nowadays? When I last looked (several years ago), the answer was essentially 'dont do it' or 'use this other tool to sort

by roel_v 9y ago

So how do largish binaries work in git nowadays? When I last looked (several years ago), the answer was essentially 'dont do it' or 'use this other tool to sort of make it work'. Can I store a few gigs of data (alongside my source code) in git?

nosefouratyou 9y ago
Git-annex might be of interest to you. https://git-annex.branchable.com/ https://git-annex.branchable.com/ I don't know how the situation has changed in regards to binaries, or if this is a good suggestion, I just remember reading about git-annex when it came out a few years ago.
- drothlis 9y ago
  Or git lfs, which is even supported by / integrated into github and other git hosting services.
- deleted 9y ago
  [deleted]
gumby 9y ago
If you're willing to use github you can use lfs. Some binaries can be serialized so they at least function as text files for vanilla git's benefit. I did a project where we used LibreOffice or OpenOffice and we had a hook than unzipped the contents and flattened them into text for checking in and reversed that when checking out. It worked, technically, but we lost most of the benefits of git (tiny changes would lead to changes to several binary tables which lead to incomprehensible diffs). On the other hand the same project used gnuCAD which saves its files as text -- which was awesome: we made some edits (id changes and the like mostly) with sed and awk! The big binaries problem is the sole reason anybody buys Perforce. It's horrible, but it does handle this case, and for folks like game developers there's little alternative. I really do wish I could use git with solid works and Cadence files. Solidworks is the worst: their files are build with some Microsoft structured file library, and contain hard-coded paths; their dreadful proprietary SCM tool understands this and edits the file while checking it out to fit it to your local machine. Aaagh! And it's all centralized. Recently we had an internet outage for a repo hosted on github: people still got work done and collaborated, pushing to and pulling from each others' machines until we had external access restored the next day. With FB down it was a pretty productive day!
- daxelrod 9y ago
  You do need to use something centralized, but LFS is not specific to GitHub. Bitbucket, GitLab, and Gitea also support it. Full implementation list https://github.com/git-lfs/git-lfs/wiki/Implementations https://github.com/git-lfs/git-lfs/wiki/Implementations

by roel_v 9y ago

nosefouratyou 9y ago

Git-annex might be of interest to you. https://git-annex.branchable.com/ https://git-annex.branchable.com/ I don't know how the situation has changed in regards to binaries, or if this is a good suggestion, I just remember reading about git-annex when it came out a few years ago.

drothlis 9y ago
Or git lfs, which is even supported by / integrated into github and other git hosting services.
deleted 9y ago
[deleted]

gumby 9y ago

If you're willing to use github you can use lfs. Some binaries can be serialized so they at least function as text files for vanilla git's benefit. I did a project where we used LibreOffice or OpenOffice and we had a hook than unzipped the contents and flattened them into text for checking in and reversed that when checking out. It worked, technically, but we lost most of the benefits of git (tiny changes would lead to changes to several binary tables which lead to incomprehensible diffs). On the other hand the same project used gnuCAD which saves its files as text -- which was awesome: we made some edits (id changes and the like mostly) with sed and awk! The big binaries problem is the sole reason anybody buys Perforce. It's horrible, but it does handle this case, and for folks like game developers there's little alternative. I really do wish I could use git with solid works and Cadence files. Solidworks is the worst: their files are build with some Microsoft structured file library, and contain hard-coded paths; their dreadful proprietary SCM tool understands this and edits the file while checking it out to fit it to your local machine. Aaagh! And it's all centralized. Recently we had an internet outage for a repo hosted on github: people still got work done and collaborated, pushing to and pulling from each others' machines until we had external access restored the next day. With FB down it was a pretty productive day!

daxelrod 9y ago
You do need to use something centralized, but LFS is not specific to GitHub. Bitbucket, GitLab, and Gitea also support it. Full implementation list https://github.com/git-lfs/git-lfs/wiki/Implementations https://github.com/git-lfs/git-lfs/wiki/Implementations