4 ms·
There is opportunity for a powerful tool to handle the very common bad practice of including giant binary blobs which don't belong in version control. I'd like
by colechristensen 13y ago
There is opportunity for a powerful tool to handle the very common bad practice of including giant binary blobs which don't belong in version control.
I'd like to see a second class of files which are only checksummed (or timestamped) on 'status', 'diff', 'add' and then binary-diffed on commit for possible compression (or perhaps deduped with checksums of blocks) with features like 'git/hg binary-add somefile.jar' to differentiate.
- mintplant 13y agoCheck out git-annex: https://git-annex.branchable.com/how_it_works/ https://git-annex.branchable.com/how_it_works/ It operates under a similar principle (committing hashes rather than file contents) but stores the actual file contents elsewhere rather than using a compression/deduplication within the repo itself.
- colechristensen 13y agoI have. It's neat, but it's more about replacing something like Dropbox than it is replacing git (despite being based on git). It also isn't particularly stable, though I have nothing but respect for the developer.
- AceJohnny2 13y agoAs I understand it, it became something to replace Dropbox after Joey realized that git-annex's task of distributed storage of large files intersected with the Dropbox use-case, but it didn't start that way, and IMHO doesn't exclude the parent's suggestion. I backed Joey's kickstarter, but to my shame I have yet to give git-annex-assistant a spin.
- tomlu 13y ago> There is opportunity for a powerful tool to handle the very common bad practice of including giant binary blobs which don't belong in version control Why do you say it's bad practice? Game assets come to mind, these aren't derived artifacts and you can't build the game without them.
- wtallis 13y agoUsually, you can build the game without them, up to the final packaging stage. Almost any game that is built on an engine can have all of the asset files swapped out without having to recompile anything, and as long as the file format and structure for assets is relatively stable there doesn't have to be a tight coupling between engine versions and asset versions except when doing QA and benchmarking. This is one of the cases where git's inability to handle rewriting a public history is a pain. The art and programming departments should be able to develop along separate branches, and the programmers should be able to pull a tree from art that has everything between tags squashed (but in a reversible fashion in case there's a need to bisect something later).
- tomlu 13y agoThere usually are dependencies between the game and its assets, but sure, these dependencies can be managed. Historically I've dealt with the problem by keeping code and assets in separate repositories (using different version control systems). While it's worthwhile to do so in order to use git for code it really is a pain.
- colechristensen 13y agoExactly my initial point. It is really worthwhile to treat code and binary data differently, and the tools don't exist to make it not-so-painful. To answer a couple of levels up, it's not bad practice because it isn't useful... there are many circumstances where having big binary blobs versioned and associated with code. It is bad practice because the tools are all written _for_ code and can get nasty when filled with binaries. So there's a problem and a solution, but nobody seems to have put together the right magic yet.
- Locke1689 13y agoThere is opportunity for a powerful tool to handle the very common bad practice of including giant binary blobs which don't belong in version control. It's only a bad practice because our tools don't support it properly. There seems no reasonable argument that there is some predefined size limit that all assets in version control must, by natural law, fall under. The criteria I have for version controlled assets is they must 1) be versionable, 2) have a comparison tool, and 3) be strongly connected to the other assets under control.
- colechristensen 13y ago>It's only a bad practice because our tools don't support it properly. Yes, exactly.