5 ms·
You are right. GIT is not DVFS, its DVCS. It was made to track source code, not binary data. If you are putting binary to DVCS, you are doing something wrong.
by Borg3 2y ago
You are right. GIT is not DVFS, its DVCS. It was made to track source code, not binary data. If you are putting binary to DVCS, you are doing something wrong.
But, there are industries that need it, like game industry. So they should use tool that allow that. I heard that Plastic-SCM is pretty decent at it. Never used it so cant tell personally.
Replacing GIT is such a stupid idea. There is no ONE tool to handle all cases. Just use right one for your workflows. I, for example, have a need to version binary files. I know GIT handles them badly, but I really like the tool. Solution? I wrote my own simple DVFS tool for that usecase: dot.exe (138KB)
Its very simple DVFS for personal use, peer to peer syncing (local, TCP, SSH). Data and Metadata are SHA-1 checksumed. Its pretty speedy for my needs :)
After weeks of use use I liked it so much, I added pack storage to handle text files and moved all my notes from SVN to DOT :)
- ozim 2y agoSecond that way of thinking, for me GIT is as good as it gets for versioning text files. Not handling binary files is not a downside for me because GIT should not be a tool to handle binary files versioning and we should use something else for that.
- fragmede 2y agoWhat do you use when you have a tiny .png or .jpg that needs to live alongside your source code now?
- nolist_policy 2y agoJust put it in your git repo.
- ozim 2y agoI can put a binary file in GIT repo especially small ones and ones that don't change - things that people want are "handling binary files well", whatever that means, but putting big binaries in GIT or a lot of binary files or versioning them is not the use case for GIT.
- kreetx 2y agoI think what they mean by "handling binary files well" is being able to selectively not clone/pull them. Otherwise, binary files are handle as any other files, except you are not shown a diff of them by default.
- Borg3 2y agoYou are wrong.. Just start putting files of GBs in size and you will see what happens.
- kreetx 2y agoSo I just added a 2.4GB iso and other than taking longer nothing happened. Push and clone also seem to work.
- Borg3 2y agoRight :) I forgot that everyone is in 64bit systems these days. This change a lot. Anyway, GIT uses single blob for single file content. This is all right, if we speak about source code and text files, as those are not large. Thing changes for arbitrary binary files.
- kreetx 2y agoAFAIK, not much of anything changes (perhaps name some of these changes?). And what is wrong with a single blob? This seems work great for deduplication, if you were using git to manage a photo library and happen to add something twice.
- Borg3 2y agoThere is several issues depending where you run git on. For example mmap() can be problematic on large files in 32bit OS. On Cygwin it gets even worse. As for deduplication, I do not think so. If you have single blob, lets say 1GB and you change just 1B, whole blob changes and no more dedup. If you use basic method of static block size, lets say 512KB, this will work much better. Futher, there are more advanced techniques to handle dedups like roling checksums to carve out even smaller sub-block.
- nmz 2y agoDVCS stands for distributed version control system, it has nothing to do with source code? Maybe you're confusing it with SCM which are source control managers, that's the only ones that handle strict source only, but scm can mean other things.
- Borg3 2y agoHard to say.. For me DVCS is more advanced version of DVFS. DVCS can do branching and merging, provides more metadata for revisions etc.. DVFS just do pretty much one thing, store binary blobs. And because binary blobs cannot be easly merged, I would not use it for storage here. But I guess, its just me :)
- nmz 2y agoWhat is this dvfs you talk about? desktop virtual file system? Dynamic voltage and frequency scaling? Google doesn't know, or well, the first page doesnt know. DVCS stands for distributed version control system, it has nothing to do with binary blobs, whether or not it supports it, is an implementation detail.
- Borg3 2y agoVFS = Versioned File System; D means Distributed.
- pantalaimon 2y agoIsn't there git-annex [0] if you want to store large binary files? [0] https://git-annex.branchable.com/ https://git-annex.branchable.com/
- Borg3 2y agoYeah I know about git-annex. It might be good solution for big data. In my case, I do NOT want to decouple storage from metadata. I want single repo for single project that is self-contained. Easier to manage, its truly distributed. No need to bother w/ backups because every replica have everything allready. Its good model for several GBs of data.