3 ms·
> These are all features that primarily have value for repos that push the limits on size, like big monorepos (with a huge amount of files) or game development
by webmaven 5y ago
> These are all features that primarily have value for repos that push the limits on size, like big monorepos (with a huge amount of files) or game development (with big asset files). But get it right, and you could massively cut down the time it takes to check out a branch and build it.
Would this be a good fit for large machine learning data sets?
Every time a new model comes out that touts having been trained on something like a quarter of a billion images, I ask myself "how the heck does someone manage and version a pile like that?"
- zem 5y agowould you ever need to version individual images? at a high level, you could version the collection as a whole by filtering by timestamp, and moving deleted files to a "deleted" directory, or maintaining filename and timestamps in a database. I'm sure there are lots of corner cases that would come up when you actually tried to build such a system, but I don't think the overall scheme needs to be as conceptually complex as source code version control
- webmaven 5y ago> would you ever need to version individual images? Potentially. It would depend on the image format and what metadata is (or can be) included in the file.