3 ms·
Eh, it's not that hard to imagine given how rare it is to zip 81k files of around 1kb each.
by darby_eight 3y ago
Eh, it's not that hard to imagine given how rare it is to zip 81k files of around 1kb each.
- iscoelho 3y agoNot that rare at all. Take a full disk zip/tar of any Linux/Windows filesystem and you'll encounter a lot of small files.
- darby_eight 3y agoOk? How are you comparing these systems to the benchmark so they might be considered relevant? Compressing "Lots of small files" describes an infinite variety of workloads. To achieve anything close to the benchmark you'd need to specifically only compress only small files in a single directory of an average small size. And even the contents of those files would have large implications as to expected performance....
- iscoelho 3y agoMy comment is not making any claims about that. It's just a correction that filesystems with "81k 1KB files" are indeed common.
- darby_eight 3y agoIf that were true, surely it would make sense to demonstrate this directly rather than with a contrived benchmark? The issue is not the preponderance of small files but rather the distribution of data shapes.
- viraptor 3y agoThat's basically any large source repo.
- fbdab103 3y agoZipping up a project directory even without git can be a big file collection. Python virtual environment or node_modules, can quickly get into thousands of small files.
- OttoCoddo 3y agoReading many files (81K in this test) is way slower than reading just one big file. For bigger files, Pack is much faster. Here is a link to some results from a kind contributor: https://forum.lazarus.freepascal.org/index.php/topic,66281.msg507177.html#msg507177 https://forum.lazarus.freepascal.org/index.php/topic,66281.m... (Too long to post here)