3 ms·
What overhead does that add in practice? Like a second or so to scan a few GB of tarball? Does that matter?
by chrisseaton 9y ago
What overhead does that add in practice? Like a second or so to scan a few GB of tarball? Does that matter?
- ctur 9y agoIt often is, especially if the tarball is compressed (especially if you use anything but Zstandard) or on a network store rather than fast local ssd. Typically a rule of thumb is a hard drive with a spindle may five around 100mb/sec throughput, so that is likely the bigger bottleneck, followed by cpu if using slower compression algorithms. Zstandard at a high compression level gives a good tradeoff of decompression speed vs underlying physical throughput, be it network or spindle. Even so, yes, the sequential nature of tar is not great for this reason. Squashfs files, which can be used on Linux and Mac (using fuse for OS X), are a very good alternative for archIves, though without some of the features described in the linked article.
- boramalper 9y agoI guess it can be cumbersome as well if there are many small* files (since you need to iterate over all of them) until you find the one you are looking for. Think of it like a linked list. *: As opposed to a few large files.
- noselasd 9y agoyou need to download the entire thing to scan it. if it already resides locally on a seekable media , there's litle drawback