3 ms·
I imagine that the compression is only for saving time during transfer. That, and hopefully you could decompress just what you needed. I.E. a tarball of compr
by blhack 17y ago
I imagine that the compression is only for saving time during transfer.
That, and hopefully you could decompress just what you needed. I.E. a tarball of compressed articles.
- _delirium 17y agoA lot of researchers just stream the decompressed stream directly from 7z into their analysis scripts. If you were to actually decompress to a giant XML file first, you'd both: 1. need a 6-TB drive; and 2. start getting disk I/O as a big bottleneck. In a lot of cases, the analysis scripts, rather than decompression, are the bottleneck anyway: 7z can feed you data faster than your XML parser and scripts can consume it.