3 ms·
What's the reason for using bz2 here? Wouldn't it be faster to do a one off conversion to zstd? It beats bzip2 in every metric at higher compression levels as f
by viraptor 1y ago
What's the reason for using bz2 here? Wouldn't it be faster to do a one off conversion to zstd? It beats bzip2 in every metric at higher compression levels as far as I know.
- declan_roberts 1y agoThat assumes you're processing the data more than once.
- deleted 1y ago[deleted]
- rwaksmunski 1y agoCommon Crawl delivers the data as bz2. Indeed I store intermediate data in zstd with ZFS.