4 ms·
Is there an easy way to use these accelerators from the CLI? Sometimes I have to decompress several TBs of gzip files, but I don't want to rollout my own decomp
by estebarb 3y ago
Is there an easy way to use these accelerators from the CLI? Sometimes I have to decompress several TBs of gzip files, but I don't want to rollout my own decompressor in C. I know that Graviton 2 includes a compression accelerator as well, but no idea how to use it (easily).
- xxs 3y agoThe accelerator (at least this particular use) won't help decompression.
- mxmlnkn 3y agoIf you have large gzip files to decompress, I can recommend igzip[1] for fastest single-core decompression or my pet project rapidgzip[2] for effective multi-core decompression, both have simple CLI tools. For using accelerators (QAT, I assume, because the article is about this) from the CLI, then QATzip[3] comes with a command line tool qatzip which can be used as described in the project ReadMe. I didn't test it, though, as I have no QAT-enabled device. [1] https://github.com/intel/isa-l/tree/master/igzip https://github.com/intel/isa-l/tree/master/igzip [2] https://github.com/mxmlnkn/rapidgzip https://github.com/mxmlnkn/rapidgzip [3] https://github.com/intel/QATzip#test-qatzip https://github.com/intel/QATzip#test-qatzip