4 ms·
Last I checked QAT was limited to 64KB backreferences and independent 128KB blocks. Now you know why they are comparing 16KB payloads only.
by klauspost 3y ago
Last I checked QAT was limited to 64KB backreferences and independent 128KB blocks.
Now you know why they are comparing 16KB payloads only.
- terrelln 3y agoYeah, that is definitely a limitation with QAT. It isn't a great fit for larger data, as it can quickly lose compression ratio due to its smaller window. However, there is a lot of compression done on data that is <64KB. E.g. compression in RocksDB. We also have some half baked ideas to combine a fast SW match finder that only looks for matches >64KB away, and supplements the matches that QAT finds.