3 ms·

Please report back your findings, I am sure I am not the only one wanting to see some real world results

throwaway81523 3y ago
I'd like to know what QAT actually is? Some kind of special purpose hardware extension for LZ and crypto? I used to compress a lot of files using xz and it is slow, but not THAT slow. I have a now-ancient 4 core i7 server and it takes a day or so hours to compress a TB of text. I haven't looked at it in a while and don't remember actual numbers, but the point is that if you don't need realtime response, the cpu usage of these algorithms is tolerable without special hardware.
- jeffbee 3y ago
  It's a PCIe device that is either on a stick (earliest generation), integrated into the platform chipset, or integrated into the CPU (latest generation). Nobody outside Intel really knows what's in it, but you submit requests to it and it encrypts, decrypts, checksums, or compresses something. If this sounds familiar it is because we had stuff like that in the 90s.
  - jevinskie 3y ago
    There is some enum or #define in a header I found recently that suggested there was once QAT acceleration of something XML related! I can’t find any documentation about it though. edit: CPA_ACC_SVC_TYPE_XML https://github.com/ravynsoft/ravynos/blob/ee81203faa28ab3e6a987b1e2c4f3b759d02ebfa/sys/dev/qat/qat_api/include/cpa.h#L419 https://github.com/ravynsoft/ravynos/blob/ee81203faa28ab3e6a...
    - jeffbee 3y ago
      https://github.com/intel/qatlib/blob/main/quickassist/include/cpa.h#L445 https://github.com/intel/qatlib/blob/main/quickassist/includ... It also seems to suggest that it could accelerate hyperscan, but considering the hardware and software are both from Intel and they don't integrate them, maybe it doesn't work or is theoretical.
    - twoodfin 3y ago
      That’s an interesting view into Intel marketing’s ideas about where CPU-bound customers are burning cycles at scale. In that context, XML (presumably parsing) is not a surprise.

I'd like to know what QAT actually is? Some kind of special purpose hardware extension for LZ and crypto? I used to compress a lot of files using xz and it is slow, but not THAT slow. I have a now-ancient 4 core i7 server and it takes a day or so hours to compress a TB of text. I haven't looked at it in a while and don't remember actual numbers, but the point is that if you don't need realtime response, the cpu usage of these algorithms is tolerable without special hardware.

jeffbee 3y ago
It's a PCIe device that is either on a stick (earliest generation), integrated into the platform chipset, or integrated into the CPU (latest generation). Nobody outside Intel really knows what's in it, but you submit requests to it and it encrypts, decrypts, checksums, or compresses something. If this sounds familiar it is because we had stuff like that in the 90s.
- jevinskie 3y ago
  There is some enum or #define in a header I found recently that suggested there was once QAT acceleration of something XML related! I can’t find any documentation about it though. edit: CPA_ACC_SVC_TYPE_XML https://github.com/ravynsoft/ravynos/blob/ee81203faa28ab3e6a987b1e2c4f3b759d02ebfa/sys/dev/qat/qat_api/include/cpa.h#L419 https://github.com/ravynsoft/ravynos/blob/ee81203faa28ab3e6a...
  - jeffbee 3y ago
    https://github.com/intel/qatlib/blob/main/quickassist/include/cpa.h#L445 https://github.com/intel/qatlib/blob/main/quickassist/includ... It also seems to suggest that it could accelerate hyperscan, but considering the hardware and software are both from Intel and they don't integrate them, maybe it doesn't work or is theoretical.
  - twoodfin 3y ago
    That’s an interesting view into Intel marketing’s ideas about where CPU-bound customers are burning cycles at scale. In that context, XML (presumably parsing) is not a surprise.