4 ms·

I'm surprised they aren't using an FPGA solution

• hbbio 4y ago
  Knowing such implementation exists, is open source, and since they don't state what they're using... you may just be right https://pure.tudelft.nl/ws/portalfiles/portal/94907930/Jianyu_zstd_compression.pdf https://pure.tudelft.nl/ws/portalfiles/portal/94907930/Jiany...
• pclmulqdq 4y ago
  FPGAs are really expensive in volume compared to other options, so it's not surprising to me.
  • londons_explore 4y ago
    But on an FPGA you can probably implement super fast and effective compression. If it is far enough above the curve of speed Vs ratio, then it will pay for itself in saved storage. Remember there is no need for said FPGA to be in every machine - in a data center with ten's of gigabits of bandwidth to every node, you can send data to another machine for compression and receive back the compressed data to store.