3 ms·
The FPGA is programmable silicon. It's not as fast as a regular CPU, but it can be re-programmed with whatever algorithms you want. Currently one big bottlenec
by rodgerd 6y ago
The FPGA is programmable silicon. It's not as fast as a regular CPU, but it can be re-programmed with whatever algorithms you want.
Currently one big bottleneck for data processing is moving the data from the drive, over a storage link, into memory, and then back to the drive.
By putting a programmable processor on the drive, you can eliminate that overhead by putting some of your processing algorithms on the storage.
So, for example, if you were running a Hadoop cluster, you could have your common Hadoop algorithms baked into the processor of the drive. Every drive now becomes a Hadoop processing accelerator. Rather than pulling a full dataset into main memory over that very slow link, you run some of the job on each drive, and return only the data you need. Every drive has its own processing power, so the more data you have, the more grunt you have, and you're eliminating the slowest steps.
Because the FPGA is reprogrammable, you can change which algorithms you push down to the drive as you change your workloads over time. Every disk you add becomes a specialized big data, ML, whatever, processing unit.