4 ms·

How about a full line rate (2x 10G ports?) key-value store database on stock x86 server? It would be cheaper than the FPGA solution. OS overhead a problem? Ei

How about a full line rate (2x 10G ports?) key-value store database on stock x86 server? It would be cheaper than the FPGA solution. OS overhead a problem? Either write the application as an OS module, or use RDMA to avoid the OS. One thing I realized long ago is that FPGA and CPU have very nearly the same design constraints. CPU has cache, FPGA has block-ram. CPU has a DRAM interface, as does an FPGA. CPU has serdes (PCIe), as does an FPGA. CPU has a lot of overhead for Tomasulo's algorithm (it boils down access to many-port memories), but FPGA has a lot of overhead for configuration memory. Except for some massively parallel simple algorithms, CPU is just as capable as FPGA (and in fact is usually much more versatile and easier to program).

mtanski 11y ago
Agreed. It seams to me like the both CPU and FPGA would be both bound by the speed of their memories for the KV store. That's why I'm not sure why it would be beneficial to put it on a FPGA. Skimming their summary, conclusion & comparison I can't really find a a good answer to it. Not saying it's a cool project but I don't see a practical case / nor pushing anything forward. I imagine a pretty low-end server class x86 processor should be able to saturate most network links. There prob is a fair overhead going from network device, memory, OS, process and back out. But you could have your KV run in kernel space or as a real time process with dedicated core(s) / network device (memory mapped).
- bravo22 11y ago
  In an FPGA based design you can have many more memory channels and have them be dedicated for this purpose. Packet processing can also be done on the wire, giving you deterministic low latency access to memory storage.
wmf 11y ago
RDMA key-value has been done: https://www.usenix.org/conference/nsdi14/technical-sessions/dragojevic https://www.usenix.org/conference/nsdi14/technical-sessions/... And DPDK: https://www.usenix.org/conference/nsdi14/technical-sessions/presentation/lim https://www.usenix.org/conference/nsdi14/technical-sessions/... And DPDK+GPU: http://kay21s.github.io/megakv/ http://kay21s.github.io/megakv/ It would be interesting to see a fair comparison of these to determine the real value of exotic technology.
cbsmith 11y ago
> Except for some massively parallel simple algorithms You mean like when you are implementing a massive key-value store? ;-)
nxmehta 11y ago
Actually, the design constraints are quite different. 1. FPGAs have between 1-2 orders of magnitude higher on-chip memory bandwidth. Imagine lighting up all of the block RAMs on your chip. That blows away the memory performance of a CPU. It's not even close. 2. FPGAs have significantly more parallel compute resources (if your problem isn't purely memory bandwidth limited). Even if it's not stupidly data-parallel, there are several clever design choices you can make to extract a lot of parallelism. It is, as you point out, much harder to program an FPGA. God save me if I have to run SP&R tools again in my life. However, if you have the resources and know what you are doing you will get 10-1000x more performance from an FPGA. Just look at any FCCM paper from the last 20 years. The more apt comparison is FPGAs vs ASSPs. Those are what have been eating FPGA's lunch for the last decade.

mtanski 11y ago

Agreed. It seams to me like the both CPU and FPGA would be both bound by the speed of their memories for the KV store. That's why I'm not sure why it would be beneficial to put it on a FPGA. Skimming their summary, conclusion & comparison I can't really find a a good answer to it. Not saying it's a cool project but I don't see a practical case / nor pushing anything forward. I imagine a pretty low-end server class x86 processor should be able to saturate most network links. There prob is a fair overhead going from network device, memory, OS, process and back out. But you could have your KV run in kernel space or as a real time process with dedicated core(s) / network device (memory mapped).

bravo22 11y ago
In an FPGA based design you can have many more memory channels and have them be dedicated for this purpose. Packet processing can also be done on the wire, giving you deterministic low latency access to memory storage.

wmf 11y ago

RDMA key-value has been done: https://www.usenix.org/conference/nsdi14/technical-sessions/dragojevic https://www.usenix.org/conference/nsdi14/technical-sessions/... And DPDK: https://www.usenix.org/conference/nsdi14/technical-sessions/presentation/lim https://www.usenix.org/conference/nsdi14/technical-sessions/... And DPDK+GPU: http://kay21s.github.io/megakv/ http://kay21s.github.io/megakv/ It would be interesting to see a fair comparison of these to determine the real value of exotic technology.

cbsmith 11y ago

> Except for some massively parallel simple algorithms You mean like when you are implementing a massive key-value store? ;-)

nxmehta 11y ago

Actually, the design constraints are quite different. 1. FPGAs have between 1-2 orders of magnitude higher on-chip memory bandwidth. Imagine lighting up all of the block RAMs on your chip. That blows away the memory performance of a CPU. It's not even close. 2. FPGAs have significantly more parallel compute resources (if your problem isn't purely memory bandwidth limited). Even if it's not stupidly data-parallel, there are several clever design choices you can make to extract a lot of parallelism. It is, as you point out, much harder to program an FPGA. God save me if I have to run SP&R tools again in my life. However, if you have the resources and know what you are doing you will get 10-1000x more performance from an FPGA. Just look at any FCCM paper from the last 20 years. The more apt comparison is FPGAs vs ASSPs. Those are what have been eating FPGA's lunch for the last decade.