4 ms·

One can program the NIC to DMA packets directly info the address space allot for FPGA. Once setup, the FPGA should be able to get hold the packets and start p

by srcmap 11y ago

One can program the NIC to DMA packets directly info the address space allot for FPGA. Once setup, the FPGA should be able to get hold the packets and start processing completely without a single CPU cycle use on data plane.

deadgrey19 11y ago
Sure. This is a possibility, although it is a bit round about and there would be an interesting song and dance in the NIC driver. NICs typically are told where to DMA to using descriptor tables programmed into the NIC by the driver. To do this truly without CPU intervention, you would need to write a hardware driver in the FPGA to program the NICs descriptor tables (can't even imagine what a nightmare that would be). Otherwise, you would have to have the CPU involved in setting up and negotiating transfers between the NIC and FPGA and a second driver between the FPGA and software. It's pretty messy either way. And given the proliferation of cheap FPGA enabled NIC's it seems like a non-starter. If the FPGA transceivers are broken out directly, then a simple adapter board would allow the FPGA to talk directly the network and/or memory device.
- gricardo99 11y ago
  > you would have to have the CPU involved in setting up and negotiating transfers between the NIC and FPGA and a second driver between the FPGA and software Plenty of "kernel bypass" and RDMA type functions use shared/user-space memory for "zero-copy" (in reality one copy), operations between NIC and software. If a similar scheme can be used with the FPGA then it would not have too much overhead. I agree, not as direct/efficient as having FPGA serdes I/O go directly to some SPF+/network transceiver, but then you'd also be taking up valuable FPGA gate capacity to run NIC PHY/MAC and standard L2/L3 processing functions that you get from a NIC.
  - deadgrey19 11y ago
    RDMA/kernel bypass NICs work by mapping chunks of RAM and then automatically DMA'ing packets into those chunks. Again, it would be a pretty round about way to give the FPGA access to packets to copy data to RAM, then copy down to the FPGA, then copy back up to RAM. Much simpler/better to let the data stream through the FPGA from/to the wire. In addition, the PHY/MAC layers these days are pretty thin for Ethernet style devices and modern FPGA's are by comparison huge. I'm not saying it can't be done, I'm just saying it seems sub-optimal when FPGA's already have a ton of I/O resources and are already used as NICs. The question as to wether these resources are exposed to the outside world is the salient one.

by srcmap 11y ago

deadgrey19 11y ago

Sure. This is a possibility, although it is a bit round about and there would be an interesting song and dance in the NIC driver. NICs typically are told where to DMA to using descriptor tables programmed into the NIC by the driver. To do this truly without CPU intervention, you would need to write a hardware driver in the FPGA to program the NICs descriptor tables (can't even imagine what a nightmare that would be). Otherwise, you would have to have the CPU involved in setting up and negotiating transfers between the NIC and FPGA and a second driver between the FPGA and software. It's pretty messy either way. And given the proliferation of cheap FPGA enabled NIC's it seems like a non-starter. If the FPGA transceivers are broken out directly, then a simple adapter board would allow the FPGA to talk directly the network and/or memory device.

gricardo99 11y ago
> you would have to have the CPU involved in setting up and negotiating transfers between the NIC and FPGA and a second driver between the FPGA and software Plenty of "kernel bypass" and RDMA type functions use shared/user-space memory for "zero-copy" (in reality one copy), operations between NIC and software. If a similar scheme can be used with the FPGA then it would not have too much overhead. I agree, not as direct/efficient as having FPGA serdes I/O go directly to some SPF+/network transceiver, but then you'd also be taking up valuable FPGA gate capacity to run NIC PHY/MAC and standard L2/L3 processing functions that you get from a NIC.
- deadgrey19 11y ago
  RDMA/kernel bypass NICs work by mapping chunks of RAM and then automatically DMA'ing packets into those chunks. Again, it would be a pretty round about way to give the FPGA access to packets to copy data to RAM, then copy down to the FPGA, then copy back up to RAM. Much simpler/better to let the data stream through the FPGA from/to the wire. In addition, the PHY/MAC layers these days are pretty thin for Ethernet style devices and modern FPGA's are by comparison huge. I'm not saying it can't be done, I'm just saying it seems sub-optimal when FPGA's already have a ton of I/O resources and are already used as NICs. The question as to wether these resources are exposed to the outside world is the salient one.