4 ms·

I'm trying to understand the implications of this thing. Please forgive my ignorance. So, a DPU (data progressing unit) is basically a daughter board that has m

I'm trying to understand the implications of this thing. Please forgive my ignorance. So, a DPU (data progressing unit) is basically a daughter board that has multiple CPUs, big network, modest RAM, but also NVMe capabilities? There's a bit of architectural stuff i didn't understated in the article. Is this the world going full circle and re introducing mini-pcs of the 80's and 90's era? I understand, I think, the usefulness of having these in a data centre, each customer having their own DPU which would present to them as a bare metal device. I understand, I think, the crypto guys loving this for compute power, easily expandible. I also understand this is not for average consumers... but this is HN... what other uses can we put through this/ what advantages does this physical architecture give us? If anyone could elucidate ....? It looks exciting, but I'm not sure of the scope.

gmueckl 4y ago
I am guessing here, but these boards seem to be geared towards accelerated data transfers to feed GPUs. Some GPU cluster workloads are limited by node to node data transfers that need to go through several slow bus transfers between GPUs in different nodes with traditional computer architectures. These DPUs are spec'ed to act as a kind of programmable beefy DMA controller bridging directly between GPUs and some fast interconnect and storage.
walterbell 4y ago
DPUs/SmartNICs are a partial industry response to AWS Nitro (based on Annapurna acquisition) which performs I/O (network/storage) virtualization. Nvidia DPU is based on Mellanox acquisition AMD DPU is based on Pensado acquisition Intel has an in-house DPU There was a 2018 talk on AWS Nitro, https://www.youtube.com/watch?v=e8DVmwj3OEs https://www.youtube.com/watch?v=e8DVmwj3OEs Cadence summary: https://community.cadence.com/cadence_blogs_8/b/breakfast-bytes/posts/the-aws-nitro-project https://community.cadence.com/cadence_blogs_8/b/breakfast-by... James Hamilton summary: https://perspectives.mvdirona.com/2019/02/aws-nitro-system/ https://perspectives.mvdirona.com/2019/02/aws-nitro-system/ The founder of Annapurna-Nitro is now at https://www.lightbitslabs.com/ https://www.lightbitslabs.com/, which has created the NVMEoF (NVME over Fabric, i.e. Ethernet or FiberChannel) standard. This is implemented via a software driver or hardware accelerator. Both DPUs and NVMEoF can be viewed as an attempt to standardize "composable data center" architectures pioneered by AWS Nitro. > but this is HN... what other uses can we put through this / what advantages does this physical architecture give us? A good place to start is the decade-old NetFPGA SmartNIC research project from University of Cambridge, now in the 5th generation of hardware, with earlier boards sometimes available on eBay. https://netfpga.org/ https://netfpga.org/ A line-rate, flexible, and open platform for research, and classroom experimentation. More than 3,500 NetFPGA systems have been deployed at over 300 institutions in over 60 countries around the world. Several hundred papers have been published: https://netfpga.org/Publications.html https://netfpga.org/Publications.html
- hardwaresofton 4y ago
  Also just to add a tiny bit to this wealth of information: I just recently did an experiment on the difference between non-nitro and nitro-enabled (m4.xlarge vs. m5.xlarge) instances, in a production-ish trendy setup -- Ceph running on Kubernetes (managed by Rook), leveraged to run postgres and pgbench. The increase in performance was around +45% TPS, just from switching to the nitro enabled instance. The absolute amount of TPS wasn't high because the setup was untuned but simply switching created quite the difference. On top of all that, the Nitro instance was actually just slightly cheaper than the non-nitro instance. AWS hit it out of park in the making -- they've been working on it since 2014 supposedly and they certainly built something worth replicating.
  - tiernano 4y ago
    just playing devil's advocate, that 45% is a little skewed though... the M4 instance was a "Intel Xeon E5-2676 v3 Haswell processor" (https://aws.amazon.com/blogs/aws/the-new-m4-instance-type-bonus-price-reduction-on-m3-c4/ https://aws.amazon.com/blogs/aws/the-new-m4-instance-type-bo...) and the M5 is a "Custom Intel® Xeon® Platinum 8175M series processors running at 2.5 GHz" (https://aws.amazon.com/blogs/aws/m5-the-next-generation-of-general-purpose-ec2-instances/ https://aws.amazon.com/blogs/aws/m5-the-next-generation-of-g...). so, you are jumping 2 generations from E5v3 to Gen 1 SP processors, plus going to custom processors for AWS (higher and more sustained turbo). I do agree that Nitro is a big change (less overhead for the hypervisor, faster IO and Networking, etc) but I wonder what the perf diff is just CPU upgrades...
    - hardwaresofton 4y ago
      Hey thanks for the note -- that's a good confounding factor I didn't take into account. > Based on Custom Intel® Xeon® Platinum 8175M series processors running at 2.5 GHz, the M5 instances are designed for highly demanding workloads and will deliver 14% better price/performance than the M4 instances on a per-core basis. If we take AWS at their word here, all other things equal 30% is a pretty decent result (with absolutely no tuning) -- I'll update the post with this caveat! All together though, this does still contribute to the point that Nitro is well worth the money.
yunohn 4y ago
> what advantages does this physical architecture give us? It also significantly decreases load on the main CPU. Makes a huge difference when doing any file/network operations that saturate the CPU for just handling I/O. More - https://www.redhat.com/en/blog/optimizing-server-utilization-datacenters-offloading-network-functions-nvidia-bluefield-2-dpus https://www.redhat.com/en/blog/optimizing-server-utilization...
Nextgrid 4y ago
The DPU can act as a PCI host or as a PCI device. In this case, it's placed into a chassis where there is no host (the backplane is only for power), acts as its own PCI host and talks to NVME SSDs on the bus. This is a demo and is otherwise pretty absurd, but the main use-case of these devices can be to do some pre-processing on the network traffic before handing off to the host. In addition to the ARM cores, these cards have hardware capable of packet filtering/etc so it can be useful in scenarios such as DDoS mitigation where the card can easily filter out malicious traffic even at rates of millions of packets per second, and only pass through legitimate traffic to the host server.
donavanm 4y ago
Your summary is about right, but these can also be used to present (virtualized) nvme devices to the host computer. Imagine presenting arbitrary hardware to a host, and being able to control all of that entirely out of band from the host. Another decent comparison would be older SBCs. Going that route you could build a big storage jbod for presenting blockdevices or blob storage over a network. That’s much cheaper than building a big x86 box with big chassis, x86 cpu, ram etc. And now you have cheaper, fast, NAS for your dc full of virtual hosts.

gmueckl 4y ago

I am guessing here, but these boards seem to be geared towards accelerated data transfers to feed GPUs. Some GPU cluster workloads are limited by node to node data transfers that need to go through several slow bus transfers between GPUs in different nodes with traditional computer architectures. These DPUs are spec'ed to act as a kind of programmable beefy DMA controller bridging directly between GPUs and some fast interconnect and storage.

walterbell 4y ago

DPUs/SmartNICs are a partial industry response to AWS Nitro (based on Annapurna acquisition) which performs I/O (network/storage) virtualization. Nvidia DPU is based on Mellanox acquisition AMD DPU is based on Pensado acquisition Intel has an in-house DPU There was a 2018 talk on AWS Nitro, https://www.youtube.com/watch?v=e8DVmwj3OEs https://www.youtube.com/watch?v=e8DVmwj3OEs Cadence summary: https://community.cadence.com/cadence_blogs_8/b/breakfast-bytes/posts/the-aws-nitro-project https://community.cadence.com/cadence_blogs_8/b/breakfast-by... James Hamilton summary: https://perspectives.mvdirona.com/2019/02/aws-nitro-system/ https://perspectives.mvdirona.com/2019/02/aws-nitro-system/ The founder of Annapurna-Nitro is now at https://www.lightbitslabs.com/ https://www.lightbitslabs.com/, which has created the NVMEoF (NVME over Fabric, i.e. Ethernet or FiberChannel) standard. This is implemented via a software driver or hardware accelerator. Both DPUs and NVMEoF can be viewed as an attempt to standardize "composable data center" architectures pioneered by AWS Nitro. > but this is HN... what other uses can we put through this / what advantages does this physical architecture give us? A good place to start is the decade-old NetFPGA SmartNIC research project from University of Cambridge, now in the 5th generation of hardware, with earlier boards sometimes available on eBay. https://netfpga.org/ https://netfpga.org/ A line-rate, flexible, and open platform for research, and classroom experimentation. More than 3,500 NetFPGA systems have been deployed at over 300 institutions in over 60 countries around the world. Several hundred papers have been published: https://netfpga.org/Publications.html https://netfpga.org/Publications.html

hardwaresofton 4y ago
Also just to add a tiny bit to this wealth of information: I just recently did an experiment on the difference between non-nitro and nitro-enabled (m4.xlarge vs. m5.xlarge) instances, in a production-ish trendy setup -- Ceph running on Kubernetes (managed by Rook), leveraged to run postgres and pgbench. The increase in performance was around +45% TPS, just from switching to the nitro enabled instance. The absolute amount of TPS wasn't high because the setup was untuned but simply switching created quite the difference. On top of all that, the Nitro instance was actually just slightly cheaper than the non-nitro instance. AWS hit it out of park in the making -- they've been working on it since 2014 supposedly and they certainly built something worth replicating.
- tiernano 4y ago
  just playing devil's advocate, that 45% is a little skewed though... the M4 instance was a "Intel Xeon E5-2676 v3 Haswell processor" (https://aws.amazon.com/blogs/aws/the-new-m4-instance-type-bonus-price-reduction-on-m3-c4/ https://aws.amazon.com/blogs/aws/the-new-m4-instance-type-bo...) and the M5 is a "Custom Intel® Xeon® Platinum 8175M series processors running at 2.5 GHz" (https://aws.amazon.com/blogs/aws/m5-the-next-generation-of-general-purpose-ec2-instances/ https://aws.amazon.com/blogs/aws/m5-the-next-generation-of-g...). so, you are jumping 2 generations from E5v3 to Gen 1 SP processors, plus going to custom processors for AWS (higher and more sustained turbo). I do agree that Nitro is a big change (less overhead for the hypervisor, faster IO and Networking, etc) but I wonder what the perf diff is just CPU upgrades...
  - hardwaresofton 4y ago
    Hey thanks for the note -- that's a good confounding factor I didn't take into account. > Based on Custom Intel® Xeon® Platinum 8175M series processors running at 2.5 GHz, the M5 instances are designed for highly demanding workloads and will deliver 14% better price/performance than the M4 instances on a per-core basis. If we take AWS at their word here, all other things equal 30% is a pretty decent result (with absolutely no tuning) -- I'll update the post with this caveat! All together though, this does still contribute to the point that Nitro is well worth the money.

yunohn 4y ago

> what advantages does this physical architecture give us? It also significantly decreases load on the main CPU. Makes a huge difference when doing any file/network operations that saturate the CPU for just handling I/O. More - https://www.redhat.com/en/blog/optimizing-server-utilization-datacenters-offloading-network-functions-nvidia-bluefield-2-dpus https://www.redhat.com/en/blog/optimizing-server-utilization...

Nextgrid 4y ago

The DPU can act as a PCI host or as a PCI device. In this case, it's placed into a chassis where there is no host (the backplane is only for power), acts as its own PCI host and talks to NVME SSDs on the bus. This is a demo and is otherwise pretty absurd, but the main use-case of these devices can be to do some pre-processing on the network traffic before handing off to the host. In addition to the ARM cores, these cards have hardware capable of packet filtering/etc so it can be useful in scenarios such as DDoS mitigation where the card can easily filter out malicious traffic even at rates of millions of packets per second, and only pass through legitimate traffic to the host server.

donavanm 4y ago

Your summary is about right, but these can also be used to present (virtualized) nvme devices to the host computer. Imagine presenting arbitrary hardware to a host, and being able to control all of that entirely out of band from the host. Another decent comparison would be older SBCs. Going that route you could build a big storage jbod for presenting blockdevices or blob storage over a network. That’s much cheaper than building a big x86 box with big chassis, x86 cpu, ram etc. And now you have cheaper, fast, NAS for your dc full of virtual hosts.