4 ms·

> Have a look at Netflix's 800Gbps CDN node presentation. If you want to hit high transfer speeds, your CPU may not have the bandwidth to read from your device

by rewmie 3y ago

> Have a look at Netflix's 800Gbps CDN node presentation. If you want to hit high transfer speeds, your CPU may not have the bandwidth to read from your device and write it to the NIC. Would it be feasible to horizontal scale those nodes instead?

toast0 3y ago
Sometimes yes, but CDN nodes are competing for space in ISP edge hosting. Fitting an 800Gbps appliance into 2-4U means you can fit into a lot more ISPs. Additionally, if you use multiple hosts in a single deployment, you often also need a router, since the ISPs want to provide a single connection. That adds a significant cost to deployment in dollars and space. Additionally, I don't know why you would choose not to use the capacity of the system you have, just because you can easily scale horizontally. If a bit of engineering time can significantly reduce the number of systems to manage and the hardware budget, seems like a good plan. Especially when these projects are fun to work on and large tech companies want to retain employees.
- rewmie 3y ago
  > Fitting an 800Gbps appliance into 2-4U means you can fit into a lot more ISPs. The work and investment to turn that 4U deployment into two 2U deployments sounds like a far more realistic and effective way to go about the problem. > Additionally, I don't know why you would choose not to use the capacity of the system you have, just because you can easily scale horizontally. The point is that the capacity does not exist, hence these patches being shot down by kernel maintainers for being fundamentally broken. Also, what are the real-world performance improvements of these memory hacks? Adding another COTS box next to the one already deployed should double capacity straight away.
  - rcxdude 3y ago
    from what I can see the kernel developers don't like the particular means used in the patch, there's no fundamental objection to the idea of enabling device-to-device DMA as a performance optimisation.

by rewmie 3y ago

toast0 3y ago

Sometimes yes, but CDN nodes are competing for space in ISP edge hosting. Fitting an 800Gbps appliance into 2-4U means you can fit into a lot more ISPs. Additionally, if you use multiple hosts in a single deployment, you often also need a router, since the ISPs want to provide a single connection. That adds a significant cost to deployment in dollars and space. Additionally, I don't know why you would choose not to use the capacity of the system you have, just because you can easily scale horizontally. If a bit of engineering time can significantly reduce the number of systems to manage and the hardware budget, seems like a good plan. Especially when these projects are fun to work on and large tech companies want to retain employees.

rewmie 3y ago
> Fitting an 800Gbps appliance into 2-4U means you can fit into a lot more ISPs. The work and investment to turn that 4U deployment into two 2U deployments sounds like a far more realistic and effective way to go about the problem. > Additionally, I don't know why you would choose not to use the capacity of the system you have, just because you can easily scale horizontally. The point is that the capacity does not exist, hence these patches being shot down by kernel maintainers for being fundamentally broken. Also, what are the real-world performance improvements of these memory hacks? Adding another COTS box next to the one already deployed should double capacity straight away.
- rcxdude 3y ago
  from what I can see the kernel developers don't like the particular means used in the patch, there's no fundamental objection to the idea of enabling device-to-device DMA as a performance optimisation.