4 ms·

your reasoning is correct (the path latency gets higer) its just that you got the final numbers wrong ;-) on modern software forwarding core (fd.io/VPP and DPD

your reasoning is correct (the path latency gets higer) its just that you got the final numbers wrong ;-) on modern software forwarding core (fd.io/VPP and DPDK) you can forward with a sub 100us latency. so your total latency ends up being roughly 1.7ms "slower". have a look at this: https://www.youtube.com/watch?v=T66BTHnENY8 https://www.youtube.com/watch?v=T66BTHnENY8

hueving 10y ago
That video is pretty light on details. I would be interested in seeing how much the routes were summarized and if it was enough that they all fit into the processor's cache. The thing that kills me with these network performance benchmark videos is that there are so many things that drastically impact performance and you never get any details about them. Just the marketing pitch.
Hikikomori 10y ago
Performance within one numa space seems to be great. What can you get between numa spaces? That would be a common forwarding path for any router with enough interfaces.
- feld 10y ago
  If you want good results you don't build a server with more than one CPU socket. NUMA is too expensive for high performance networking.
  - Hikikomori 10y ago
    So you're basically stuck with the number of cores of a single socket per 1U, seems like a waste of space and possibly power usage when 2 cores are utilized per 10GE port (in this video). Doesn't seem to scale well now that 100GE is getting more common, so 2 100GE ports per 1U? Depending on where this device would be used the recent 32port 100Ge 1U switches seems to be more interesting, small fib, but with the correct protocol support it could fit a lot of use cases, especially with something like sir[1]. 1. https://github.com/dbarrosop/sir https://github.com/dbarrosop/sir
ra1n85 10y ago
Except that DPDK has a hard ceiling in terms of total PPS.
Cerium 10y ago
I've done some work with DPDK about a year ago. At that time I was getting about 12us latency for small packets. Interestingly the power consumption is maximum at minimum load. DPDK uses poll mode drivers to minimize latency, when load is light it uses more power constantly polling than the cpu uses doing computations.

hueving 10y ago

That video is pretty light on details. I would be interested in seeing how much the routes were summarized and if it was enough that they all fit into the processor's cache. The thing that kills me with these network performance benchmark videos is that there are so many things that drastically impact performance and you never get any details about them. Just the marketing pitch.

Hikikomori 10y ago

Performance within one numa space seems to be great. What can you get between numa spaces? That would be a common forwarding path for any router with enough interfaces.

feld 10y ago
If you want good results you don't build a server with more than one CPU socket. NUMA is too expensive for high performance networking.
- Hikikomori 10y ago
  So you're basically stuck with the number of cores of a single socket per 1U, seems like a waste of space and possibly power usage when 2 cores are utilized per 10GE port (in this video). Doesn't seem to scale well now that 100GE is getting more common, so 2 100GE ports per 1U? Depending on where this device would be used the recent 32port 100Ge 1U switches seems to be more interesting, small fib, but with the correct protocol support it could fit a lot of use cases, especially with something like sir[1]. 1. https://github.com/dbarrosop/sir https://github.com/dbarrosop/sir

ra1n85 10y ago

Except that DPDK has a hard ceiling in terms of total PPS.

Cerium 10y ago

I've done some work with DPDK about a year ago. At that time I was getting about 12us latency for small packets. Interestingly the power consumption is maximum at minimum load. DPDK uses poll mode drivers to minimize latency, when load is light it uses more power constantly polling than the cpu uses doing computations.