4 ms·

> "user-mode network stack" Kernels typically cannot use vector instructions because if they did they would need to save and restore the vector register state

> "user-mode network stack" Kernels typically cannot use vector instructions because if they did they would need to save and restore the vector register state when servicing interrupts. There is a very large performance cost to doing that. Moving packet processing into userspace means adding latency, including TLB pressure, in order to do the context switch. I imagine that we might get some innovation by allowing to configure the system such that the kernel owns the vector registers and userspace is not allowed to use them. If your primary interest in vector registers/instructions is packet processing, and you're doing that in kernelspace, you might not mind it if userspace can't use those registers.

wmf 3y ago
Moving packet processing into userspace means adding latency, including TLB pressure, in order to do the context switch. This isn't the case because VPP polls the NIC from userspace and never enters the kernel. There are no context switches.
- vacuity 3y ago
  Is there something like an IOMMU to provide secure access to the NIC?
  - ADSSDA 3y ago
    Yes, DPDK (which VPP is built on) heavily utilizes IOMMU to provide host protection. https://doc.dpdk.org/guides/linux_gsg/linux_drivers.html https://doc.dpdk.org/guides/linux_gsg/linux_drivers.html discusses it a bit.
crest 3y ago
The vector in vector packet processing has little to nothing to do with vector instruction sets (SSE/AVX, VMX, RVV, etc.). Only the very latest CPUs (and historical supercomputers) have scatter/gather instructions capable of efficiently extracting packet header fields from multiple packets in parallel and if you have a large enough batch of packets process switching the vector register file is worth it to the kernel. It's just that most kernels don't spill the userspace vector registers on every context switch because it's more common to switch back to the same thread (or an other userspace thread) than using the vector registers inside the kernel. Both Linux and *BSD can and do make limited use of vector registers inside the kernel e.g. for fast encryption/decryption because it's worth the start up cost. If I understood the VPP design and implementation details correctly they try to reduce the amortised cache misses for a batch of packets by running all packets in a batch through each software pipeline stage before processing the next stage. This should result in very good average instruction cache hit rates and should also help with data cache hit rates because packet headers are small and can be prefetched while the forwarding data structures e.g. 1 million IPv4 prefixes and their next hops can be hard to fit into L2 data caches and won't fit into L1 data caches. I assume a carefully tuned implementation can make further gains by dedicating cores to specific pipeline stages to keep the data caches hotter at the cost of copying processed packet headers their next stage in new (sub-)batches. The actual packet content is only relevant for a few operations like encryption/decryption and modern highend NICs have line rate crypto engines to help with IPsec or TLS.
- benou 3y ago
  Batching packets bring several benefits: - amortizing cache misses are you mentioned - better use of out-of-order, superscalar processors: by processing multiple independent packets in parallel, the processor can fill more execution units - enable the use of vector instructions (SSE/AVX, VMX etc): again, processing multiple independent packets in parallel means you can leverage SIMD. SIMD instructions are used pervasively in VPP
sweetjuly 3y ago
> Kernels typically cannot use vector instructions because if they did they would need to save and restore the vector register state when servicing interrupts. There is a very large performance cost to doing that. While this is true, it definitely doesn't seem to stop people like Apple [1] from using SIMD extensions in the kernel anyways. On ARMv8, it's an extra 512 bytes (32 qword registers) or eight (and change, depending on alignment) dirty cache lines. Whether or not this causes a serious performance impact will depend on how well the kernel can actually make use of SIMD (saving and restoring might cost 1% perf but if speeding up the kernel wins 5%, who cares!). It could be interesting to play with this on other kernels to verify these assumptions, it might be worth enabling in the kernel on some devices! Kernels do tend to do a lot of moving data around, which these extensions (on ARM, anyways) excel at. [1] https://github.com/apple-oss-distributions/xnu/blob/main/osfmk/arm64/exception_asm.h#L162-L177 https://github.com/apple-oss-distributions/xnu/blob/main/osf...

wmf 3y ago

Moving packet processing into userspace means adding latency, including TLB pressure, in order to do the context switch. This isn't the case because VPP polls the NIC from userspace and never enters the kernel. There are no context switches.

vacuity 3y ago
Is there something like an IOMMU to provide secure access to the NIC?
- ADSSDA 3y ago
  Yes, DPDK (which VPP is built on) heavily utilizes IOMMU to provide host protection. https://doc.dpdk.org/guides/linux_gsg/linux_drivers.html https://doc.dpdk.org/guides/linux_gsg/linux_drivers.html discusses it a bit.

crest 3y ago

The vector in vector packet processing has little to nothing to do with vector instruction sets (SSE/AVX, VMX, RVV, etc.). Only the very latest CPUs (and historical supercomputers) have scatter/gather instructions capable of efficiently extracting packet header fields from multiple packets in parallel and if you have a large enough batch of packets process switching the vector register file is worth it to the kernel. It's just that most kernels don't spill the userspace vector registers on every context switch because it's more common to switch back to the same thread (or an other userspace thread) than using the vector registers inside the kernel. Both Linux and *BSD can and do make limited use of vector registers inside the kernel e.g. for fast encryption/decryption because it's worth the start up cost. If I understood the VPP design and implementation details correctly they try to reduce the amortised cache misses for a batch of packets by running all packets in a batch through each software pipeline stage before processing the next stage. This should result in very good average instruction cache hit rates and should also help with data cache hit rates because packet headers are small and can be prefetched while the forwarding data structures e.g. 1 million IPv4 prefixes and their next hops can be hard to fit into L2 data caches and won't fit into L1 data caches. I assume a carefully tuned implementation can make further gains by dedicating cores to specific pipeline stages to keep the data caches hotter at the cost of copying processed packet headers their next stage in new (sub-)batches. The actual packet content is only relevant for a few operations like encryption/decryption and modern highend NICs have line rate crypto engines to help with IPsec or TLS.

benou 3y ago
Batching packets bring several benefits: - amortizing cache misses are you mentioned - better use of out-of-order, superscalar processors: by processing multiple independent packets in parallel, the processor can fill more execution units - enable the use of vector instructions (SSE/AVX, VMX etc): again, processing multiple independent packets in parallel means you can leverage SIMD. SIMD instructions are used pervasively in VPP

sweetjuly 3y ago

> Kernels typically cannot use vector instructions because if they did they would need to save and restore the vector register state when servicing interrupts. There is a very large performance cost to doing that. While this is true, it definitely doesn't seem to stop people like Apple [1] from using SIMD extensions in the kernel anyways. On ARMv8, it's an extra 512 bytes (32 qword registers) or eight (and change, depending on alignment) dirty cache lines. Whether or not this causes a serious performance impact will depend on how well the kernel can actually make use of SIMD (saving and restoring might cost 1% perf but if speeding up the kernel wins 5%, who cares!). It could be interesting to play with this on other kernels to verify these assumptions, it might be worth enabling in the kernel on some devices! Kernels do tend to do a lot of moving data around, which these extensions (on ARM, anyways) excel at. [1] https://github.com/apple-oss-distributions/xnu/blob/main/osfmk/arm64/exception_asm.h#L162-L177 https://github.com/apple-oss-distributions/xnu/blob/main/osf...