35 ms·

I don’t know what PPS is, but that is an inaccurate sentiment. Running as a kernel module allows you to achieve higher throughout and lower latency.

lstamour 7y ago
PPS means Packets Per Second. Here’s a press release from the FD.io project (“Fido”), take it as you will: https://fd.io/latest/singles/kubernetes/ https://fd.io/latest/singles/kubernetes/
- dahfizz 7y ago
  FD.io uses the VPP stack under the hood, which ultimately uses DPDK[1] for the actual IO acceleration. The reason it's faster is because it's polling your hardware. You spend a lot of cpu time to buy the lower latency. [1] http://doc.dpdk.org/guides/prog_guide/overview.html http://doc.dpdk.org/guides/prog_guide/overview.html
  - lstamour 7y ago
    It's possible to write a driver that doesn't use polling - for example https://arxiv.org/pdf/1901.10664.pdf https://arxiv.org/pdf/1901.10664.pdf which discusses using vfio though doing so is apparently much harder than polling alone due to the complexity of the vfio interface and that most users care more about maximizing PPS performance at peak vs saving CPU cycles when less busy. Presumably one other option would be to try and tune the CPU to run slower when less demand is expected, and only run at faster peak performance when absolutely required. Also, it sounds like you might want to look at pinning cores to workloads, such that only one of your CPU cores is polling while the rest act as dynamic worker cores and could do other activities. Note that I'm largely speaking theoretically, for my use cases (mostly at home), the incompatibility of user space networking stacks often gets in the way.
    - dahfizz 7y ago
      > It's possible to write a driver that doesn't use polling Yup, definitely possible. It just doesn't give the PPS numbers as polling does. The project you linked to is really cool, but it is an academic project. It's not trying to compete with DPDK or anything. IO is one of those things in computer science that forces us to make trade ofs. What is more important to you? Raw throughput? Latency? Overall resource usage? You can't optimize all three. > it sounds like you might want to look at pinning cores to workloads, such that only one of your CPU cores is polling Correct again. It may take more than one cpu (DPDK is commercial software running on 88+ cpu systems) but cpu pinning is definitely the recommended way to go. In the spirit of spending CPU to buy IO speed, dedicating physical cores to the task always gives you the best bang for your buck.
lern_too_spel 7y ago
You're correct. I don't know why you're downvoted. Anything you can do to get performance with kernel bypass techniques can be done inside the kernel as well.
- ahmedalsudani 7y ago
  ¯\_(ツ)_/¯ It’s alright. You sometimes get downvoted by the early two people who are offended that I’d dare comment without knowing what PPS is (or something equally offensive).

lstamour 7y ago

PPS means Packets Per Second. Here’s a press release from the FD.io project (“Fido”), take it as you will: https://fd.io/latest/singles/kubernetes/ https://fd.io/latest/singles/kubernetes/

dahfizz 7y ago
FD.io uses the VPP stack under the hood, which ultimately uses DPDK[1] for the actual IO acceleration. The reason it's faster is because it's polling your hardware. You spend a lot of cpu time to buy the lower latency. [1] http://doc.dpdk.org/guides/prog_guide/overview.html http://doc.dpdk.org/guides/prog_guide/overview.html
- lstamour 7y ago
  It's possible to write a driver that doesn't use polling - for example https://arxiv.org/pdf/1901.10664.pdf https://arxiv.org/pdf/1901.10664.pdf which discusses using vfio though doing so is apparently much harder than polling alone due to the complexity of the vfio interface and that most users care more about maximizing PPS performance at peak vs saving CPU cycles when less busy. Presumably one other option would be to try and tune the CPU to run slower when less demand is expected, and only run at faster peak performance when absolutely required. Also, it sounds like you might want to look at pinning cores to workloads, such that only one of your CPU cores is polling while the rest act as dynamic worker cores and could do other activities. Note that I'm largely speaking theoretically, for my use cases (mostly at home), the incompatibility of user space networking stacks often gets in the way.
  - dahfizz 7y ago
    > It's possible to write a driver that doesn't use polling Yup, definitely possible. It just doesn't give the PPS numbers as polling does. The project you linked to is really cool, but it is an academic project. It's not trying to compete with DPDK or anything. IO is one of those things in computer science that forces us to make trade ofs. What is more important to you? Raw throughput? Latency? Overall resource usage? You can't optimize all three. > it sounds like you might want to look at pinning cores to workloads, such that only one of your CPU cores is polling Correct again. It may take more than one cpu (DPDK is commercial software running on 88+ cpu systems) but cpu pinning is definitely the recommended way to go. In the spirit of spending CPU to buy IO speed, dedicating physical cores to the task always gives you the best bang for your buck.

lern_too_spel 7y ago

You're correct. I don't know why you're downvoted. Anything you can do to get performance with kernel bypass techniques can be done inside the kernel as well.

ahmedalsudani 7y ago
¯\_(ツ)_/¯ It’s alright. You sometimes get downvoted by the early two people who are offended that I’d dare comment without knowing what PPS is (or something equally offensive).