4 ms·

Xen vs. KVM: network, CPU, and disk performance compared

21 points by logic 16y ago

sophacles 16y ago
We use virtual machines to emulate larger networks, particularly when we need true independent actors at the edges. We chose KVM over Xen because it is easier to get an embedded system to run on emulation than it is to run on a hypervisor kernel. Also, some systems are just not able to be hacked at in that way. Our performance has been good, its nice to see how our intuition shakes out compared to actual measurements. Some notes I have on top of the article: Network performance monitoring: --iperf has a tendency to use a ton of cpu. This can impact network performance metrics. Lately I like using the 0mq performance tests, slightly modified to last longer than the default 10s. --In my setups KVM adds approx .5ms latency to network transit, when using para-virtualization through a tap device and bridge, having the host os do the routing. --The virtualized e1000 works better in most cases than any other NIC KVM will virtualize. CPU performance: --The best results for KVM come when setting cpu affinity, if you have a workload that coincides nicely to 1 vm/core, there is a performance boost.
doki_pen 16y ago
With the way the Xen kernel has lagged, this in very encouraging. For KVM, all that I have to do is enable it in my latest kernel sources.
Nwallins 16y ago
I found it odd and jarring that for the first 2 graphs, KVM is blue and Xen is green, while for the CPU graph, Xen is blue and KVM is green. For all subsequent 3-way comparisons, KVM is back to blue and Xen is back to green.
mgrouchy 16y ago
Those graphs are brutal. The graphs are 3d and on angles, so numbers that are the same actually look different. Just a complete mess. I know thats not what the post is about, but It makes the post considerably less useful and the graphs considerably less usable.
- logic 16y ago
  They're a bit more readable at full-size, but I have to agree, that angle is just annoying. But, better hard-to-read data than no data, I suppose. (I'm not responsible for the linked-to test, it was just mentioned on an internal mailing at $EMPLOYER.) For what it's worth, his results and conclusions correlate well with similar testing done here.
- tzs 16y ago
  Note also that it swaps the colors between Xen and KVM between the first group of graphs and the second group. Very poor presentation.
jz 16y ago
A bit off topic, but is anyone using BSD jails these days?
- jolan 16y ago
  Nope, but I use Linux-VServer which is basically jail for Linux. It works well with little performance penalty.
  - rufugee 16y ago
    How's your experience with vserver? I use openvz. It's ok...the technology works, but the docs are less than desirable. I'd switch to vserver if I could maintain the same experience.

21 points by logic 16y ago

We use virtual machines to emulate larger networks, particularly when we need true independent actors at the edges. We chose KVM over Xen because it is easier to get an embedded system to run on emulation than it is to run on a hypervisor kernel. Also, some systems are just not able to be hacked at in that way. Our performance has been good, its nice to see how our intuition shakes out compared to actual measurements. Some notes I have on top of the article: Network performance monitoring: --iperf has a tendency to use a ton of cpu. This can impact network performance metrics. Lately I like using the 0mq performance tests, slightly modified to last longer than the default 10s. --In my setups KVM adds approx .5ms latency to network transit, when using para-virtualization through a tap device and bridge, having the host os do the routing. --The virtualized e1000 works better in most cases than any other NIC KVM will virtualize. CPU performance: --The best results for KVM come when setting cpu affinity, if you have a workload that coincides nicely to 1 vm/core, there is a performance boost.

doki_pen 16y ago

With the way the Xen kernel has lagged, this in very encouraging. For KVM, all that I have to do is enable it in my latest kernel sources.

Nwallins 16y ago

I found it odd and jarring that for the first 2 graphs, KVM is blue and Xen is green, while for the CPU graph, Xen is blue and KVM is green. For all subsequent 3-way comparisons, KVM is back to blue and Xen is back to green.

mgrouchy 16y ago

Those graphs are brutal. The graphs are 3d and on angles, so numbers that are the same actually look different. Just a complete mess. I know thats not what the post is about, but It makes the post considerably less useful and the graphs considerably less usable.

logic 16y ago
They're a bit more readable at full-size, but I have to agree, that angle is just annoying. But, better hard-to-read data than no data, I suppose. (I'm not responsible for the linked-to test, it was just mentioned on an internal mailing at $EMPLOYER.) For what it's worth, his results and conclusions correlate well with similar testing done here.
tzs 16y ago
Note also that it swaps the colors between Xen and KVM between the first group of graphs and the second group. Very poor presentation.

jz 16y ago

A bit off topic, but is anyone using BSD jails these days?

jolan 16y ago
Nope, but I use Linux-VServer which is basically jail for Linux. It works well with little performance penalty.
- rufugee 16y ago
  How's your experience with vserver? I use openvz. It's ok...the technology works, but the docs are less than desirable. I'd switch to vserver if I could maintain the same experience.