2 ms·

To me this is where the "software defined networking" type of virtualization can really make an impact. The network performance of a known cluster of virtualiz

by codemac 13y ago

To me this is where the "software defined networking" type of virtualization can really make an impact. The network performance of a known cluster of virtualized instances could be extremely quick if you just lie, and say your packet went through a network, when really you just pass a pointer in the hypervisor.. I assume this has already been done, but at almost 200 microseconds, you know it hasn't been done in these experiments.

montecarl 13y ago
That is only the case for networking inside of a single physical machine. Most HPC MPI use cases span many machines, for which this is typical latency of ethernet.
- codemac 13y ago
  In cases where I've used OpenMPI it was spanning machines as well as being multiple processes on the same box. The goal was making that interchangeable (in my use case). I imagine google's compute engine isn't all on separate machines, but utilizes VMs heavily.. although it's probably a bad idea to put one customer's VM's on all the same box. That's a long way of saying "of course you're right, I guess my thought doesn't contribute as much as I thought" :)
  - montecarl 13y ago
    You are also right of course. Intra-machine latency is quite important. Many problems can be decomposed in to smaller parallel parts that can be done per machine.
ori_b 13y ago
That's an interesting hack for MPI, but I don't think it would be worth the complexity of implementing it. There are two possible use cases here: One: You're testing multiple VMs on one box to simulate a network. This is a sane use case, but a bit of overhead from network latency isn't going to kill you. It definitely isn't worth the effort of messing around with faking your network. Two: You're trying to wring whatever performance you can out of a single box. In this case, you can ditch the VMs, and just run your MPI nodes as threads or processes on the local machine, using far faster and more efficient transports than TCP. Also, in this specific case, note that the instances are not necessarily on the same machine.

by codemac 13y ago

montecarl 13y ago

That is only the case for networking inside of a single physical machine. Most HPC MPI use cases span many machines, for which this is typical latency of ethernet.

codemac 13y ago
In cases where I've used OpenMPI it was spanning machines as well as being multiple processes on the same box. The goal was making that interchangeable (in my use case). I imagine google's compute engine isn't all on separate machines, but utilizes VMs heavily.. although it's probably a bad idea to put one customer's VM's on all the same box. That's a long way of saying "of course you're right, I guess my thought doesn't contribute as much as I thought" :)
- montecarl 13y ago
  You are also right of course. Intra-machine latency is quite important. Many problems can be decomposed in to smaller parallel parts that can be done per machine.

ori_b 13y ago

That's an interesting hack for MPI, but I don't think it would be worth the complexity of implementing it. There are two possible use cases here: One: You're testing multiple VMs on one box to simulate a network. This is a sane use case, but a bit of overhead from network latency isn't going to kill you. It definitely isn't worth the effort of messing around with faking your network. Two: You're trying to wring whatever performance you can out of a single box. In this case, you can ditch the VMs, and just run your MPI nodes as threads or processes on the local machine, using far faster and more efficient transports than TCP. Also, in this specific case, note that the instances are not necessarily on the same machine.