4 ms·

Well I am talking about true live migration where not a single icmp echo request is lost.

viraptor 13y ago
I still don't know which hypervisor are you using, but if it's anything libvirt-based, I don't believe such seamless migration is possible. Simply moving the networking information around is taking it's time. You're going to be down for at least the time it takes for example qemu to move the last bit of dirty memory over to the new host. Sending MAC updates to the routers will also take some time. Since ICMP is not retried, it's going to be lost.
- mgalkiewicz 13y ago
  I am using libvirt with KVM. I have successfully migrated many instances without losing any icmp. I understand that memory must be transferred. It just works fast in my environment. MAC address does not change if I remember correctly.
  - viraptor 13y ago
    You simply avoided the race condition. It won't happen every time and it doesn't mean that the migration would never lose any icmp packet. It's impossible without organising the whole network around this idea. MAC doesn't have to change, but the upstream router still needs to be notified about the new location of your instance. That's why nova sends the gratuitous arp packet (more than one actually since they are not guaranteed). At least that's the case for the old networking model. Not sure what's needed for quantum updates, but that will require at least one RabbitMQ message on Nova's side and an HTTP request to Quantum. Another way to explain the problem: you can write to local memory faster than you can move those changes to another host over the network. Any application heavily processing the data and using all available memory will make memory pages dirty faster than it takes qemu to move them across to the next host. The result is that your instance has to be paused until the copying process completes (unless you're running on a non-standard architecture). Otherwise the copy would never complete.
notacoward 13y ago
What technology short of hardware FT (which I've worked on BTW) will prevent that? You'll lose plenty of ICMP echo requests before you even realize a migration is necessary, and no storage infrastructure is going to help with that. Why single out one when it's a problem common to all?

I still don't know which hypervisor are you using, but if it's anything libvirt-based, I don't believe such seamless migration is possible. Simply moving the networking information around is taking it's time. You're going to be down for at least the time it takes for example qemu to move the last bit of dirty memory over to the new host. Sending MAC updates to the routers will also take some time. Since ICMP is not retried, it's going to be lost.

mgalkiewicz 13y ago
I am using libvirt with KVM. I have successfully migrated many instances without losing any icmp. I understand that memory must be transferred. It just works fast in my environment. MAC address does not change if I remember correctly.
- viraptor 13y ago
  You simply avoided the race condition. It won't happen every time and it doesn't mean that the migration would never lose any icmp packet. It's impossible without organising the whole network around this idea. MAC doesn't have to change, but the upstream router still needs to be notified about the new location of your instance. That's why nova sends the gratuitous arp packet (more than one actually since they are not guaranteed). At least that's the case for the old networking model. Not sure what's needed for quantum updates, but that will require at least one RabbitMQ message on Nova's side and an HTTP request to Quantum. Another way to explain the problem: you can write to local memory faster than you can move those changes to another host over the network. Any application heavily processing the data and using all available memory will make memory pages dirty faster than it takes qemu to move them across to the next host. The result is that your instance has to be paused until the copying process completes (unless you're running on a non-standard architecture). Otherwise the copy would never complete.

notacoward 13y ago

What technology short of hardware FT (which I've worked on BTW) will prevent that? You'll lose plenty of ICMP echo requests before you even realize a migration is necessary, and no storage infrastructure is going to help with that. Why single out one when it's a problem common to all?