Yes, but here's the thing: the virtual memory system already does this kind of "magic", maintaining the mapping of virtual addresses onto the physical ones, and

Yes, but here's the thing: the virtual memory system already does this kind of "magic", maintaining the mapping of virtual addresses onto the physical ones, and it doesn't need to actually move data between physical pages at all since the mapping is discontinuous.