It's not really about difficulty (although encoding is definitely far more difficult), it's about there being multiple valid encodings. If you have two red box

It's not really about difficulty (although encoding is definitely far more difficult), it's about there being multiple valid encodings. If you have two red boxes on a black background in one frame, and a single red box in the middle on the next, UIUI there are at least two ways to encode this: "left box moved right a bit; right box disappears" and "right box mounts left a bit; let box disappears". In a complex scene, there is a huge space of possible ways to encode a frame that give identical outputs, and even more that are lossy. Choosing one that compresses well but is visually similar (and not too slow to find) is a quality issue for the codec. It wouldn't make sense to specify the algorithm for that in the spec.