With the way this optimizes it would produce crap, even if using the same seed frame to frame. One very interesting prospect is in taking the motion vectors out

With the way this optimizes it would produce crap, even if using the same seed frame to frame. One very interesting prospect is in taking the motion vectors out of the video and applying them to overlapping primitives in the shape data form as a first step to apply when fitting the shapes, that is to do this greedy hill climb in tandem with mpeg motion decoding. Maybe it wouldn't be too hard but it's not something I've got the motivation to try out and tweak right now