Reading the paper, what's interesting is that they really build upon text-to-image models. Instead of generating 1 image with diffusion model, they generate 16,

Reading the paper, what's interesting is that they really build upon text-to-image models. Instead of generating 1 image with diffusion model, they generate 16, and then use several upsampling/interpolation methods to increase the resolution/framerate. For the text-image model, they use the DALL-E 2 architecture, so in theory just switching to a latent diffusion model (like stable diffusion) would boost training/inference cost of the model The shortcoming is that, since they don't use any video/text annotation (just image/text pairs), complex temporal prompts would probably not work, something which require multiple steps like "a bear scoring a goal, then doing the Ronaldo celebration"