This paper is about training the diffusion models on progressively higher resolutions, spending comparatively more iterations on smaller resolutions (which use

This paper is about training the diffusion models on progressively higher resolutions, spending comparatively more iterations on smaller resolutions (which use much less time and memory per iteration). Their method is flexible enough for any multidimensional problem which is amenable to similar scaling (the use the example of generating video, which includes the additional dimension of time). The models converge much more quickly during training by using this multi-resolution paradigm. There's a figure (4) in their paper showing how quickly these models converge to a good accuracy on a standard training set at 256x256 resolution, comparing them in particular to a Latent Diffusion implementation. They showed that their baseline models were slightly better than Latent diffusion in terms of training efficiency. However, when they pretrained their models on 64x64, they converged much much faster. The Apple models converged at around 60k iterations to a better score than where Latent Diffusion arrived at in around 300k iterations. The caveat is that for my example above, they pretrained the model for 390k iterations on 64x64 before setting it to train at 256x256, which is indeed more than Latent diffusion got the opportunity to train for - however, those 390k iterations were using something proportional to 16 times less resources. This would be a very good paper for our friends creating Stable Diffusion to copy when creating the next versions, it would likely save them a lot of GPU time and I bet it would improve robustness for different dimension output.