3 ms·
How Diffusion Models Can Achieve Seemingly Arbitrarily Large Compression Ratios
- 0xdead1eaf 4y agoPertaining to recent discussions around AI art ethics and the ability for image diffusion models to "copy" training samples, this post provides intuition for how generative models such as diffusion models compress information from a dataset. It provides intuition showing that the amount of information that is compressed during learning is not determined by the dataset’s memory footprint, but by the nature and complexity of the underlying distribution that generated the data in the first place.
- eternalban 4y agoA simple intuitive model is to assume functional black box f() is a polynomial of degree N, where N is the number of parameters.