3 ms·
This paper lists the benefits in the introduction: https://proceedings.neurips.cc/paper_files/paper/2019/file/378a063b8fdb1db941e34f4bde584c7d-Paper.pdf https:/
by jiggawatts 2y ago
This paper lists the benefits in the introduction: https://proceedings.neurips.cc/paper_files/paper/2019/file/378a063b8fdb1db941e34f4bde584c7d-Paper.pdf https://proceedings.neurips.cc/paper_files/paper/2019/file/3...
- Simplicity and Stability: An EBM is the only object that needs to be trained and designed. Separate networks are not tuned to ensure balance.
- Sharing of Statistical Strength: Since the EBM is the only trained object, it requires fewer model parameters than approaches that use multiple networks.
- Adaptive Computation Time: Implicit sample generation is an iterative stochastic optimization process, which allows for a trade-off between generation quality and computation time.
- VAEs and flow-based models are bound by the manifold structure
of the prior distribution and consequently have issues modelling discontinuous data manifolds, often assigning probability mass to areas unwarranted by the data. EBMs avoid this issue by directly modelling particular regions as high or lower energy.
- Compositionality: If we think of energy functions as costs for a certain goals or constraints, summation of two or more energies corresponds to satisfying all their goals or constraints.
- programjames 2y agoAs far as I can tell, flow-based models are bound by the exact same requirements as energy based models (flow = diffusion/normalizing flow/flow-matching models). But they're absolutely right about VAEs. Those are a memetic virus that need to die off in favor of more theoretically grounded encoders.