The insight that made it possible for me to grasp VAEs was digging into the probabilistic setup that leads to this formulation. The neural networks are "just" a

The insight that made it possible for me to grasp VAEs was digging into the probabilistic setup that leads to this formulation. The neural networks are "just" a powerful function approximators applied on top of this probabilistic framework.