5 ms·
What is wrong with variational autoencoders?
- stochastic_monk 9y agoThe equations on this page don't render for me on either Firefox or Chrome, making reading this a little jarring. See their paper [0] for what I think is easier to follow. *A more informative title would be something like "Improving Variational Autoencoders with improved Evidence-Based Lower Bound [ELBO] methods. The "What's wrong with VAEs?" title, followed by paragraphs explaining how VAEs work, made me wonder why I was reading it in the first place, but practically improving ELBO methods for VAE implementations is a meaningful contribution. [0] "Tighter Variational Bounds are Not Necessarily Better", https://arxiv.org/abs/1802.04537 https://arxiv.org/abs/1802.04537
- jessaustin 9y agoWorks fine for me on Chrome, but gradually. Do you have javascript turned off?
- stochastic_monk 9y agoI was just able to get them to display in Chrome (not Firefox) by finding an option for loading 'unsafe scripts'. It wasn't an option on Firefox, even after disabling several security extensions I use. [Ghostery, privacy badger, adblock.]
- mijoharas 9y agoIt's Https everywhere, which forces you onto HTTPS, the site has non-https links which your browser doesn't load. Try turning off https everywhere for the site, and visiting it via http (then turn it back on afterwards :) ).
- gpm 9y agoWorks fine on Firefox too, both stable and nightly. I think it must be a local problem (noscript comes to mind, or an overly aggressive tracker blocker).
- akosiorek 9y agoAuthor of the post here; I'm not sure what's preventing the equations from showing. The blog are just a bunch of static sites generated by Jekyll, math is from mathjax.
- stochastic_monk 9y agomijoharas below correctly identified it as being the HTTPS Everywhere browser add-on I use which prevented them from showing. I don't know enough about browser/website design to be able to help fix the problem. Thank you for your work, the post, and the response!
- visarga 9y agoI thought it was that VAEs generate blurry images. Has anyone been able to make VAEs as good as GANs for image generation?
- twanvl 9y agoIIRC there was a NIPS 2016 paper that showed that the blurry images are because of the variational approximation used in the generator/decoder network. They replaced this by doing MCMC inference on the encoder network, and showed that you get sharper images, but much less efficiently. In general, variational inference will always tend to model the mean or mode of the data.
- trextrex 9y agoThere was one theoretical paper to enable learning more complex distributions in the encoding layer(s) while still using a factorized gaussian or similarly simple distribution to derive the learning rules [1]. This approach should, in theory, lead to generation of sharper images, although that particular paper doesn't show examples of it. [1] https://arxiv.org/abs/1505.05770 https://arxiv.org/abs/1505.05770
- jostmey 9y agoWhere are the results? At least show me the MNIST samples either in the blog-post or the paper. And while your at it, show the closet datapoint to the sample. Is the model just memorizing the datapoints in the dataset?
- akosiorek 9y agoAuthor of the blogpost here; the main results are the convergence curves of IWAE-64 and log p(x) in the paper, but thats a good point, thanks. We'll include samples in the camera-ready version of the paper.