An interesting aspect is the sampling from a different distribution on testing versus training. The "truncation trick" samples from a truncated Normal distribu

An interesting aspect is the sampling from a different distribution on testing versus training. The "truncation trick" samples from a truncated Normal distribution rather than N(0,sigma) for the latent variables. (The values above a particular threshold are just sampled again until they are below that threshold.) I don't completely get this. What's going on there? Is there a mode in the network that defines the prototypical dog and are other dogs defined by nonzero values in the latent variable space? Then this seem to show that the layer exhibits an non-intuitive decomposition of the task at hand. I would expect a zero vector to correspond to an "abstract dog" and have all nonzero parameters contribute in an attribute like fashion. This seems to be more prototype-like, similar to old-fashioned vector quantization. The censored normal max[N(0,sigma),0] is interesting. It reminds me of nonnegative matrix factorization. Check the paper in Nature or just a nice blog post: https://yliapis.github.io/Non-Negative-Matrix-Factorization/ https://yliapis.github.io/Non-Negative-Matrix-Factorization/. By using a nonnegative constraint the representation becomes additive (part-based) and sparse. That's quite different from prototype-like methods. I'm myself experimenting with more complex priors and in my experience it's difficult to find priors that blow everything out of the water. Very nice appendix E. :-)