7 ms·
Hello! Can you comment on how this relates to: https://arxiv.org/pdf/1804.00779.pdf https://arxiv.org/pdf/1804.00779.pdf
by diego898 8y ago
Hello! Can you comment on how this relates to: https://arxiv.org/pdf/1804.00779.pdf https://arxiv.org/pdf/1804.00779.pdf
- duvenaud 8y agoGood question. The model developed in that paper, (neural autoregressive flows) is a discrete-layered architecture. It's a member of the Normalizing Flows family of models. Normalizing flows define a parametric density by transforming a sample from a Gaussian by a series of transformations: z0 ~ Normal(0, I) z1 = f1(z0) z2 = f2(z1) z3 = f3(z2) x = f4(z3) and they use the change of variables formula to compute p(x): log p(x) = log p(z0) + log det | df/dz0 | In our paper, we propose a continuous-time version of normalizing flows, called Continuous Normalizing Flows. We derived a continuous-time version of the change of variables formula: dlogp(z(t)) = -trace(df/dz) Anyways, the confusion is probably that both models are called flows. We wish that Normalizing Flows had used a different name, so that we could save the word "flow" for continuous-time transformations, but it's too late for that :) Neural autoregressive flows are powerful density models, but are computationally costly to sample from. Continuous normalizing flows cost about the same to evaluate densities and to sample. We compared the two in a follow-up paper: [https://arxiv.org/abs/1810.01367 https://arxiv.org/abs/1810.01367]