3 ms·
Haven't tried ADVI yet, but I think it is (likely) a great idea. In general automatic differentiation is a wildly powerful technique and is underutilized in com
by probdist 11y ago
Haven't tried ADVI yet, but I think it is (likely) a great idea. In general automatic differentiation is a wildly powerful technique and is underutilized in computational science. Even in pure optimization problems people rarely leverage it to get J or H information as often as they probably should. I'll admit to not being super familiar with all the machinery under the hood in the ADVI paper [1]. It wasn't out when I started working on my current project.
[1] http://arxiv.org/abs/1506.03431 http://arxiv.org/abs/1506.03431
- proditus 11y agodo let me know how it goes if you do try ADVI. we're also working on making it more robust to initialization and step-sizes. stay tuned.
- mjw 11y agoAVDI is incredibly awesome. Thanks so much for this work. Really nice readable paper on it too. As someone working with large datasets I think automating variational inference + SGD to work with a broad class of models is really the way forwards and a big force multiplier for machine learning.