6 ms·
Any hint how much faster? Not looking for defensible benchmarks, but are you talking an order of magnitude? Multiple orders? Does it make the same probabilis
by feral 10y ago
Any hint how much faster? Not looking for defensible benchmarks, but are you talking an order of magnitude? Multiple orders?
Does it make the same probabilistic guarantees as the methods used in Stan etc? Or is it trading validity for speed?
- dragandj 10y agoNothing is universal and guaranteed, of course. YMMV, and all that. For example, robust linear regression from chapter 17, that fits 300 points over 4 parameters (easy, but far from trivial) runs in 180 seconds in JAGS and 485 in Stan, in parallel with 4 chains, taking 20,000 samples. Bayadera takes 276,297,912 samples in 300 milliseconds, giving much fine-grained estimations. So, depending on how you count the difference, it would be 500-1000 times faster for this particular analysis, while per-sample ratio is something like 7,000,000 (compared to JAGS). Of course, JAGS and Stan are mature software packages, while Bayadera is still pre-release...
- feral 10y agoThanks. About the second part of my question - are you doing much the same stuff as JAGS/Stan? Like, they do a lot of work to make sure that their MCMC is validly converging to the posterior - does Bayadera make similar guarantees? Is the speedup coming from a better implementation, or because GPUs are just way faster, or because it cuts statistical corners? If its cutting corners, are they sensible?
- dragandj 10y agoIt uses different MCMC algorithm - affine invariant ensemble MCMC. The difference comes from the fact that this algorithm is parallelizable, while JAGS/Stan's isn't. So, many GPU cores are the main factor. But, the algorithm is also a factor, in a sense that parallel chains always mutually inform each other. They may do a lot of work to make sure that MCMC is validly converging, and Bayadera also does its stuff on that front, but the truth is, and you'll find it in any book on MCMC (Gelman included) that you can never guarantee MCMC convergence.
- nextos 10y agoLooks very nice. I wonder if the upcoming Xeon Phi will make the task of parallel sampling simpler. Or at least compiling and optimising automatic parallel samplers on the fly. Macros might be great for this. That's the ultimate probabilistic programming goal. Write the model and get efficient sampling for free.
- dragandj 10y agoThanks. I doubt that XeonPhi would be any faster than my old AMD R9 290X, and the 10x price tag is also not inviting.
- eli_gottlieb 10y agoCan you point me to any good documentation on parallel MCMC algorithms and any info you might have written down on how you parallelized it? This sounds extremely worth porting over to some other probabilistic programming languages.
- dragandj 10y agoI'll be glad to send you the paper once it gets accepted.
- eli_gottlieb 10y agoThanks!
- patall 10y agoValidity probably depends on whether you use a professional GPU or gaming devices. I am currently running NMF on a GTX980 and from time to time the algorithm totally fails, probably due to missing EC. I hope our new Tesla server will solve the issue
- dragandj 10y agoI can not guarantee, but this should not be a problem for this particular algorithm (MCMC).