3 ms·
This is terrible and frankly self serving advice. "Don't build on top of deep learning. Build on top of MCMC-like methods" I used to do research into such met
by light_hue_1 3y ago
This is terrible and frankly self serving advice.
"Don't build on top of deep learning. Build on top of MCMC-like methods"
I used to do research into such methods. That game is over. It's a massive waste of time at the moment. The whole idea is how do we import what was good about those methods into the modern deep learning toolkit. How do I sample from distribution with dl? How do I get uncertainty estimates? How do I compose models, get disentangled representations, get few shot learning, etc.
The idea that people should go back to tools like MCMC today is pretty absurd. That entire research program was a failure and never scaled to anything. I say this of my many dozens of papers in the area too.
I would never give this advice to my PhD students.
Maybe in a decade or two someone will rescue MCMC like methods. In the meantime your PhD students will suffer by being irrelevant and having skills that no one needs.
- mrcode007 3y agoYou dismissed most of the modern computational finance and a lot of cross valuation adjustments mechanisms that are behind the modern financial system and risk estimates. Edit: not to mention modern statistical mechanics…
- tnecniv 3y agoWhich then cycles back to ML methods. Diffusion models are non-equilibrium stat mech. Sure building a better HMC algorithm might not be a good use of time, but the spirit of MCMC is alive
- theGnuMe 3y agoYep. Capture the distribution you want to sample from by training a DNN on the data and then sample it via generative/diffusion. Basically the DNN is a learnt parametrized distribution and replaces the Markov chain. The adversarial training approach seems to me like a forward backward algorithm over the dataset anyway.
- 3abiton 3y agoBut does this validate OP's claim?