5 ms·
The notion of running one giant model that has many sub-talents is epic. I can imagine that all the disparate models they run today could fuse into a giant netw
by iandanforth 9y ago
The notion of running one giant model that has many sub-talents is epic. I can imagine that all the disparate models they run today could fuse into a giant network that melds predictions and guides computation as required by the task. That seems like a very Jeff Dean scale endeavor.
- cosminro 9y agoHe is referring to this paper https://research.googleblog.com/2017/06/multimodel-multi-task-machine-learning.html https://research.googleblog.com/2017/06/multimodel-multi-tas... Where they used one model to do image recognition, speech recognition and translation in the same network.
- deleted 9y ago[deleted]
- shpx 9y agoHe also mentions sparsely activating only the neurons that matter, which they explore in https://arxiv.org/pdf/1701.06538.pdf https://arxiv.org/pdf/1701.06538.pdf Personally I didn't find it very satisfying, I imagine something more fundamental and self referential.
- soVeryTired 9y agoSounds like a joy to debug
- erikpukinskis 9y agoIt's a great idea, and human beings are proof that conjoined colonies of intelligent agents are going to be smarter than individuals. But it also runs up against tradeoffs, as evidenced by the fact that many humans exist, and we didn't evolve into just one giant ur-human, even after we all got our internet implant. Sometimes a few agents with different strategies working independently is better than a single agent choosing amongst the strategies of several others.