4 ms·
Embodied intelligence, Inductive bias, etc are the right words but we rarely know how to build them into our models in all but the simplest problems :-) Eg: Our
by ssivark 7y ago
Embodied intelligence, Inductive bias, etc are the right words but we rarely know how to build them into our models in all but the simplest problems :-) Eg: Our reinforcement learning models are largely stupid and without any great inductive biases. That’s why it takes a crazy amount of training and reward tweaking to even do anything simple.
> There are some really brave ideas being kicked around among the researchers who focus on their passion of AI, as opposed to those who post ever higher benchmark scores. They don't optimize their algorithms because they feel the algo should be serviceable out of the box. Cool stuff is happening.
Where do I find out more about this? :-)
- ganzuul 7y agoTensor networks come from quantum physics but are finding use in ML research. The papers on Neural Turing Machines, and Neural Tangent Kernels, which is related to Gaussian Processes, are getting a lot of citations. There is work on making these kernels composable. There are some papers combining some of these approaches but not yet all of them, that I have seen. If I have understood correctly: TN in the context of ML seems to be a pre-optimization step which lets you deal with inductive bias. The NTM learns to use its own memory through an attention mechanism. The NTK builds compressed representations which should be of great interest to anyone needing interpretable models. Gaussian Processes are a very different perspective on ML which lets you talk about composition of probability distributions. It seems to me that these components could be assembled into an ML algo capable of general intelligence (not 'strong AI' IMO) given a curriculum and enough compute. It wouldn't compete with the smartest humans but at least it should make for useful robots, because of the cost of training.