5 ms·
In other words, the RL system needs to acquire high order concepts (such as "climb the ladder" or "run from the bad guy"), and equipped with those it fares much
by visarga 10y ago
In other words, the RL system needs to acquire high order concepts (such as "climb the ladder" or "run from the bad guy"), and equipped with those it fares much better. The problem with learning abstract concepts is that they are combinations of combinations of low level features and it is exponentially hard to search for them in raw data. Humans inputting such concepts as "command phrase + reward in case the agent correctly executes the command" cut through the search space and help it learn to operate over the new concepts it learned.
Imagine how would you classify even a simple concept, such as "riding" - a human could be riding a horse, or a monkey could be riding an elephant, or there could be tons of other cases. Riding would be much simpler to detect if there is a classifier that selects "objects used for rides" and "agents that can ride" and the spatial relation between them - that would be a high order concept that we can't simply learn from images, we have to have preliminary abstractions that help in classifying it.
I think this is the future in AI - higher order concepts, based on compositions of previously known concepts. Both language and the physical world are made of objects and relations between objects, so it would be necessary to learn to combine concepts into new concepts, even when training data is very small. It would solve the problem of sharing knowledge between tasks. Another benefit would be the ease of inspecting the internal state of the system - which would be a graph of language based concepts, unlike the internal states of neural nets which are inscrutable. An agent that has higher order abstractions and an object-relations graph would also be programmable in plain language and capable of reasoning over facts - that would make AI accessible to the public at large.
Another way of putting it is that up until now, we used plain vectors, as if it was untyped data, but now we need to operate over strongly typed vectors with higher order operators. We need type theory into neural nets, to apply type constraints and to convert from one type to another, by applying operations. Such operations are hard to learn directly from labeled sets of images.
- nojvek 10y agoYou make a great point. Rather than neural nets operating on vectors they need to operate on typed graph structure and compute recursive functions with memory stacks. Kind of like how computers work but learn the code and reason about data structures automatically
- Houshalter 10y agoYou can't differentiate through discrete data structures though. The main reason NNs work at all is because vectors are continuous. Small changes in the input lead to small changes in the output. Then you can work backwards and find out exactly what parameters to tweak.
- sjg007 10y agoYou'd probably want to start with propositional logic. But I think the vectors would still be numbers but they would encode some higher order task e.g encode another neural network or store its output in memory that could be accesssed on demand. So you would have a NN that would be climb ladder. Basically a collection of these hybrid memory neural networks.