3 ms·
> To think that we can simply abandon symbol-manipulation is to suspend disbelief. Looks like another one of the symbolic folks is mad that they aren't cited a
by ShamelessC 5y ago
> To think that we can simply abandon symbol-manipulation is to suspend disbelief.
Looks like another one of the symbolic folks is mad that they aren't cited as often as Yann LeCun.
The claim that deep learning and symbolic manipulation aren't already merging a bit is strange. Grounding is an area of active research.
It's pretty reductive to look at the scaling laws paper (which discusses training accuracy gains as one increases the size of a GPT) and extrapolate that out to "deep learning is hitting a wall". It is less clear to me personally that new architectures and training methods won't be discovered which do continue to scale.
- 9wzYQbTYsAIc 5y ago> It is less clear to me personally that new architectures and training methods won't be discovered which do continue to scale. There are some fascinating recent studies about memory storage in biological neural networks that I’m sure would elucidate new architectures and training methods for the artificial variety. Multi-phasic layers, etc.