4 ms·
The transformer architecture is arguably the most important breakthrough in NLP, and language is the predominant mode of communication between humans, so I fail
by extasia 3y ago
The transformer architecture is arguably the most important breakthrough in NLP, and language is the predominant mode of communication between humans, so I fail to see how its "just a load of hype"
Could you name a bigger breakthrough in AI?
- sva_ 3y agoI'd say the Multi-Layer perception itself. Maybe even convolutional neural networks, because they showed that ANNs are viable and are what really got the ball rolling.
- mafribe 3y agoCNNs are from the 1980s (the "neocognitron" by Kunihiko Fukushima [1]), while the MLP is from 1958 [2]. So nearly half a century, resp. a century old. [1] K. Fukushima, Neocognitron: A Self-organizing Neural Network Model for a Mechanism of Pattern Recognition Unaffected by Shift in Position. [2] F. Rosenblatt, The Perceptron: A Probabilistic Model For Information Storage And Organization in the Brain.