4 ms·
> In total, the cerebellum contains 80% of all neurons! Apples and oranges, but that's so reminiscent of MLPs in Transformers. A similarly large fraction of th
by woadwarrior01 3y ago
> In total, the cerebellum contains 80% of all neurons!
Apples and oranges, but that's so reminiscent of MLPs in Transformers. A similarly large fraction of the weights in transformers come from the MLPs in each layer.
- eurekin 3y agoWell, MLPs is the actual neural network (the approximator), whereas the attention (the rest) is more of a text to relevant embedding extractor