3 ms·
> since their brains work fundamentally like human brains, while transformers are something completely different. Are they? You realize that's entirely specula
by naasking 3y ago
> since their brains work fundamentally like human brains, while transformers are something completely different.
Are they? You realize that's entirely speculative right? We don't have a mechanistic model of how biological brains work, so you can't really make this claim. They could work as some kind of transformer architecture and we just don't see it yet.
- samus 3y agoWe at least have in common with them that we are mammals. Therefore, we can very much assume that our brain is more similar to theirs than, say, an octopus' brain. Apart from that, we very much know how certain parts of the human brain work, and there is no sign that backpropagation is going on in there. And I'd rather argue that parts of our brains are similar to RNNs than to transformers. Transformers rule over RNNs because we are better at training them than RNNs, but brains learn completely differently.