3 ms·
By a statistical model I mean a very large, highly dimensional data manifold on which classification is done to predict the next token. The correct token can be
by adammarples 4y ago
By a statistical model I mean a very large, highly dimensional data manifold on which classification is done to predict the next token. The correct token can be predicted very often because somehow the common patterns in the written training data are being efficiently compressed in this highly dimensional space. Humans are also able to efficiently compress the data they are trained on, but the structures inside a human and a transformer are completely different, what happens at inference time is completely different. They both have language in common as an output and can both answer TOM questions but this doesn't mean they both have a theory of mind.
- bottlepalm 4y agoWhy do you think the wiring of a biological and artificial neural net are completely different? The results from both are remarkably similar.
- adammarples 4y agoWe know from neuroscience that real neurons are very different from ANN neurons. For example, brains don't even use backpropagation. The neurons are connected differently. They have way more going on than just activation. Humans and ML models behave very differently.