3 ms·
> they are not like humans at all, and only resemble them superficially We have no idea how LLMs work on a high level, and we have no idea how the human mind w
by p-e-w 3y ago
> they are not like humans at all, and only resemble them superficially
We have no idea how LLMs work on a high level, and we have no idea how the human mind works on a high level. Therefore, such claims are rather overconfident. The fact that the two are dissimilar at the plumbing layer doesn't mean they cannot be alike in how they "really" operate. Either way, we simply do not know, so any such talk is (bad) philosophy, not science.
- krainboltgreene 3y ago> We have no idea how LLMs work on a high level This is just incredibly wrong. Do you think that LLM's were created out of thin air? A random combination of bytes that we happened to discover that we also happen to be improving/changing?
- frabcus 3y agoWe understand the training algorithm, but not the network that was trained. There’s the whole interpretability sub-discipline which has limited but interesting results. Those just indicate even more that we really have no idea how something like GPT-4 works inside its trained model.
- p-e-w 3y agoNobody knows how LLMs work on a high level. There's an algorithm that consumes Terabytes of raw text and produces 175 billion parameters, that when used as a statistical model magically reproduce human language. But the conceptual connection between those parameters and what the model can do is a complete mystery.