4 ms·
Large language models can do jaw-dropping things. But nobody knows why
- deleted 3y ago[deleted]
- dkjaudyeqooe 3y agoThis goes to the heart of what's interesting about LLMs. Many people see what LLMs do and are happily convinced that there is some sort of intelligence in there and extrapolate to AGI. But LLMs aren't proof of any sort of machine intelligence, let alone AGI. Human evaluation of LLMs as a basis for determining intelligence is unscientific to say the least (Turing was way off in this respect). What is interesting about LLMs is that such extraordinary performance emerges just based on scale. Give it more and you suddenly get a much, much better predictive function from the data. What's really missing from current AI research (from what I can tell) is design of, or even speculation about, higher order architectures. Our brains are clearly higher order: we think about thinking and its correctness, we're constantly examining ourselves and examining our examinations, and so on. Various parts of our brains manage other parts our brain. Generative adversarial networks seem like just the tip of the iceberg of higher-order possibilities for AI.