4 ms·
I don't think you understand how LLMs work. They're not merely re-arranging pre-existing blocks of code. And they have been shown to develop emergent properti
by coldtea 5mo ago
I don't think you understand how LLMs work.
They're not merely re-arranging pre-existing blocks of code.
And they have been shown to develop emergent properties that weren't in their training set time and again.
They generate novel things as much as the average programmer (which works after himself having practice, exposure to codebases, and training, and reading API documentation, and such) generates novel things.
- sirsinsalot 5mo agoNot to be pedantic but the emergent properties are in the training set, and thus the model and algorithm. There's no magic coming from the universe. What makes the behavior emergent is that it can't be predicted at training time. The emergent and unpredictable output is the result of massive vector complexity being encoded.
- coldtea 5mo ago>Not to be pedantic but the emergent properties are in the training set, and thus the model and algorithm. There's no magic coming from the universe. You are either being pedantic or missing the point of emergent however. Yes, it's not some novel unforeseen thing, like a magical Marvel Universe material or some unknown to humanity mode of thinking. Same way when people make something new they still recombine known words, or colors, or physical things in the universe. It is however new capabilities that is not explicitly in the training set and can't be predicted by it. Like teaching something only calculus training materials and it figures out boolean algebra. >The emergent and unpredictable output is the result of massive vector complexity being encoded As opposed to what in humans? God given revelation?
- sirsinsalot 5mo agoMy point was that if training data + encoding/training = model with emergent behaviour The emergent behaviour is in the training data and/or encoding/training. So while I agree it is emergent from the complexity, it isn't some unknown mechanism. Just complexity at scale.
- coldtea 5mo ago>So while I agree it is emergent from the complexity, it isn't some unknown mechanism. Just complexity at scale. So like humans? Like the universe?
- dominotw 5mo agosince you understand how llms really work. show us what novel items llms have generated for you.
- coldtea 5mo agoHow about a novel theorem escaping mathematicians for close to a century? https://www.scientificamerican.com/article/ai-just-solved-an-80-year-old-erdos-problem-and-mathematicians-are-amazed/ https://www.scientificamerican.com/article/ai-just-solved-an...
- hansmayer 5mo agoThe question was, what did they solve FOR YOU.
- hansmayer 5mo ago> And they have been shown to develop emergent properties that weren't in their training set time and again. Ah yes, the famous emergent properties - like suggesting that we should walk to the car wash?