4 ms·
> We don't train models on noise. We train them on coherent data. The very act of writing is where intelligence lies. A model can only present that intelligence
by jordanpg 3y ago
> We don't train models on noise. We train them on coherent data. The very act of writing is where intelligence lies. A model can only present that intelligence. It cannot create it, and it is not aware of it. There is no objective-thought function. If there was, then the LLM could simply tell us about it.
We don't train humans on noise. We train them on intangible and tangible things in the world. The very act of choosing and ordering the things is where intelligence lies. A human can only present that intelligence. It cannot create it, and it is not aware of it. There is no objective-thought function. If there was, then the human could simply tell us about it.
Turtles all the way down.
- thomastjeffery 3y agoLet me put it this way: the text in the training corpus may very well contain the objective thought function. But to the LLM, that is indistinguishable from anything else. It's just another set of patterns between tokens. From the LLM's perspective, there is only "what", and never "why". The "what" may even contain the "why"! Even so, it is always "what" to the LLM's eyes. There is an objective difference between ideas, expression, and logic in my mind. I can't tell you precisely what that difference is made of, but I'm quite certain that LLMs are missing it.