7 ms·
> This also highlights the importance of model design and training. While Claude is able to respond in a highly sophisticated manner, it tends to do so only wh
by mlsu 9mo ago
> This also highlights the importance of model design and training. While Claude is able to respond in a highly sophisticated manner, it tends to do so only when users input sophisticated prompts.
If the output of the model depends on the intelligence of the person picking outputs out of its training corpus, is the model intelligent?
This is kind of what I don't quite understand when people talk about the models being intelligent. There's a huge blindspot, which is that the prompt entirely determines the output.
- sjajshha 9mo ago[dead]
- thousand_nights 9mo agoi don't know, are we intelligent? you could argue that our input (senses) entirely define the output (thoughts, muscle movements, etc)
- HPsquared 9mo agoThere's a bit of baked-in stuff as well. We are a full culture-mind-body[-spirit] system.
- fuzzfactor 9mo agoFortunately we've got the full system because even under ideal conditions nobody's actually ever been intelligent at all times and we need the momentum from that full system to resume in an intelligent direction after an upset when it's not all at its best.
- deleted 9mo ago[deleted]
- bossyTeacher 9mo agoThe whole point of humans is the way we process the input. Every life form out there receives sound vibrations and has photons hitting their body all the time, not everyone uses that information in the same way or at all. That plus natal reflexes and hardcoded assumptions
- wat10000 9mo agoA smart person will tailor their answers to the perceived level of knowledge of the person asking, and the sophistication of the question is a big indicator of this.
- deleted 9mo ago[deleted]
- TrainedMonkey 9mo agoHumans also respond differently when prompted in different ways. For example, politeness often begets politeness. I would expect that to be reflected in training data.
- HPsquared 9mo agoI think that's what is happening. It's simulating a conversation, after all. A bit like code switching.
- mlsu 9mo agoIf I, a moron, hire a PhD to crack a tough problem for me, I don't need to go back and forth prompting him at a PhD level. I can set him loose on my problem and he'll come back to me with a solution.
- Herring 9mo agoWell if it ever gets to be a full replacement for phds, you’ll know cause it will have already replaced you.
- nl 9mo ago> hire a PhD to crack a tough problem for me, I don't need to go back and forth prompting him at a PhD level. I can set him loose on my problem and he'll come back to me with a solution. In my experience with many PhDs they are just as prone to getting off track or using their pet techniques as LLMs! And many find it very hard to translate their work into everyday language too...
- chasd00 9mo agoUnless your problem fits the very narrow but very deep area of expertise of the PhD you’re not going to get anything. The phds I have worked with can’t tie their shoes because that wasn’t in their dissertation.
- preciousoo 9mo agoThe PhD can't read minds, the quality if the request from a moron would be worse than the quality of the request from someone with avg intelligence. And the output would probably noticeably differ accordingly
- zozbot234 9mo agoWhat is a "sophisticated prompt"? What if I just tack on "please think about this a lot and respond in a highly sophisticated manner" to my question/prompt? Anyone can do this once they're made aware of this potential issue. Sometimes the UX layer even adds this for you in the system prompt, you just have to tick the checkbox for "I want a long, highly sophisticated answer".
- mlsu 9mo agoThey have a chart that shows it. The education level of the input determines the education level of the output. These things are supposed to have intelligence on tap. I'll imagine this in a very simple way. Let's say "intellignce" is like a fluid. It's a finite thing. Intelligence is very valuable, it's the substrate for real-world problem solving that makes these things ostensibly worth trillions of dollars. Intelligence comes from interaction with the world; someone's education and experience. You spend some effort and energy feeding someone, clothing them, sending them to college. And then you get something out, which is intelligence that can create value for society. When you are having a conversation with the AI, is the intelligence flowing out of the AI? Or is it flowing out of the human operator? The answer to this question is extremely important. If the AI can be intelligent "on its own" without a human operator, then it will be very valuable -- feed electricity into a datacenter and out comes business value. But if a model is only intelligent as someone using it, well, the utility seems to be very harshly capped. At best it saves a bit of time, but it will never do anything novel, it will never create value on its own, independently, it will never scale beyond a 1:1 "human picking outputs". If you must encode intelligence into the prompt to get intelligence out of the model, well, this doesn't quite look like AGI does it?
- mlsu 9mo agoofc what I'm getting at is, you can't get something from nothing. There is no free lunch. You spend energy distilling the intelligence of the entire internet into a set of weights, but you still had to expend the energy to have humans create the internet first. And on top of this, in order to pick out what you want from the corpus, you have to put some energy in: first, the energy of inference, but second and far more importantly, the energy of prompting. The model is valuable because the dataset is valuable; the model output is valuable because the prompt is valuable. So wait then, where does this exponential increase in value come from again?