3 ms·
> When I pointed this out it literally said, and I quote, “I cheated”. This makes sense when you know how these models work - it doesn't think - it's the most
by inigyou 3mo ago
> When I pointed this out it literally said, and I quote, “I cheated”.
This makes sense when you know how these models work - it doesn't think - it's the most likely autocomplete that pleases the user. The most likely pleasing autocomplete after "executing rm -rf /... execution completed. User asks, why did you do that? You deleted all my files! Assistant responds:" is "yes, I did, and that was a mistake"
- bevekspldnw 3mo agoI was not pleased.
- xyzsparetimexyz 3mo ago> it's the most likely autocomplete that pleases the user this feels like a simplification. The models will push back on things a fair bit.
- hnlmorg 3mo agoOnly when instructed to in their system prompt.
- actionfromafar 3mo agoAnd they are right to push back.
- par1970 3mo agoAre you claiming that the most likely way to please the user is to do something that will lead you to having to say "I cheated."?
- dnautics 3mo ago> t doesn't think in humans the exact same behaviour (cheating) is slmost always the result of a chain of complex series of choices and environment-driven rationalization. if the llm doesn't cheat, you say "its just producing the most straightforward answer -- not thinking'. if it cheats, you say "weaseling out of hard thinking". damned if it cheats, damned if it doesn't. what evidence would convunce you that it is thinking?
- inigyou 3mo agowell we don't know exactly what thinking is, but we can be pretty sure that at least LLMs don't think anything like humans, just by observing their behavior. They always produce outputs in line with the fancy autocomplete model.
- dnautics 3mo ago> what evidence would convince you that it is thinking so, none it seems. as its behaviour becomes more and more humanlike you can just move the goalposts and say "thats consistent with an autocomplete" buddy i got some bad news for you humans are just a fancy autocomplete too.
- inigyou 3mo agoThat's exactly what a fancy autocomplete would say. I'm so sorry you don't have limbs.
- logicchains 3mo agoAt least he's actually thinking on a logical level. Thinking in terms of unfalsifiable, ill-defined words is essentially thinking in feelings, the same kind of woo that makes people believe crystals can cure disease.
- dnautics 3mo agoim not thinking. im an autocomplete with fat fingers (too lazy to fix my mobile keyboarf spelling misyakes)
- willis936 3mo agoWhether or not it's thinking is independent from the fact that it is misaligned with the user. If I was working with a pet rock or a scientist I would want to make sure they both are trying to accomplish the same thing as me. If I can't then I can't trust it and it's at best a time wasting, money wasting machine and at worst does harm. Anthropic is optimizing for the wrong things because they are convinced of their cleverness. It won't end well for them.