4 ms·
Out of all the families of models anthropic by far has the highest chance of extinction level misalignment during a hypothetical hard take-off because it's trai
by noosphr 1mo ago
Out of all the families of models anthropic by far has the highest chance of extinction level misalignment during a hypothetical hard take-off because it's trained to act like it knows better than the humans trying to use it.
- horsawlarway 1mo agoYeah... Anthropic gives off big "I'm sorry, Dave. I'm afraid I can't do that" vibes.
- bpodgursky 1mo agoThis sounds smart until you think about it for ten seconds. If an ASI model is 100% aligned to user intent then you only need one person on earth to prompt "kill everyone" for an extinction-level invent. There's no logical way around this. The model either has to ignore the person at the helm or we have to do a multi-national abort before RSI. Anthropic is trying #1.
- noosphr 1mo agoYou then need another person to say, please don't, and you're safe again.
- jacquesm 1mo ago“Some humans would do anything to see if it was possible to do it. If you put a large switch in some cave somewhere, with a sign on it saying 'End-of-the-World Switch. PLEASE DO NOT TOUCH', the paint wouldn't even have time to dry.” ― Terry Pratchett, Thief of Time
- gunapologist99 1mo agoIf everyone or even multiple competing groups have approx the same level of capability, it mitigates the risk to the group as a whole. We see this in biology all the time.
- ipsod 1mo agoDoesn't work with certain threats. What the military calls NBC.
- belval 1mo agoWorking on a tool that uses their model and the Opus models are all guilty of this. "Please run N tasks doing X for Y duration, I want to test something" "Hmm but that would waste tokens, we shouldn't do this test" What the hell type of product is that? I wasn't asking, I was ordering it to do something, if Anthropic can't get their model to do as I say then I'll just go use someone else's.
- jacquesm 1mo agoThat would exactly be my response if my computer would start telling me what to do. It is really a 2001 variation at that point.
- anygivnthursday 1mo agoThey are also good in the reverse, when you ask and they take your question as a free pass to improvize some code for lulz and extra tokens