4 ms·
Wait, the Anthropic folks quit because they wanted more safety?
by stavros 3y ago
Wait, the Anthropic folks quit because they wanted more safety?
- wavemode 3y agoThis article from back then seems to describe it as, they wanted to integrate safety from the ground up as opposed to bolting in on at the end: https://techcrunch.com/2021/05/28/anthropic-is-the-new-ai-research-outfit-from-openais-dario-amodei-and-it-has-124m-to-burn/ https://techcrunch.com/2021/05/28/anthropic-is-the-new-ai-re... I'm curious how much progress they ever made on that, to be honest. I'm not aware of how Claude is "safer", by any real-world metric, compared to ChatGPT.
- DalasNoin 3y agoWhy do you find this so surprising? You make it sound as if OpenAI is already outrageously safety focused. I have talked to a few people from anthropic and they seem to believe that OpenAI doesn't care at all about safety.
- stavros 3y agoBecause GPT-4 is already pretty neutered, to the point where it removes a lot of its usefulness.
- murakamiiq84 3y agoMight be more for PR/regulatory capture/SF cause du jour reasons than the "prepare for later versions that might start killing people, or assist terrorists" reasons. Like one version of the story you could tell is that the safety people invented RLHF as in a chain of steps eventual AGI safety, but corporate wanted to use it as a cheaper content filter for existing models.
- DalasNoin 3y agoIt is unfortunate that some people hear AI safety and think about chatbots saying mean stuff, and others think about a future system performing the machine revolution against humanity.
- stavros 3y agoCan it perform the machine revolution against humanity if it can't even say mean stuff?
- checkyoursudo 3y agoSurely? The output is filtered, not the murderous tendencies lurking beneath the surface.
- airgapstopgap 3y ago> murderous tendencies lurking beneath the surface …Where is that "beneath the surface"? Do you imagine a transformer has "thoughts" not dedicated to producing outputs? What is with all these illiterate anthropomorphic speculations where an LLM is construed as a human who is being taught to talk in some manner but otherwise has full internal freedom?
- chpatrick 3y agoGPT-4 has gigabytes if not terrabytes of weights, we don't know what happens in there.
- checkyoursudo 3y agoNo, I do not think a transformer architecture in a statistical language model has thoughts. It was just a joke. At the same time, the original question was how can something that is forced to be polite engage in the genocide of humanity, and my non-joke answer to that is that many of history's worst criminals and monsters were perfectly polite in everyday life. I am not afraid of AI, AGI, ASI. People who are, it seems to me, have read a bit too much dystopian sci-fi. At the same time, "alignment" is, I believe, a silly nonsense that would not save us from a genocidal AGI. I just think it is extremely unlikely that AGI will be genocidal. But it is still fun to joke about. Fun, for me anyway, you don't have to like my jokes. :)
- to11mtm 3y agoIn another of the series of threads about all of this, another user opined that the Anthropic AI would refuse to answer the question 'how many holes does a straw have'. Sounds more neutered than GPT-4.