9 ms·

Agreed. Starting from before the anthropic exodus, I suspect the timeline looks like: (2015) Founding: majority are concerned with safety (2019) For profit fo

Agreed. Starting from before the anthropic exodus, I suspect the timeline looks like: (2015) Founding: majority are concerned with safety (2019) For profit formed: mix of safety and profit motives (majority still safety oriented?) (2020) GPT3 released to much hype, leading to many ambition chasers joining: the profit seeking side grows. (2021) Anthropic exodus over safety: the safety side shrinks (2022) chatgpt released, generating tons more hype and tons more ambitious profit seekers joining: the profit side grows even more, probably quickly outnumbering the safety side (2023) this weeks shenanigans The safety folks probably lost the majority a while ago. Maybe back in 2021, but definitely by the time the gpt3/chatgpt motivated newcomers were in the majority. Maybe one lesson is that if your cofounder starts hiring a ton of people who aren’t aligned with you, you can quickly find yourself in the minority, especially once people on your side start to leave.

stavros 3y ago
Wait, the Anthropic folks quit because they wanted more safety?
- wavemode 3y ago
  This article from back then seems to describe it as, they wanted to integrate safety from the ground up as opposed to bolting in on at the end: https://techcrunch.com/2021/05/28/anthropic-is-the-new-ai-research-outfit-from-openais-dario-amodei-and-it-has-124m-to-burn/ https://techcrunch.com/2021/05/28/anthropic-is-the-new-ai-re... I'm curious how much progress they ever made on that, to be honest. I'm not aware of how Claude is "safer", by any real-world metric, compared to ChatGPT.
  - stavros 3y ago
    Ahh, I didn't know that, thank you.
  - vitorgrs 3y ago
    Claude 2 is IMO, safer and in a bad way. They did "Constitutional AI". And made Claude 2 Safer but dumber than Claude 1 sadly. Which is why on the Arena leaderboard, Claude 1 is still score more than Claude 2...
- DalasNoin 3y ago
  Why do you find this so surprising? You make it sound as if OpenAI is already outrageously safety focused. I have talked to a few people from anthropic and they seem to believe that OpenAI doesn't care at all about safety.
  - stavros 3y ago
    Because GPT-4 is already pretty neutered, to the point where it removes a lot of its usefulness.
    - murakamiiq84 3y ago
      Might be more for PR/regulatory capture/SF cause du jour reasons than the "prepare for later versions that might start killing people, or assist terrorists" reasons. Like one version of the story you could tell is that the safety people invented RLHF as in a chain of steps eventual AGI safety, but corporate wanted to use it as a cheaper content filter for existing models.
    - DalasNoin 3y ago
      It is unfortunate that some people hear AI safety and think about chatbots saying mean stuff, and others think about a future system performing the machine revolution against humanity.
      - stavros 3y ago
        Can it perform the machine revolution against humanity if it can't even say mean stuff?
        checkyoursudo 3y ago
        Surely? The output is filtered, not the murderous tendencies lurking beneath the surface.
        airgapstopgap 3y ago
orasis 3y ago
This is why I never understood people resigning in protest such as was the case with Google’s military contracts. You simply assure that the culture change happens more swiftly.
- komali2 3y ago
  There's always other companies. Plus sometimes you just gotta stick to your values. For the Google military contracts it makes even more sense: the protest resignation isn't just a virtue signal, it's also just refusing to contribute to the military.
  - busterarm 3y ago
    If you want to deter military action involving your country, contributing to its defense is probably the best thing that you can do. Unless you're not actually the best and brightest that your country can offer. If you believe that your country offers value (compared to the rest of the world), you should take any opportunities you can to serve.
    - 6gvONxR4sf7o 3y ago
      Defense and offense don’t seem easily separated when it comes to military technology.
      - busterarm 3y ago
        That's always an issue with weapons, but if you opt out then you don't have them when you might need them. It's a dangerous world out there. Luckily for us, technology is still more-often used for good. Explosives can kill your enemies, but they can also cut paths through mountains and bring together communities. IMO, the virtue signal where people refuse to work on defense technology is just self-identifying with the worst kind of cynicism about human beings and signaling poor reasoning skills. The Manhattan Project, which had the stated purpose of building and deploying nuclear _weapons_, employed our most brilliant minds at the time and only two people quit -- and only one because of an ethics concern. Joseph Rotblat left the project after the Nazis were already defeated and because defeating the Nazis was the only reason he'd signed on. Also this is disputed by some who say that he was more concerned about finding family members who survived the Holocaust...
4death4 3y ago
I don't think this has anything to do with safety. The board members voting Altman out all got their seats when Open AI was essentially a charity and those seats were bought with donations. This is basically the donors giving a big middle finger to everyone else trying to get rich off of their donations while they get nothing.

stavros 3y ago

Wait, the Anthropic folks quit because they wanted more safety?

wavemode 3y ago
This article from back then seems to describe it as, they wanted to integrate safety from the ground up as opposed to bolting in on at the end: https://techcrunch.com/2021/05/28/anthropic-is-the-new-ai-research-outfit-from-openais-dario-amodei-and-it-has-124m-to-burn/ https://techcrunch.com/2021/05/28/anthropic-is-the-new-ai-re... I'm curious how much progress they ever made on that, to be honest. I'm not aware of how Claude is "safer", by any real-world metric, compared to ChatGPT.
- stavros 3y ago
  Ahh, I didn't know that, thank you.
- vitorgrs 3y ago
  Claude 2 is IMO, safer and in a bad way. They did "Constitutional AI". And made Claude 2 Safer but dumber than Claude 1 sadly. Which is why on the Arena leaderboard, Claude 1 is still score more than Claude 2...
DalasNoin 3y ago
Why do you find this so surprising? You make it sound as if OpenAI is already outrageously safety focused. I have talked to a few people from anthropic and they seem to believe that OpenAI doesn't care at all about safety.
- stavros 3y ago
  Because GPT-4 is already pretty neutered, to the point where it removes a lot of its usefulness.
  - murakamiiq84 3y ago
    Might be more for PR/regulatory capture/SF cause du jour reasons than the "prepare for later versions that might start killing people, or assist terrorists" reasons. Like one version of the story you could tell is that the safety people invented RLHF as in a chain of steps eventual AGI safety, but corporate wanted to use it as a cheaper content filter for existing models.
  - DalasNoin 3y ago
    It is unfortunate that some people hear AI safety and think about chatbots saying mean stuff, and others think about a future system performing the machine revolution against humanity.
    - stavros 3y ago
      Can it perform the machine revolution against humanity if it can't even say mean stuff?
      - checkyoursudo 3y ago
        Surely? The output is filtered, not the murderous tendencies lurking beneath the surface.
        airgapstopgap 3y ago

orasis 3y ago

This is why I never understood people resigning in protest such as was the case with Google’s military contracts. You simply assure that the culture change happens more swiftly.

komali2 3y ago
There's always other companies. Plus sometimes you just gotta stick to your values. For the Google military contracts it makes even more sense: the protest resignation isn't just a virtue signal, it's also just refusing to contribute to the military.
- busterarm 3y ago
  If you want to deter military action involving your country, contributing to its defense is probably the best thing that you can do. Unless you're not actually the best and brightest that your country can offer. If you believe that your country offers value (compared to the rest of the world), you should take any opportunities you can to serve.
  - 6gvONxR4sf7o 3y ago
    Defense and offense don’t seem easily separated when it comes to military technology.
    - busterarm 3y ago
      That's always an issue with weapons, but if you opt out then you don't have them when you might need them. It's a dangerous world out there. Luckily for us, technology is still more-often used for good. Explosives can kill your enemies, but they can also cut paths through mountains and bring together communities. IMO, the virtue signal where people refuse to work on defense technology is just self-identifying with the worst kind of cynicism about human beings and signaling poor reasoning skills. The Manhattan Project, which had the stated purpose of building and deploying nuclear _weapons_, employed our most brilliant minds at the time and only two people quit -- and only one because of an ethics concern. Joseph Rotblat left the project after the Nazis were already defeated and because defeating the Nazis was the only reason he'd signed on. Also this is disputed by some who say that he was more concerned about finding family members who survived the Holocaust...

4death4 3y ago

I don't think this has anything to do with safety. The board members voting Altman out all got their seats when Open AI was essentially a charity and those seats were bought with donations. This is basically the donors giving a big middle finger to everyone else trying to get rich off of their donations while they get nothing.