6 ms·

One way or another the current OpenAI team will split into two teams - one headed by Sam and focused on making money and another headed by Ilya and focused on c

One way or another the current OpenAI team will split into two teams - one headed by Sam and focused on making money and another headed by Ilya and focused on creating a safe AGI.

naveen99 3y ago
Ilya if he wants to fight will probably just hire Wachtell, Lipton, Rosen & Katz. the teams on both sides will be lawyers, not vc’s or engineers.
ren_engineer 3y ago
pretty sure that's the reason Anthropic exists, so Ilya would probably just go there. This isn't the first clash over AI safety Altman has caused
sgift 3y ago
Only if we define safe as "controlled by Ilya and/or some clique of 'great men'". But at least they are good at PR. After all, who could say something against safety? And no, I also don't believe that Sam A. has my best interests at heart, but at least he doesn't hide behind "safety" to pretend otherwise.
- altpaddle 3y ago
  The attitudes I'm seeing in this debate are mind blowing to me, explain to me why you think it's better that the future of AI and therefore the future of humanity is better off controlled by narrow-minded shareholder value maximizing entities like Microsoft or Silicon Valley VCs. What's wrong with the idea that AI should be developed with the goal of being safe and for the benefit of all humanity? As for the objection to safe being defined by a narrow group 'great men'. I'm sure Ilya would agree that it's not ideal that such an important milestone in human history being decided by such a small group of people. But what alternatives do we have right now? What if OpenAI decided to turn over their technology to the US government or the UN? Would that make you feel better?
  - sgift 3y ago
    > What's wrong with the idea that AI should be developed with the goal of being safe and for the benefit of all humanity? Nothing. I just don't believe that's what will happen here. All I see is two sides trying to steer things in a direction they profit most from, with one side saying - more or less openly - that that's what they want to do, while the other side says "oh no, we don't want to do this at all. We only have your best interests at heart!" and I hate people that try to take me for a ride. > What if OpenAI decided to turn over their technology to the US government or the UN? Would that make you feel better? Replace it with "a UN replacement with only democratic governments" and yes, that would make me feel better immensely. Or put it out into the open, which would be the best option if they really care about humanity.
    - altpaddle 3y ago
      I dunno man, not sure why you're so cynical. There actually are people out there who are motivated by higher ideals than their own self-interest. I don't know if Ilya and the OpenAI board are in that category but I believe the world is a better place if OpenAI is still governed by the principles in its current charter than just another extension of Microsoft/VC will
      - sgift 3y ago
        > I dunno man, not sure why you're so cynical. I've seen the "We are motivated by higher goals, trust us" plot play out one too many times, just with different actors. At some point I stopped assuming the best and instead opted for "assume the worst, maybe get pleasantly surprised once in a while". So far, I've been proven correct more often than not - as sad as that is. > There actually are people out there who are motivated by higher ideals than their own self-interest. Oh, sure there are. Just many more who aren't. And it's really hard to tell them apart, especially if both say the same things. But we live in a society which rewards the latter, so the safe option (the irony isn't lost on me) is to assume people aren't. > I don't know if Ilya and the OpenAI board are in that category That's kind of the problem, isn't it? If they aren't then OpenAI isn't governed by the principles in its charter in either case. And in that case: What is better, an OpenAI which is scrutinized at every step, because no one trusts Microsoft or an OpenAI which can do whatever it wants, cause "safe AI is our goal, trust us" and then wake up one day and find out they too betrayed our trust?
- reducesuffering 3y ago
  > But at least they are good at PR. What could possibly make you think that Ilya and AI Notkilleveryoneism is possibly winning the PR battle? HN, Tech Twitter, VC's, and major industry are all raking them over the coals. All of those people are trying to make money and get rich. Imagine thinking you're the underdog. The scientists are trying to prevent catastrophe for everyone. This is the climate change debate all over again. You wouldn't have the cajones to think climate change was real if that view was unpopular.
- phowat 3y ago
  "AI safety" is just a rehashed version of "think of the children".
happytiger 3y ago
This assumes that AGI is capable of being done “safely.” That is an interesting assumption.
- capableweb 3y ago
  Properly and absolutely airgapped should do it. Internet is the big vector that can't be closed once exploited.
  - happytiger 3y ago
    That’s a very reasonable argument.
  - ben_w 3y ago
    That's like saying we can solve aging and cancer by a combination of genetic and epigenetic interventions. It might even work, if you could do it. Actually doing that is going to be essentially impossible, though the reasons are completely different.
    - capableweb 3y ago
      Huh? We technically know how to airgap something already. You wanna be absolutely sure? Build a bunker underground, make the power and all utilities independent, allow nothing connected in there, and there you go. Do we technically know how to solve aging and cancer already?
      - ben_w 3y ago
        > though the reasons are completely different For example, but not limited to: there's no point having an AI you're not using, using it — even as an Oracle — involves some kind of interaction with the outside world in the form of what questions you ask and what actions you take in response.

One way or another the current OpenAI team will split into two teams - one headed by Sam and focused on making money and another headed by Ilya and focused on creating a safe AGI.

naveen99 3y ago

Ilya if he wants to fight will probably just hire Wachtell, Lipton, Rosen & Katz. the teams on both sides will be lawyers, not vc’s or engineers.

ren_engineer 3y ago

pretty sure that's the reason Anthropic exists, so Ilya would probably just go there. This isn't the first clash over AI safety Altman has caused

sgift 3y ago

Only if we define safe as "controlled by Ilya and/or some clique of 'great men'". But at least they are good at PR. After all, who could say something against safety? And no, I also don't believe that Sam A. has my best interests at heart, but at least he doesn't hide behind "safety" to pretend otherwise.

altpaddle 3y ago
The attitudes I'm seeing in this debate are mind blowing to me, explain to me why you think it's better that the future of AI and therefore the future of humanity is better off controlled by narrow-minded shareholder value maximizing entities like Microsoft or Silicon Valley VCs. What's wrong with the idea that AI should be developed with the goal of being safe and for the benefit of all humanity? As for the objection to safe being defined by a narrow group 'great men'. I'm sure Ilya would agree that it's not ideal that such an important milestone in human history being decided by such a small group of people. But what alternatives do we have right now? What if OpenAI decided to turn over their technology to the US government or the UN? Would that make you feel better?
- sgift 3y ago
  > What's wrong with the idea that AI should be developed with the goal of being safe and for the benefit of all humanity? Nothing. I just don't believe that's what will happen here. All I see is two sides trying to steer things in a direction they profit most from, with one side saying - more or less openly - that that's what they want to do, while the other side says "oh no, we don't want to do this at all. We only have your best interests at heart!" and I hate people that try to take me for a ride. > What if OpenAI decided to turn over their technology to the US government or the UN? Would that make you feel better? Replace it with "a UN replacement with only democratic governments" and yes, that would make me feel better immensely. Or put it out into the open, which would be the best option if they really care about humanity.
  - altpaddle 3y ago
    I dunno man, not sure why you're so cynical. There actually are people out there who are motivated by higher ideals than their own self-interest. I don't know if Ilya and the OpenAI board are in that category but I believe the world is a better place if OpenAI is still governed by the principles in its current charter than just another extension of Microsoft/VC will
    - sgift 3y ago
      > I dunno man, not sure why you're so cynical. I've seen the "We are motivated by higher goals, trust us" plot play out one too many times, just with different actors. At some point I stopped assuming the best and instead opted for "assume the worst, maybe get pleasantly surprised once in a while". So far, I've been proven correct more often than not - as sad as that is. > There actually are people out there who are motivated by higher ideals than their own self-interest. Oh, sure there are. Just many more who aren't. And it's really hard to tell them apart, especially if both say the same things. But we live in a society which rewards the latter, so the safe option (the irony isn't lost on me) is to assume people aren't. > I don't know if Ilya and the OpenAI board are in that category That's kind of the problem, isn't it? If they aren't then OpenAI isn't governed by the principles in its charter in either case. And in that case: What is better, an OpenAI which is scrutinized at every step, because no one trusts Microsoft or an OpenAI which can do whatever it wants, cause "safe AI is our goal, trust us" and then wake up one day and find out they too betrayed our trust?
reducesuffering 3y ago
> But at least they are good at PR. What could possibly make you think that Ilya and AI Notkilleveryoneism is possibly winning the PR battle? HN, Tech Twitter, VC's, and major industry are all raking them over the coals. All of those people are trying to make money and get rich. Imagine thinking you're the underdog. The scientists are trying to prevent catastrophe for everyone. This is the climate change debate all over again. You wouldn't have the cajones to think climate change was real if that view was unpopular.
phowat 3y ago
"AI safety" is just a rehashed version of "think of the children".

happytiger 3y ago

This assumes that AGI is capable of being done “safely.” That is an interesting assumption.

capableweb 3y ago
Properly and absolutely airgapped should do it. Internet is the big vector that can't be closed once exploited.
- happytiger 3y ago
  That’s a very reasonable argument.
- ben_w 3y ago
  That's like saying we can solve aging and cancer by a combination of genetic and epigenetic interventions. It might even work, if you could do it. Actually doing that is going to be essentially impossible, though the reasons are completely different.
  - capableweb 3y ago
    Huh? We technically know how to airgap something already. You wanna be absolutely sure? Build a bunker underground, make the power and all utilities independent, allow nothing connected in there, and there you go. Do we technically know how to solve aging and cancer already?
    - ben_w 3y ago
      > though the reasons are completely different For example, but not limited to: there's no point having an AI you're not using, using it — even as an Oracle — involves some kind of interaction with the outside world in the form of what questions you ask and what actions you take in response.