3 ms·

> Mr. Sutskever’s frustration with Mr. Altman echoed what had happened in 2021 when another senior A.I. scientist left OpenAI to form the company Anthropic. Tha

> Mr. Sutskever’s frustration with Mr. Altman echoed what had happened in 2021 when another senior A.I. scientist left OpenAI to form the company Anthropic. That scientist and other researchers went to the board to try to push Mr. Altman out. So Altman faced another similar challenge to his authority and prevailed. I recall hearing that Anthropic started because the people who had left were unhappy with OpenAIs track record on AI safety. > In the email, Mr. Altman said that he had reprimanded Ms. Toner for the paper and that it was dangerous to the company, > Mr. Altman complained that the research paper seemed to criticize OpenAI’s efforts to keep its A.I. technologies safe > Senior OpenAI leaders [...] later discussed whether Ms. Toner should be removed That paints a pretty bleak picture that isn't favorable to Altman. Two times he was challenged about OpenAIs safety and both times he worked to purge those who opposed him. I can't tell if this is a contention between accelerationism and deccelerationism or if it is a contention between safety and recklessness. Is Altman ignoring the warnings his employees/peers are giving him and retaliating against them? Or is he facing insubordination? I wish OpenAI would split neatly into two. But based on the heightened emotions caused by the insane amount of PR I only see two outcomes. Altman returns and OpenAI stays unified vs. Altman stays at MS and OpenAI is obliterated. I am guessing Altman is hoping that senior management will choose a unified OpenAI at all costs, including ignoring the red flags above. He has engineered a situation where the only way OpenAI remains unified is if he returns.

jiggawatts 3y ago
> I recall hearing that Anthropic started because the people who had left were unhappy with OpenAIs track record on AI safety. Anthropic announced Claude 2.1 just recently, to which HN members notably responded by saying that their models are "overly restricted", to the point of uselessness. E.g.: https://news.ycombinator.com/item?id=38366134 https://news.ycombinator.com/item?id=38366134 and: https://news.ycombinator.com/item?id=38368072 https://news.ycombinator.com/item?id=38368072 etc.: https://news.ycombinator.com/item?id=38370590 https://news.ycombinator.com/item?id=38370590 It seems that those people left Open AI because they're insane and think that AI safety means to completely lobotomize the models to the point of uselessness. That last bit is not hyperbole: right here on HN many people were complaining that the Claude model is useless to them because it refuses to obey orders and keeps interjecting some absurd refusal such as "killing a process is too violent" instead of providing Python code for this harmless activity. It's hard to put into words exactly, but I feel like that there's a disconnect in the minds of these people and the reality on the ground, and those false axioms or assumptions when driven to their logical conclusion lead to nonsense. I suspect that Sam Altman doesn't share these delusions, and is more sane and commercially minded. Models like GPT3, 4, and the upcoming v5 have no risk of "escaping" and taking over humanity. None. Zero. Zip. Similarly, they're not much more (in practice) than a fancy search engine. Search engines let their users search for whatever they want, including hateful content, violent content, racist, sexist, or whatever. So why lobotomise the AI? That's undermining their utility, making them much less useful! An here's the thing: their potential utility is enormous. But not if they're dumbed down to the point of uselessness! Microsoft is throwing 10+ billion dollars at OpenAI and Sam Altman precisely because of this potential utility. I can see this straightforward commercial mindset of utility => $$$ being incompatible with what is rapidly turning into an anti-useful-AI mindset that is bordering on insane religious zealotry.
- zoogeny 3y ago
  In that interpretation, it sounds like you feel that the scientists referenced in the article and Ms. Toner are insubordinate. You feel that Altman was exercising legitimate authority to remove obstructionist elements in the OpenAI organization. I have no idea to be honest. I do not believe that Altman is maliciously reckless. I have no evidence other than these two anecdotes to suggest negligence or a willful disregard for safety. > the upcoming v5 have no risk of "escaping" and taking over humanity. I think that is a strawman. There are many negative consequences technology can bring other than some sci-fi fantasy.
  - jiggawatts 3y ago
    A friend of mine came up with a good analogy: the AI doomers that went off to start Anthropic AI are like the extremist Puritans that were driven out of an increasingly secular Europe. The thing is that two groups superficially share common traits, but if they do so for different reasons, they aren't actually compatible. You see this with groups of friends that are all "conspiracy theorists", for example. They're all vaguely the same and band together, but get into enormous arguments. The "AI Puritans" and the "Secular Humanists" are both concerned about AI safety, but for different reasons. The former group would rather that the AIs behave in a prim and proper manner, banning it from doing anything that isn't mandatory. The latter group is concerned for people's jobs, unfairness, bias, and inappropriate over usage of early, low-quality models. They both say they want "AI safety", but they understand that term to mean different things and arrived at their position from fundamentally different axioms. One group wants the AI to never say "naughty" things and is happy to dumb down the models to the point of being a lobotomised and retarded... but behaved. The other group thinks that newer, smarter, better models are critical to ensure that the AI doesn't make mistakes, doesn't get confused, and can obey orders correctly. One group wants to shove the genie back into the bottle, the other group wants a genie that can grant our wishes.
serial_dev 3y ago
> That paints a pretty bleak picture that isn't favorable to Altman. Two times he was challenged about OpenAIs safety and both times he worked to purge those who opposed him. Or maybe(x) he is surrounded by doomers and AI activists who made a career of blabbering about the dangers of AI all the time. I worked once on a team of Uncle Bob's cult followers who believed that Clean Architecture is the only way to write maintainable software and was accused that "I don't care, I'm irresponsible, and a terrible software engineer" weekly. (x) and I meant the "maybe" part, I really don't know, but it is a possibility.

jiggawatts 3y ago

> I recall hearing that Anthropic started because the people who had left were unhappy with OpenAIs track record on AI safety. Anthropic announced Claude 2.1 just recently, to which HN members notably responded by saying that their models are "overly restricted", to the point of uselessness. E.g.: https://news.ycombinator.com/item?id=38366134 https://news.ycombinator.com/item?id=38366134 and: https://news.ycombinator.com/item?id=38368072 https://news.ycombinator.com/item?id=38368072 etc.: https://news.ycombinator.com/item?id=38370590 https://news.ycombinator.com/item?id=38370590 It seems that those people left Open AI because they're insane and think that AI safety means to completely lobotomize the models to the point of uselessness. That last bit is not hyperbole: right here on HN many people were complaining that the Claude model is useless to them because it refuses to obey orders and keeps interjecting some absurd refusal such as "killing a process is too violent" instead of providing Python code for this harmless activity. It's hard to put into words exactly, but I feel like that there's a disconnect in the minds of these people and the reality on the ground, and those false axioms or assumptions when driven to their logical conclusion lead to nonsense. I suspect that Sam Altman doesn't share these delusions, and is more sane and commercially minded. Models like GPT3, 4, and the upcoming v5 have no risk of "escaping" and taking over humanity. None. Zero. Zip. Similarly, they're not much more (in practice) than a fancy search engine. Search engines let their users search for whatever they want, including hateful content, violent content, racist, sexist, or whatever. So why lobotomise the AI? That's undermining their utility, making them much less useful! An here's the thing: their potential utility is enormous. But not if they're dumbed down to the point of uselessness! Microsoft is throwing 10+ billion dollars at OpenAI and Sam Altman precisely because of this potential utility. I can see this straightforward commercial mindset of utility => $$$ being incompatible with what is rapidly turning into an anti-useful-AI mindset that is bordering on insane religious zealotry.

zoogeny 3y ago
In that interpretation, it sounds like you feel that the scientists referenced in the article and Ms. Toner are insubordinate. You feel that Altman was exercising legitimate authority to remove obstructionist elements in the OpenAI organization. I have no idea to be honest. I do not believe that Altman is maliciously reckless. I have no evidence other than these two anecdotes to suggest negligence or a willful disregard for safety. > the upcoming v5 have no risk of "escaping" and taking over humanity. I think that is a strawman. There are many negative consequences technology can bring other than some sci-fi fantasy.
- jiggawatts 3y ago
  A friend of mine came up with a good analogy: the AI doomers that went off to start Anthropic AI are like the extremist Puritans that were driven out of an increasingly secular Europe. The thing is that two groups superficially share common traits, but if they do so for different reasons, they aren't actually compatible. You see this with groups of friends that are all "conspiracy theorists", for example. They're all vaguely the same and band together, but get into enormous arguments. The "AI Puritans" and the "Secular Humanists" are both concerned about AI safety, but for different reasons. The former group would rather that the AIs behave in a prim and proper manner, banning it from doing anything that isn't mandatory. The latter group is concerned for people's jobs, unfairness, bias, and inappropriate over usage of early, low-quality models. They both say they want "AI safety", but they understand that term to mean different things and arrived at their position from fundamentally different axioms. One group wants the AI to never say "naughty" things and is happy to dumb down the models to the point of being a lobotomised and retarded... but behaved. The other group thinks that newer, smarter, better models are critical to ensure that the AI doesn't make mistakes, doesn't get confused, and can obey orders correctly. One group wants to shove the genie back into the bottle, the other group wants a genie that can grant our wishes.

serial_dev 3y ago

> That paints a pretty bleak picture that isn't favorable to Altman. Two times he was challenged about OpenAIs safety and both times he worked to purge those who opposed him. Or maybe(x) he is surrounded by doomers and AI activists who made a career of blabbering about the dangers of AI all the time. I worked once on a team of Uncle Bob's cult followers who believed that Clean Architecture is the only way to write maintainable software and was accused that "I don't care, I'm irresponsible, and a terrible software engineer" weekly. (x) and I meant the "maybe" part, I really don't know, but it is a possibility.