4 ms·

> I recall hearing that Anthropic started because the people who had left were unhappy with OpenAIs track record on AI safety. Anthropic announced Claude 2.1 j

> I recall hearing that Anthropic started because the people who had left were unhappy with OpenAIs track record on AI safety. Anthropic announced Claude 2.1 just recently, to which HN members notably responded by saying that their models are "overly restricted", to the point of uselessness. E.g.: https://news.ycombinator.com/item?id=38366134 https://news.ycombinator.com/item?id=38366134 and: https://news.ycombinator.com/item?id=38368072 https://news.ycombinator.com/item?id=38368072 etc.: https://news.ycombinator.com/item?id=38370590 https://news.ycombinator.com/item?id=38370590 It seems that those people left Open AI because they're insane and think that AI safety means to completely lobotomize the models to the point of uselessness. That last bit is not hyperbole: right here on HN many people were complaining that the Claude model is useless to them because it refuses to obey orders and keeps interjecting some absurd refusal such as "killing a process is too violent" instead of providing Python code for this harmless activity. It's hard to put into words exactly, but I feel like that there's a disconnect in the minds of these people and the reality on the ground, and those false axioms or assumptions when driven to their logical conclusion lead to nonsense. I suspect that Sam Altman doesn't share these delusions, and is more sane and commercially minded. Models like GPT3, 4, and the upcoming v5 have no risk of "escaping" and taking over humanity. None. Zero. Zip. Similarly, they're not much more (in practice) than a fancy search engine. Search engines let their users search for whatever they want, including hateful content, violent content, racist, sexist, or whatever. So why lobotomise the AI? That's undermining their utility, making them much less useful! An here's the thing: their potential utility is enormous. But not if they're dumbed down to the point of uselessness! Microsoft is throwing 10+ billion dollars at OpenAI and Sam Altman precisely because of this potential utility. I can see this straightforward commercial mindset of utility => $$$ being incompatible with what is rapidly turning into an anti-useful-AI mindset that is bordering on insane religious zealotry.

zoogeny 3y ago
In that interpretation, it sounds like you feel that the scientists referenced in the article and Ms. Toner are insubordinate. You feel that Altman was exercising legitimate authority to remove obstructionist elements in the OpenAI organization. I have no idea to be honest. I do not believe that Altman is maliciously reckless. I have no evidence other than these two anecdotes to suggest negligence or a willful disregard for safety. > the upcoming v5 have no risk of "escaping" and taking over humanity. I think that is a strawman. There are many negative consequences technology can bring other than some sci-fi fantasy.
- jiggawatts 3y ago
  A friend of mine came up with a good analogy: the AI doomers that went off to start Anthropic AI are like the extremist Puritans that were driven out of an increasingly secular Europe. The thing is that two groups superficially share common traits, but if they do so for different reasons, they aren't actually compatible. You see this with groups of friends that are all "conspiracy theorists", for example. They're all vaguely the same and band together, but get into enormous arguments. The "AI Puritans" and the "Secular Humanists" are both concerned about AI safety, but for different reasons. The former group would rather that the AIs behave in a prim and proper manner, banning it from doing anything that isn't mandatory. The latter group is concerned for people's jobs, unfairness, bias, and inappropriate over usage of early, low-quality models. They both say they want "AI safety", but they understand that term to mean different things and arrived at their position from fundamentally different axioms. One group wants the AI to never say "naughty" things and is happy to dumb down the models to the point of being a lobotomised and retarded... but behaved. The other group thinks that newer, smarter, better models are critical to ensure that the AI doesn't make mistakes, doesn't get confused, and can obey orders correctly. One group wants to shove the genie back into the bottle, the other group wants a genie that can grant our wishes.

zoogeny 3y ago

In that interpretation, it sounds like you feel that the scientists referenced in the article and Ms. Toner are insubordinate. You feel that Altman was exercising legitimate authority to remove obstructionist elements in the OpenAI organization. I have no idea to be honest. I do not believe that Altman is maliciously reckless. I have no evidence other than these two anecdotes to suggest negligence or a willful disregard for safety. > the upcoming v5 have no risk of "escaping" and taking over humanity. I think that is a strawman. There are many negative consequences technology can bring other than some sci-fi fantasy.

jiggawatts 3y ago
A friend of mine came up with a good analogy: the AI doomers that went off to start Anthropic AI are like the extremist Puritans that were driven out of an increasingly secular Europe. The thing is that two groups superficially share common traits, but if they do so for different reasons, they aren't actually compatible. You see this with groups of friends that are all "conspiracy theorists", for example. They're all vaguely the same and band together, but get into enormous arguments. The "AI Puritans" and the "Secular Humanists" are both concerned about AI safety, but for different reasons. The former group would rather that the AIs behave in a prim and proper manner, banning it from doing anything that isn't mandatory. The latter group is concerned for people's jobs, unfairness, bias, and inappropriate over usage of early, low-quality models. They both say they want "AI safety", but they understand that term to mean different things and arrived at their position from fundamentally different axioms. One group wants the AI to never say "naughty" things and is happy to dumb down the models to the point of being a lobotomised and retarded... but behaved. The other group thinks that newer, smarter, better models are critical to ensure that the AI doesn't make mistakes, doesn't get confused, and can obey orders correctly. One group wants to shove the genie back into the bottle, the other group wants a genie that can grant our wishes.