4 ms·

>What's even more egregious is that the people behind the "slowing down" camp are also the people who've had 10s of billions invested in their companies. It sme

>What's even more egregious is that the people behind the "slowing down" camp are also the people who've had 10s of billions invested in their companies. It smells of "we got ours, now you can't follow". It reeks of regulatory capture, which is a proven net downside for everyone else. This sort of cynicism feels un-falsifiable. If companies advocate against regulation, the cynics will say: "These corporations want capitalism to run amok with no regulation to interfere with their profits. They need to go full steam ahead in order to show profits for their investors." If companies advocate in favor of regulation, the cynics will say: "Regulatory capture." If you have legitimate concerns, how about a constructive discussion regarding how those concerns could be addressed, in light of the "risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems" which over 1000 frontier AI company employees expressed worry about? I don't think it's justified to jump immediately to cynicism, if you haven't even made a token effort at good-faith engagement. >The moment they started adding "nucular" to the acronyms of "safety alignment" you could tell this is just kabuki at best. There's absolutely nothing that a chatbot will tell you that will get you closer to anything nuclear. It's a joke. What are you referring to? Are you referring to this quote from the OP? "The world is locked in a deadly race towards an intelligence explosion, where AI’s ability to create better AIs reaches a critical point, just like a runaway nuclear chain reaction. Going slower would give us much-needed time to make it go well, but no individual actor is willing to stop unilaterally. To survive, we must coordinate to slow down the race." Your response seems to be a total non sequitur, or failure to understand Leo's point. Exponential growth can easily get out of control, as we saw early in COVID. The point about nuclear is mainly an analogy.

watwut 3mo ago
Both OpenAI and Antropic already shown themselves to be extraordinary self centered, selfish and willing to lie to get their way. We are cynical, because it is blatantly clear these are not good faith actors. And yes, regulations they are for have an effect of restricting their competition. > I don't think it's justified to jump immediately to cynicism, if you haven't even made a token effort at good-faith engagement. I would like to see a token effort of good faith engagement from AI companies. For a change.
- 0xDEAFBEAD 3mo ago
  >Both OpenAI and Antropic already shown themselves to be extraordinary self centered, selfish and willing to lie to get their way. I buy your claim for OpenAI. I'm not sure about Anthropic. Didn't they turn down the big Pentagon contract because it violated their principles? >And yes, regulations they are for have an effect of restricting their competition. Why not just say something like "Any regulation should include a big antitrust component so it doesn't create an advantage for incumbents"? >I would like to see a token effort of good faith engagement from AI companies. For a change. What would qualify? Anthropic donated $100M to Project Glasswing to improve security in open source and critical infrastructure. Does that count for anything, or will you explain it away with cynicism somehow?
  - watwut 3mo ago
    > Didn't they turn down the big Pentagon contract because it violated their principles? They sell to Paladin. They had argument with Pentagon about, frankly, minor differences in how to use it on the edges. They are completely comfortable for majority of the uses which is bad enough. They donated to their own project? Oh, how generous. They have been pretty clear about their own goal which is to be the monopoly, the only one allowed to control the all powerful AI. Meanwhile, their idea of safety does not make me feel safe, their idea of harm reduction has shows no care about actual harms they are causing. They themselves are the threat to the safety and well being of the rest of us. And intentionally so, it is not about technology as such.
    - 0xDEAFBEAD 3mo ago
      Even if the differences are minor, the fact that Anthropic left a bunch of money on the table to maintain those differences seems interesting. >They donated to their own project? Oh, how generous. "Anthropic is committing up to $100M in usage credits for Mythos Preview across these efforts, as well as $4M in direct donations to open-source security organizations." https://www.anthropic.com/glasswing https://www.anthropic.com/glasswing An in-kind donation is still a donation. If a bread company donates bread to the homeless, you don't say "They donated products from their own factories? Oh, how generous." >They have been pretty clear about their own goal which is to be the monopoly, the only one allowed to control the all powerful AI. Meanwhile, their idea of safety does not make me feel safe, their idea of harm reduction has shows no care about actual harms they are causing. They themselves are the threat to the safety and well being of the rest of us. And intentionally so, it is not about technology as such. In that case, I think you should support some type of regulation, so a third party can step in? Your argument should be along the lines of ensuring that companies don't unduly influence the regulators, not a blanket opposition to regulation?
NitpickLawyer 3mo ago
> which over 1000 frontier AI company employees expressed worry about? Are they the same as these [1] 700 employees? > More than 700 of OpenAI’s roughly 770 employees have called on the company’s Board of Directors to resign following the removal of former chief executive Sam Altman, suggesting the Board is “incapable of overseeing” the firm, according to multiple news outlets, amid threats by the employees to leave the company and join Altman at Microsoft. They had the chance. They chose number goes up. Now, once they're on top, they want "everyone" to slow down? How is this not highly hypocritical? There's no cynicism here. It's just black and white that their interest is regulatory capture. Nothing else explains the two choices they've made along the journey. Remember, the board wanted to slow down the race. The board fired sama. The employees chose sama, fired the board, changed the structure and whatever eles they did. [1] - https://www.forbes.com/sites/tylerroush/2023/11/20/more-than-500-openai-employees-threaten-to-quit-over-sam-altmans-removal/ https://www.forbes.com/sites/tylerroush/2023/11/20/more-than...
- 0xDEAFBEAD 3mo ago
  This seems like an oversimplification. OpenAI was on top at the time sama was fired as well. Furthermore, the reason Anthropic was formed is because a bunch of employees at OpenAI didn't trust sama. And many signatories of this letter are from Anthropic. So those signatories, at least, appear to be taking a consistent position. The fact that employees across diverse AI firms signed this letter suggests that it's not an effort to achieve tactical advantage by any single AI firm.
red_green_yell 3mo ago
These 1000 employees have generational wealth riding on the ability for their companies to IPO. They know that those IPOs won't happen if open weight models continue to kill their companies' profit margins. This is an obvious conflict of interest that anyone who cares to take their head out of the sand can see. Nuclear scientists didn't have this conflict of interest. You are correct that attacking someone's motives is un-falsifiable. So is the claim that "ai is improving therefore it will keep improving therefore it will become dangerous sometime in the future. No one can prove whether these employees had angels or demons in their hearts. What the skeptics are asking is much simpler: Make a believable argument for (1) the danger of AI and (2) that regulation can be effective. Your entire argument about cynicism or unfalsifiableness is simply a distraction from having that discussion. And it's a discussion reasonable, well intentioned people can have. For example, if 1000 high school science teachers (who didn't have massive stock options in frontier labs) said 80% of our students could make a bioweapon 10x more lethal than an AR15 mass shooting with nothing more than ChatGPT and an amazon account, then that would be something reasonable people could discuss. On the other hand, 1000 employees conflicted by $Bs of options saying "trust us, this is dangerous" is not a claim that can be constructively critiqued.

watwut 3mo ago

Both OpenAI and Antropic already shown themselves to be extraordinary self centered, selfish and willing to lie to get their way. We are cynical, because it is blatantly clear these are not good faith actors. And yes, regulations they are for have an effect of restricting their competition. > I don't think it's justified to jump immediately to cynicism, if you haven't even made a token effort at good-faith engagement. I would like to see a token effort of good faith engagement from AI companies. For a change.

0xDEAFBEAD 3mo ago
>Both OpenAI and Antropic already shown themselves to be extraordinary self centered, selfish and willing to lie to get their way. I buy your claim for OpenAI. I'm not sure about Anthropic. Didn't they turn down the big Pentagon contract because it violated their principles? >And yes, regulations they are for have an effect of restricting their competition. Why not just say something like "Any regulation should include a big antitrust component so it doesn't create an advantage for incumbents"? >I would like to see a token effort of good faith engagement from AI companies. For a change. What would qualify? Anthropic donated $100M to Project Glasswing to improve security in open source and critical infrastructure. Does that count for anything, or will you explain it away with cynicism somehow?
- watwut 3mo ago
  > Didn't they turn down the big Pentagon contract because it violated their principles? They sell to Paladin. They had argument with Pentagon about, frankly, minor differences in how to use it on the edges. They are completely comfortable for majority of the uses which is bad enough. They donated to their own project? Oh, how generous. They have been pretty clear about their own goal which is to be the monopoly, the only one allowed to control the all powerful AI. Meanwhile, their idea of safety does not make me feel safe, their idea of harm reduction has shows no care about actual harms they are causing. They themselves are the threat to the safety and well being of the rest of us. And intentionally so, it is not about technology as such.
  - 0xDEAFBEAD 3mo ago
    Even if the differences are minor, the fact that Anthropic left a bunch of money on the table to maintain those differences seems interesting. >They donated to their own project? Oh, how generous. "Anthropic is committing up to $100M in usage credits for Mythos Preview across these efforts, as well as $4M in direct donations to open-source security organizations." https://www.anthropic.com/glasswing https://www.anthropic.com/glasswing An in-kind donation is still a donation. If a bread company donates bread to the homeless, you don't say "They donated products from their own factories? Oh, how generous." >They have been pretty clear about their own goal which is to be the monopoly, the only one allowed to control the all powerful AI. Meanwhile, their idea of safety does not make me feel safe, their idea of harm reduction has shows no care about actual harms they are causing. They themselves are the threat to the safety and well being of the rest of us. And intentionally so, it is not about technology as such. In that case, I think you should support some type of regulation, so a third party can step in? Your argument should be along the lines of ensuring that companies don't unduly influence the regulators, not a blanket opposition to regulation?

NitpickLawyer 3mo ago

> which over 1000 frontier AI company employees expressed worry about? Are they the same as these [1] 700 employees? > More than 700 of OpenAI’s roughly 770 employees have called on the company’s Board of Directors to resign following the removal of former chief executive Sam Altman, suggesting the Board is “incapable of overseeing” the firm, according to multiple news outlets, amid threats by the employees to leave the company and join Altman at Microsoft. They had the chance. They chose number goes up. Now, once they're on top, they want "everyone" to slow down? How is this not highly hypocritical? There's no cynicism here. It's just black and white that their interest is regulatory capture. Nothing else explains the two choices they've made along the journey. Remember, the board wanted to slow down the race. The board fired sama. The employees chose sama, fired the board, changed the structure and whatever eles they did. [1] - https://www.forbes.com/sites/tylerroush/2023/11/20/more-than-500-openai-employees-threaten-to-quit-over-sam-altmans-removal/ https://www.forbes.com/sites/tylerroush/2023/11/20/more-than...

0xDEAFBEAD 3mo ago
This seems like an oversimplification. OpenAI was on top at the time sama was fired as well. Furthermore, the reason Anthropic was formed is because a bunch of employees at OpenAI didn't trust sama. And many signatories of this letter are from Anthropic. So those signatories, at least, appear to be taking a consistent position. The fact that employees across diverse AI firms signed this letter suggests that it's not an effort to achieve tactical advantage by any single AI firm.

red_green_yell 3mo ago

These 1000 employees have generational wealth riding on the ability for their companies to IPO. They know that those IPOs won't happen if open weight models continue to kill their companies' profit margins. This is an obvious conflict of interest that anyone who cares to take their head out of the sand can see. Nuclear scientists didn't have this conflict of interest. You are correct that attacking someone's motives is un-falsifiable. So is the claim that "ai is improving therefore it will keep improving therefore it will become dangerous sometime in the future. No one can prove whether these employees had angels or demons in their hearts. What the skeptics are asking is much simpler: Make a believable argument for (1) the danger of AI and (2) that regulation can be effective. Your entire argument about cynicism or unfalsifiableness is simply a distraction from having that discussion. And it's a discussion reasonable, well intentioned people can have. For example, if 1000 high school science teachers (who didn't have massive stock options in frontier labs) said 80% of our students could make a bioweapon 10x more lethal than an AR15 mass shooting with nothing more than ChatGPT and an amazon account, then that would be something reasonable people could discuss. On the other hand, 1000 employees conflicted by $Bs of options saying "trust us, this is dangerous" is not a claim that can be constructively critiqued.