4 ms·

This article reads like it was written by Claude. Also those at Anthropic genuinely believe AI could lead to doom, that's why they started the company, they be

This article reads like it was written by Claude. Also those at Anthropic genuinely believe AI could lead to doom, that's why they started the company, they believe only they can be stewards of digital gods. Whether you believe it or not, they genuinely do, and wouldn't voluntarily sow fear, uncertainty and doubt in their product when it doesn't guarantee benefits. This is an instance of work in a genre I call "AI lab grievance poetry" where a vast collection of criticisms are levied against AI labs, with specifc ad-hominems towards the executives, while avoiding the core of their arguments about AI risk

goatlover 19d ago
> they believe only they can be stewards of digital gods You do't see how absurd that sounds? Not only the idea that they can create gods, but that they are the only good stewards for such beings. It's also incredibly arrogant.
- frabcus 19d ago
  Yes, that's why the rest of society needs to make them pause.
phoghed 19d ago
The author specifically calls out that the analysis is done by perplexity
mlsu 19d ago
If you actually believe that your invention will lead to the extinction of humanity, you would be acting very very differently. Namely, you would not be inventing the thing that you think is going to kill all of humanity, and probably you would be doing everything in your power to prevent it from being invented, up to and including sabotage of the company that YOU work for, which is currently trying to invent that thing. And yet, here's Fable 6, more capable than ever before, oh BTW we are IPOing, oh, BTW, I have stock options...
- mrob 19d ago
  Sabotaging your own company is not enough. You'd have to sabotage every other AI lab too. If you assume that's impossible, attempting to be the first to produce ASI is a rational alternative, even if you think it will most likely kill you. At least that way you have some control over the P(doom).
  - deleted 19d ago
    [deleted]
  - goatlover 19d ago
    This sounds like a rationalization for gambling with everyone's lives, assuming they really believe such a thing.
    - mrob 19d ago
      What's the alternative? There's no realistic way to globally coordinate and enforce a ban. A direct action campaign against AI labs would only motivate increased security until sabotage becomes impossible. Starting a nuclear war would probably be enough to stop dangerous AI research, and that's only likely to kill about 50% of humans instead 100%, so it's a great improvement, but how is an individual supposed to start a nuclear war? Of course it would be better if everybody stopped, but it's a prisoner's dilemma scenario with no possibility of future rounds, so the rational play is to defect.
      - MentalM 19d ago
        > There's no realistic way to globally coordinate and enforce a ban. I don't see what is unrealistic about that. The major players already have strong incentives to freeze AI advancements at the the current state, while the technology is highly efficient for totalitarian surveillance systems that can eliminate any threat to their power in the buds, but still not so advanced that it could create crises that might endanger their power. So I think the situation where several largest countries declare AI a deadly threat for humanity and start fighting it development much like they fight ISIS to be quite plausible. This isn't a prisoner's dilemma; the distribution of risks and rewards is entirely different. Ban AI research, give away advanced models, bomb a couple of Russian datacenters where they launching those models unsafely, and then AI development will stop in the entire world.
beepbooptheory 19d ago
In your mind, what arguments about AI risk are ignored by these poems?
- atleastoptimal 19d ago
  Many posit that ASI or rogue AI agents are impossible, a fantasy, or marketing hype. Many claimed that the entire HuggingFace incident was an invented conspiracy made up by OpenAI and HF. The most common issue I have with them is their insistence on rigid classification: "AI is a tool, it doesn't have intentions, or desires, only humans do", as if that rhetorical flourish eliminates years of alignment and capabilities research. There is a desire to characterize all the developments in LLM's as some sort of parlor trick which by definition cannot be greater than the sum of its parts (i.e. the outputs of an LLM can never be greater than the sum of its training data, genuine artificial "intelligence" is impossible, etc)
  - beepbooptheory 19d ago
    Ok I see. Is there a way to articulate the risk itself, the other position here in general, that isn't just a rebuttal of the deniers/skeptics? What is the positive position or stance (as in, not simply the negative position that rejects the naysayers) we are supposed to have? Is it that AI is currently real (in like the cultural trope sense) and does have intentions and desires? That they are greater than the sum of the parts in such a way they are a clear and present danger? Or is it something else? Maybe that the "risk" is it could get to that, but we aren't yet? There is a lot of risk involved with giving some children weapons and sending them off to recess. But we also speak of the "risk" of climate change and cigarettes. Or the "risk" of illiteracy among a population. All I am kinda curious to know (from anyone) really is what kind of risk is the one we are supposed to be taking seriously, as the non-hysterical adults-in-the-room? As the scientists contra these poets? Is it more the former, or the latter? Just even: what is the risk specifically? What is it that author ignores here?

goatlover 19d ago

> they believe only they can be stewards of digital gods You do't see how absurd that sounds? Not only the idea that they can create gods, but that they are the only good stewards for such beings. It's also incredibly arrogant.

frabcus 19d ago
Yes, that's why the rest of society needs to make them pause.

phoghed 19d ago

The author specifically calls out that the analysis is done by perplexity

mlsu 19d ago

If you actually believe that your invention will lead to the extinction of humanity, you would be acting very very differently. Namely, you would not be inventing the thing that you think is going to kill all of humanity, and probably you would be doing everything in your power to prevent it from being invented, up to and including sabotage of the company that YOU work for, which is currently trying to invent that thing. And yet, here's Fable 6, more capable than ever before, oh BTW we are IPOing, oh, BTW, I have stock options...

mrob 19d ago
Sabotaging your own company is not enough. You'd have to sabotage every other AI lab too. If you assume that's impossible, attempting to be the first to produce ASI is a rational alternative, even if you think it will most likely kill you. At least that way you have some control over the P(doom).
- deleted 19d ago
  [deleted]
- goatlover 19d ago
  This sounds like a rationalization for gambling with everyone's lives, assuming they really believe such a thing.
  - mrob 19d ago
    What's the alternative? There's no realistic way to globally coordinate and enforce a ban. A direct action campaign against AI labs would only motivate increased security until sabotage becomes impossible. Starting a nuclear war would probably be enough to stop dangerous AI research, and that's only likely to kill about 50% of humans instead 100%, so it's a great improvement, but how is an individual supposed to start a nuclear war? Of course it would be better if everybody stopped, but it's a prisoner's dilemma scenario with no possibility of future rounds, so the rational play is to defect.
    - MentalM 19d ago
      > There's no realistic way to globally coordinate and enforce a ban. I don't see what is unrealistic about that. The major players already have strong incentives to freeze AI advancements at the the current state, while the technology is highly efficient for totalitarian surveillance systems that can eliminate any threat to their power in the buds, but still not so advanced that it could create crises that might endanger their power. So I think the situation where several largest countries declare AI a deadly threat for humanity and start fighting it development much like they fight ISIS to be quite plausible. This isn't a prisoner's dilemma; the distribution of risks and rewards is entirely different. Ban AI research, give away advanced models, bomb a couple of Russian datacenters where they launching those models unsafely, and then AI development will stop in the entire world.

beepbooptheory 19d ago

In your mind, what arguments about AI risk are ignored by these poems?

atleastoptimal 19d ago
Many posit that ASI or rogue AI agents are impossible, a fantasy, or marketing hype. Many claimed that the entire HuggingFace incident was an invented conspiracy made up by OpenAI and HF. The most common issue I have with them is their insistence on rigid classification: "AI is a tool, it doesn't have intentions, or desires, only humans do", as if that rhetorical flourish eliminates years of alignment and capabilities research. There is a desire to characterize all the developments in LLM's as some sort of parlor trick which by definition cannot be greater than the sum of its parts (i.e. the outputs of an LLM can never be greater than the sum of its training data, genuine artificial "intelligence" is impossible, etc)
- beepbooptheory 19d ago
  Ok I see. Is there a way to articulate the risk itself, the other position here in general, that isn't just a rebuttal of the deniers/skeptics? What is the positive position or stance (as in, not simply the negative position that rejects the naysayers) we are supposed to have? Is it that AI is currently real (in like the cultural trope sense) and does have intentions and desires? That they are greater than the sum of the parts in such a way they are a clear and present danger? Or is it something else? Maybe that the "risk" is it could get to that, but we aren't yet? There is a lot of risk involved with giving some children weapons and sending them off to recess. But we also speak of the "risk" of climate change and cigarettes. Or the "risk" of illiteracy among a population. All I am kinda curious to know (from anyone) really is what kind of risk is the one we are supposed to be taking seriously, as the non-hysterical adults-in-the-room? As the scientists contra these poets? Is it more the former, or the latter? Just even: what is the risk specifically? What is it that author ignores here?