4 ms·

Basically all critiques of Anthropic's policy moves on these topics boil down to people not believing the fundamental concerns are real, and often then going a

Basically all critiques of Anthropic's policy moves on these topics boil down to people not believing the fundamental concerns are real, and often then going a step further to conclude that Anthropic doesn't actually believe their concerns either. If you believe Anthropic believes what they say they do, all of it makes sense.

shimman 5mo ago
What are you referring to? The cult belief that they are ushering in a machine god or that they strictly care about making as much money as humanely possibly while ignoring the absolutely destructive impacts these companies have had on society? IMO they are using the cult messaging to distract the public so they take out all the oxygen in the room regarding people that care about the immediate impacts (climate exacerbation, ease of scamming, degrading job prospects, increasing income inequality). Whenever real concerns are brought up against these companies they are always ignored while claiming the real concern is the fantasy of a machine god turning into skynet.
- estearum 5mo ago
  "Why don't they just not participate in the arms race?!" - guy who's never heard of arms races If they believe they're creating "a machine god" and that it's better it's their machine god than someone else's (which, given the other contenders, I tend to agree with), then all the corollaries you mention are mostly irrelevant. Whether you believe they're creating a machine god is irrelevant. They believe that they are. It would be helpful if you could create an actually good argument for why they cannot or are not creating a machine god, but it turns out there are no good arguments for why it's impossible to do so. And so... they shall try.
  - shimman 5mo ago
    Oh okay, they're all just legit crazy and are allowed to poison the environment, murder teenagers, and ruin the material lives of millions for fantasy level delusions. Good to know.
  - paulhebert 5mo ago
    A lot of people would prefer nuclear deproliferation over building more nukes. Arms races always work out great for arms dealers. Less so for the average Joe.
    - estearum 5mo ago
      Yes, totally. But those people's preferences don't matter. What matters is the core competitive dynamic.
  - Dylan16807 5mo ago
    Sometimes governments have to deal with the weapons made by their enemies and that gets them stuck in an arms race. Companies don't have to do that. If they're getting into actually dangerous territory, they can stop as soon as they want to.
    - estearum 5mo ago
      If you believe in the AI doom scenario then yes, you do need to do that. Because it's very important that your "less ethical" and "less good" competitors do not get to the machine god first. If you don't believe that, or you don't believe that the frontier labs believe that, then sure, it makes no sense. But they probably do. The people at these companies literally dedicated their lives to building this specific thing that, up until people had to make tradeoffs between "that looks risky" and "that looks useful", virtually everyone agreed would be a dangerous technology. What apparently many people on HN failed to appreciate is that the thing that makes it dangerous is the fact that it grows in utility.
    - FrustratedMonky 5mo ago
      Joking? Companies absolutely do get into arms races.
      - Dylan16807 5mo ago
        They do but they very very very don't have to. They can stop.
        estearum 5mo ago
        In reality there is basically no mechanism by which a large company (i.e. with a board) can do this so long as there is 1) significant profit motive and 2) no government intervention.
jcgrillo 5mo ago
But the things they say they believe are insane and totally unmoored from physical, societal, and economic reality. If they actually believe those things they're untrustworthy because they're delusional. If they don't, they're untrustworthy because they're fraudulent. Either way it's not good..
- reducesuffering 5mo ago
  They're not. They're in the eye of the storm and see what's going on the clearest. They were ahead of the curve to be where they're at now, and they're still ahead of the curve for where we're going. All the other heads of labs like Sam Altman and Demis have been saying the same thing since 2015-2016 way before any of this "marketing" would ever have been at play.
  - jcgrillo 5mo ago
    There's a simpler explanation that fits the data better: they're lying. Generally, in the past when tech companies have made outlandish claims that were not backed by evidence, they're later found out to have lied. This is an ancient pattern going back to the dotcom era and before, but for recent examples you need only look back a few years to the web3 era. If they're not lying, they can show it by producing the results they claim. Until then, they're probably just lying.
    - estearum 5mo ago
      What data does "they're lying" fit better than "they're earnest?" > If they're not lying, they can show it by producing the results they claim. Until then, they're probably just lying Brilliant framework: Anyone making claims about the future is not just speculating, not just wrong, but they are lying.
      - jcgrillo 5mo ago
        [flagged]
caconym_ 5mo ago
Even if you believe the concerns have merit, it's hard not to be cynical about people (e.g. Anthropic leadership) paying lip service to those concerns while so obviously leveraging their power and wealth (which depend, by the way, on accelerating the world toward those hypothetical "concerning" scenarios as fast as possible) to position themselves such that they will become unimaginably richer if things go their way, and will also come out on top pretty much no matter what happens. It's like a prisoner's dilemma where one party is loudly lecturing the other about the obvious benefits of cooperation while also obviously working on defecting. They want to have their cake and eat it too. Maybe they really are the pure-of-heart Chosen Ones destined to lead us around the great filter, but I don't see why I should believe that's the case when their behavior is just as easily explained as maneuvering toward being the winner who takes it all.
- estearum 5mo ago
  > (which depend, by the way, on accelerating the world toward those hypothetical "concerning" scenarios as fast as possible) Yes, this dynamic is exactly the one that anyone who's concerned about AI is concerned about. I don't know why you state this as if it's evidence against the concerns lol. Someone being concerned about the incentives of a situation doesn't de facto make them immune to those incentives, obviously. The implication that someone who's concerned about an arms race dynamic could simply opt out of the system that produces that dynamic is simply confused about what arms race dynamics are. The entire point is that it's a trap, and it's a trap even if you know it's a trap, and even if you don't like that it's a trap. There's nothing dishonest or hypocritical about being in the trap: it is literally a trap –– that is what it does and why it is bad! I'm confused by these comments that imply people believe Dario et al are "pure-of-heart Chosen Ones destined to lead us around the great filter." Who? I've never seen it. And any AI-doomer is probably of the opinion that the entire question of Dario's or anyone else's personal moral character is 99% irrelevant. Because, again, it's a trap. The dynamics at play are so much larger than whether someone irks people for their lecturing tone. I would much rather give my money to Dario, who seems like a generally good person, versus Sama, who seems like a complete snake, but I'm under no illusions that doing so changes the fundamental dynamics that are steering us to AI doom. I doubt anyone does. And yes, obviously they are angling toward being the winner who takes it all. That is literally the trap. If you believe what they believe, yelling "let's cooperate!" while hurdling towards the finish line and tripping your competitors is the only reasonable thing to do. That is the problem.

shimman 5mo ago

What are you referring to? The cult belief that they are ushering in a machine god or that they strictly care about making as much money as humanely possibly while ignoring the absolutely destructive impacts these companies have had on society? IMO they are using the cult messaging to distract the public so they take out all the oxygen in the room regarding people that care about the immediate impacts (climate exacerbation, ease of scamming, degrading job prospects, increasing income inequality). Whenever real concerns are brought up against these companies they are always ignored while claiming the real concern is the fantasy of a machine god turning into skynet.

estearum 5mo ago
"Why don't they just not participate in the arms race?!" - guy who's never heard of arms races If they believe they're creating "a machine god" and that it's better it's their machine god than someone else's (which, given the other contenders, I tend to agree with), then all the corollaries you mention are mostly irrelevant. Whether you believe they're creating a machine god is irrelevant. They believe that they are. It would be helpful if you could create an actually good argument for why they cannot or are not creating a machine god, but it turns out there are no good arguments for why it's impossible to do so. And so... they shall try.
- shimman 5mo ago
  Oh okay, they're all just legit crazy and are allowed to poison the environment, murder teenagers, and ruin the material lives of millions for fantasy level delusions. Good to know.
- paulhebert 5mo ago
  A lot of people would prefer nuclear deproliferation over building more nukes. Arms races always work out great for arms dealers. Less so for the average Joe.
  - estearum 5mo ago
    Yes, totally. But those people's preferences don't matter. What matters is the core competitive dynamic.
- Dylan16807 5mo ago
  Sometimes governments have to deal with the weapons made by their enemies and that gets them stuck in an arms race. Companies don't have to do that. If they're getting into actually dangerous territory, they can stop as soon as they want to.
  - estearum 5mo ago
    If you believe in the AI doom scenario then yes, you do need to do that. Because it's very important that your "less ethical" and "less good" competitors do not get to the machine god first. If you don't believe that, or you don't believe that the frontier labs believe that, then sure, it makes no sense. But they probably do. The people at these companies literally dedicated their lives to building this specific thing that, up until people had to make tradeoffs between "that looks risky" and "that looks useful", virtually everyone agreed would be a dangerous technology. What apparently many people on HN failed to appreciate is that the thing that makes it dangerous is the fact that it grows in utility.
  - FrustratedMonky 5mo ago
    Joking? Companies absolutely do get into arms races.
    - Dylan16807 5mo ago
      They do but they very very very don't have to. They can stop.
      - estearum 5mo ago
        In reality there is basically no mechanism by which a large company (i.e. with a board) can do this so long as there is 1) significant profit motive and 2) no government intervention.

jcgrillo 5mo ago

But the things they say they believe are insane and totally unmoored from physical, societal, and economic reality. If they actually believe those things they're untrustworthy because they're delusional. If they don't, they're untrustworthy because they're fraudulent. Either way it's not good..

reducesuffering 5mo ago
They're not. They're in the eye of the storm and see what's going on the clearest. They were ahead of the curve to be where they're at now, and they're still ahead of the curve for where we're going. All the other heads of labs like Sam Altman and Demis have been saying the same thing since 2015-2016 way before any of this "marketing" would ever have been at play.
- jcgrillo 5mo ago
  There's a simpler explanation that fits the data better: they're lying. Generally, in the past when tech companies have made outlandish claims that were not backed by evidence, they're later found out to have lied. This is an ancient pattern going back to the dotcom era and before, but for recent examples you need only look back a few years to the web3 era. If they're not lying, they can show it by producing the results they claim. Until then, they're probably just lying.
  - estearum 5mo ago
    What data does "they're lying" fit better than "they're earnest?" > If they're not lying, they can show it by producing the results they claim. Until then, they're probably just lying Brilliant framework: Anyone making claims about the future is not just speculating, not just wrong, but they are lying.
    - jcgrillo 5mo ago
      [flagged]

caconym_ 5mo ago

Even if you believe the concerns have merit, it's hard not to be cynical about people (e.g. Anthropic leadership) paying lip service to those concerns while so obviously leveraging their power and wealth (which depend, by the way, on accelerating the world toward those hypothetical "concerning" scenarios as fast as possible) to position themselves such that they will become unimaginably richer if things go their way, and will also come out on top pretty much no matter what happens. It's like a prisoner's dilemma where one party is loudly lecturing the other about the obvious benefits of cooperation while also obviously working on defecting. They want to have their cake and eat it too. Maybe they really are the pure-of-heart Chosen Ones destined to lead us around the great filter, but I don't see why I should believe that's the case when their behavior is just as easily explained as maneuvering toward being the winner who takes it all.

estearum 5mo ago
> (which depend, by the way, on accelerating the world toward those hypothetical "concerning" scenarios as fast as possible) Yes, this dynamic is exactly the one that anyone who's concerned about AI is concerned about. I don't know why you state this as if it's evidence against the concerns lol. Someone being concerned about the incentives of a situation doesn't de facto make them immune to those incentives, obviously. The implication that someone who's concerned about an arms race dynamic could simply opt out of the system that produces that dynamic is simply confused about what arms race dynamics are. The entire point is that it's a trap, and it's a trap even if you know it's a trap, and even if you don't like that it's a trap. There's nothing dishonest or hypocritical about being in the trap: it is literally a trap –– that is what it does and why it is bad! I'm confused by these comments that imply people believe Dario et al are "pure-of-heart Chosen Ones destined to lead us around the great filter." Who? I've never seen it. And any AI-doomer is probably of the opinion that the entire question of Dario's or anyone else's personal moral character is 99% irrelevant. Because, again, it's a trap. The dynamics at play are so much larger than whether someone irks people for their lecturing tone. I would much rather give my money to Dario, who seems like a generally good person, versus Sama, who seems like a complete snake, but I'm under no illusions that doing so changes the fundamental dynamics that are steering us to AI doom. I doubt anyone does. And yes, obviously they are angling toward being the winner who takes it all. That is literally the trap. If you believe what they believe, yelling "let's cooperate!" while hurdling towards the finish line and tripping your competitors is the only reasonable thing to do. That is the problem.