4 ms·

"are priced according to the narrative that they have produced or will in the very near future produce a fully automated drop-in replacement for most knowledge

"are priced according to the narrative that they have produced or will in the very near future produce a fully automated drop-in replacement for most knowledge workers, " No, they're really not. They're priced in a way that would imply AI will be universal form of compute, alongside traditional deterministic systems - which it will be. And that they will capture most of that ... which they won't. The Frontier Labs are a very bad buy at a high price, but that partly has to do with wacky pricing, but actually mostly has to do with their relatively weak place in the value chain. The money is going to Nvidia, who have the most powerful position. A bit like how a retailer can take all the margins of some innovative product, if they own the channel. AI is over-hyped, the Frontier Labs are over priced - but AI is here to stay, and will grow. Not like Skynet, but like a new form of compute. And it will take it's time, and the profits will be reaped by those with the power.

lukewarm707 16d ago
ai has a >10% chance of causing human extinction, according to anthropic big heads. if that's true, you are wrong. if that's false, anthropic is dishonest. why trust a dishonest company to be worth anything?
- bluegatty 16d ago
  I think that the AI people believe in their own nonsense a bit. Like - the guy on TV talking about 'AI will destroy everything' ... I don't think he's lying. I think they are like we here on HN and Reddit and a bit caught up in our own thoughts. If AI were unleashed, in raw form today, it could cause havoc. Bad. Maybe very bad but I think we'd get over it. It would probably trigger a recession (because we are in a bubble - it would pop it), and people would 'blame the AI' for sure. But it would be a bit dot-com ish kind of recession. The amplifiers would be geopolitical instability.
  - krapp 16d ago
    >If AI were unleashed, in raw form today, it could cause havoc. What is "raw form?"
    - bluegatty 16d ago
      The SOTA models are heavily 'guardrailed' today; they won't let you do all sorts of things.
    - AndrewKemendo 16d ago
      My exact question and afaict since I’m running models locally and can inspect and retrain etc..I assume I already have the raw form?
      - bluegatty 15d ago
        You are not running SOTA models locally, the gap is material. At least for now ... but by next year, the models we can run locally might cross that 'risk' threshold.
- nl 16d ago
  > if that's false, anthropic is dishonest. why trust a dishonest company to be worth anything? This logic doesn't follow at all. If their argument is that there is 10% chance of extinction then they also believe there is a 90% chance it won't.
  - lukewarm707 15d ago
    recalling the exact phrasing, several senior people at anthropic made public statements agreeing a 10% chance of causing extinction in less than 10 years. 10% is uninsurable, priced in with ordinary treatment of risk it suggests that anthropic should be worth zero today. creating that risk would put every executive in jail. on top of that it would demand under existing laws of conflict, a military campaign to destroy anthropic. that is not optional, it is demanded now to save lives. hard to make comparisons but we mourned and rembered 9/11 recently. a 10% risk of hundreds of millions dead in 10 years would make anthropic a thousands of times greater threat than al qaeda. many countries would assassinate dario amodei and the leadership of anthropic now, within weeks or months. actually just on the vague risk of having a nuclear weapon in 10 years, the USA killed ayatollah khamenei, his daughter, his son-in-law, his daughter-in-law and his 14 month old granddaughter. then, they killed over 120 children ages 6-12 by accidentally bombing a school. in the sense that i would analyse a company, at least, the claim is false. it's not true that ai has a 10% chance of causing human extinction within 10 years. they are making false claims about the technology they sell. i have a fairly inflexible approach to that. sure, exaggerate but outright lies about the nature of the product don't work for me.
    - nl 15d ago
      > it's not true that ai has a 10% chance of causing human extinction within 10 years. I agree with this. The rest of it I don't. For external actions to be taken there needs to be a consensus and there clearly isn't that. And this sort of thing happens all the time. For example Zuckerburg thought Facebook was worth more than the $1B Yahoo offered him and he was right even though no one else agreed. He though the metaverse was worth the $20B+ they spent on it and he was wrong. There's a gap between people thinking something within a company and people external to the company agreeing with it.
webern777 15d ago
They are pricing as a first mover AGI monopoly that makes no sense. I think the core mistake is this partial-equilibrium reasoning. Take the new technology and then hold everything else fixed. $40 trillion of knowledge work routed unchanged through a new toll booth. Profit. This has nothing to do with reality. Nvidia on the other hand does have the CUDA monopoly so their toll booth is printing money but that will get routed around or broken at some point.
Zambyte 15d ago
Traditional non-chaotic systems* LLMs are deterministic. They are chaotic, which people confuse for non-deterministic.
- sp527 15d ago
  > They are chaotic, which people confuse for non-deterministic. That's an odd argument, because a lot of people who have struggled to decipher complex chaotic systems would tell you this is a distinction without much of a difference.
  - gmm1990 15d ago
    Yeah if you give the llm the exact same input over and over you really shouldn’t be using an llm.
  - Zambyte 15d ago
    It's at least important from philosophical perspective regarding whether LLMs have free will or not. It's hard to say whether or not a human thought or action is driven by free will, people have debated it for centuries. We can provide the same input to an LLM and get the same output, because they are deterministic. Surely it is obvious that a pure function does not have free will, no matter how expensive it is to compute.
- SkyBelow 15d ago
  I used that description some months ago and I think you are the first person I see who put it the same way. The only gotcha with this is that they are theoretically deterministic, but rarely in practice. A few examples: - Harness specific settings that user can't control (anything from timestamp to prng seeding. - Batching requests in a way that leads to a single request being processed different depending upon the batch (say MoE where your first choice expert is assigned to someone else's token so you go to your second choice vs a batch where you get your first choice). - Graphics card itself carrying out floating point arithmetic in slightly different orders leading to floating point non associativity causing different outputs. But all of these can be controlled for (at some cost) and the model can be ran deterministically. For the average user, it might as well be non-deterministic, but when considering theoretical capabilities, chaotic deterministic system seems the better description.

lukewarm707 16d ago

ai has a >10% chance of causing human extinction, according to anthropic big heads. if that's true, you are wrong. if that's false, anthropic is dishonest. why trust a dishonest company to be worth anything?

bluegatty 16d ago
I think that the AI people believe in their own nonsense a bit. Like - the guy on TV talking about 'AI will destroy everything' ... I don't think he's lying. I think they are like we here on HN and Reddit and a bit caught up in our own thoughts. If AI were unleashed, in raw form today, it could cause havoc. Bad. Maybe very bad but I think we'd get over it. It would probably trigger a recession (because we are in a bubble - it would pop it), and people would 'blame the AI' for sure. But it would be a bit dot-com ish kind of recession. The amplifiers would be geopolitical instability.
- krapp 16d ago
  >If AI were unleashed, in raw form today, it could cause havoc. What is "raw form?"
  - bluegatty 16d ago
    The SOTA models are heavily 'guardrailed' today; they won't let you do all sorts of things.
  - AndrewKemendo 16d ago
    My exact question and afaict since I’m running models locally and can inspect and retrain etc..I assume I already have the raw form?
    - bluegatty 15d ago
      You are not running SOTA models locally, the gap is material. At least for now ... but by next year, the models we can run locally might cross that 'risk' threshold.
nl 16d ago
> if that's false, anthropic is dishonest. why trust a dishonest company to be worth anything? This logic doesn't follow at all. If their argument is that there is 10% chance of extinction then they also believe there is a 90% chance it won't.
- lukewarm707 15d ago
  recalling the exact phrasing, several senior people at anthropic made public statements agreeing a 10% chance of causing extinction in less than 10 years. 10% is uninsurable, priced in with ordinary treatment of risk it suggests that anthropic should be worth zero today. creating that risk would put every executive in jail. on top of that it would demand under existing laws of conflict, a military campaign to destroy anthropic. that is not optional, it is demanded now to save lives. hard to make comparisons but we mourned and rembered 9/11 recently. a 10% risk of hundreds of millions dead in 10 years would make anthropic a thousands of times greater threat than al qaeda. many countries would assassinate dario amodei and the leadership of anthropic now, within weeks or months. actually just on the vague risk of having a nuclear weapon in 10 years, the USA killed ayatollah khamenei, his daughter, his son-in-law, his daughter-in-law and his 14 month old granddaughter. then, they killed over 120 children ages 6-12 by accidentally bombing a school. in the sense that i would analyse a company, at least, the claim is false. it's not true that ai has a 10% chance of causing human extinction within 10 years. they are making false claims about the technology they sell. i have a fairly inflexible approach to that. sure, exaggerate but outright lies about the nature of the product don't work for me.
  - nl 15d ago
    > it's not true that ai has a 10% chance of causing human extinction within 10 years. I agree with this. The rest of it I don't. For external actions to be taken there needs to be a consensus and there clearly isn't that. And this sort of thing happens all the time. For example Zuckerburg thought Facebook was worth more than the $1B Yahoo offered him and he was right even though no one else agreed. He though the metaverse was worth the $20B+ they spent on it and he was wrong. There's a gap between people thinking something within a company and people external to the company agreeing with it.

webern777 15d ago

They are pricing as a first mover AGI monopoly that makes no sense. I think the core mistake is this partial-equilibrium reasoning. Take the new technology and then hold everything else fixed. $40 trillion of knowledge work routed unchanged through a new toll booth. Profit. This has nothing to do with reality. Nvidia on the other hand does have the CUDA monopoly so their toll booth is printing money but that will get routed around or broken at some point.

Zambyte 15d ago

Traditional non-chaotic systems* LLMs are deterministic. They are chaotic, which people confuse for non-deterministic.

sp527 15d ago
> They are chaotic, which people confuse for non-deterministic. That's an odd argument, because a lot of people who have struggled to decipher complex chaotic systems would tell you this is a distinction without much of a difference.
- gmm1990 15d ago
  Yeah if you give the llm the exact same input over and over you really shouldn’t be using an llm.
- Zambyte 15d ago
  It's at least important from philosophical perspective regarding whether LLMs have free will or not. It's hard to say whether or not a human thought or action is driven by free will, people have debated it for centuries. We can provide the same input to an LLM and get the same output, because they are deterministic. Surely it is obvious that a pure function does not have free will, no matter how expensive it is to compute.
SkyBelow 15d ago
I used that description some months ago and I think you are the first person I see who put it the same way. The only gotcha with this is that they are theoretically deterministic, but rarely in practice. A few examples: - Harness specific settings that user can't control (anything from timestamp to prng seeding. - Batching requests in a way that leads to a single request being processed different depending upon the batch (say MoE where your first choice expert is assigned to someone else's token so you go to your second choice vs a batch where you get your first choice). - Graphics card itself carrying out floating point arithmetic in slightly different orders leading to floating point non associativity causing different outputs. But all of these can be controlled for (at some cost) and the model can be ran deterministically. For the average user, it might as well be non-deterministic, but when considering theoretical capabilities, chaotic deterministic system seems the better description.