12 ms·

LLMs are a 400-year-long confidence trick

117 points by Growtika 9mo ago

mossTechnician 9mo ago
"AI safety" groups are part of what's described here: you might assume from the general "safety" label that organizations like PauseAI or ControlAI would focus things like data center pollution, the generation of sexual abuse material, causing mental harm, or many other things we can already observe. But they don't. Instead, "AI safety" organizations all appear to exclusively warn of unstoppable, apocalyptic, and unprovable harms that seem tuned exclusively to instill fear.
- ltbarcly3 9mo ago
  You are the masses. Are you afraid?
  - Xss3 9mo ago
    Hn commenters are not representative
    - ltbarcly3 9mo ago
      Everyone thinks they are special right? Thinking you are special suggests you likely aren't that special (not saying this about you personally, but still).
  - deleted 9mo ago
    [deleted]
  - das_keyboard 9mo ago
    They don't need to instill fear in everyone, but only a critical mass and most importantly _regulators_. So there will be laws because not everyone can be trusted to host and use this "dangerous", new tech. And then you have a few "trusted" big tech firms forming an oligopoly of ai, with all of the drawbacks.
    - deleted 9mo ago
      [deleted]
- iNic 9mo ago
  We should do both and it makes sense that different orgs have different focuses. It makes no sense to berate one set of orgs for not working on the exact type of thing that you want. PauseAI and ControlAI have each received less than $1 million in funding. They are both very small organizations as far as these types of advocacy non-profits go.
  - mossTechnician 9mo ago
    If it makes sense to handle all of these issues, then couldn't these organizations just acknowledge all of these issues? If reducing harm is the goal, I don't see a reason to totally segregate different issues, especially not by drawing a dividing line between the ones OpenAI already acknowledges and the ones it doesn't. I've never seen any self-described "AI safety" organizations that tackles any of the present-day issues AI companies cause.
    - iNic 9mo ago
      If you've never seen it then you haven't been paying attention. For example Anthropic (the biggest AI org which is "safety" aligned) released a big report last year on metal well being [1]. Also here is their page on societal impacts [2]. Here is PauseAI's list of risks [3], it has deepfakes as its second issue! The problem is not that no one is trying to solve the issues that you mentioned, but that it is really hard to solve them. You will probably have to bring large class action law suits, which is expensive and risky (if it fails it will be harder to sue again). Anthropic can make their own models safe, and PauseAI can organize some protests, but neither can easily stop grok from producing endless CSAM. [1] https://www.anthropic.com/news/protecting-well-being-of-users https://www.anthropic.com/news/protecting-well-being-of-user... [2] https://www.anthropic.com/research/team/societal-impacts https://www.anthropic.com/research/team/societal-impacts [3] https://pauseai.info/risks https://pauseai.info/risks
      - mossTechnician 9mo ago
        PauseAI's official proposal recommends[0]: "Only allow deployment of models after no dangerous capabilities are present." Their list of dangerous capabilities[1] does not include deepfakes, but it does include several unrealized ones that fit the description of this post here, including "a recursive loop of self-improvement, spinning rapidly out of control... called an intelligence explosion". I appreciate you pointing out the Risks page though, as it does disprove my hyperbole about ignoring present-day harms completely, although I was disheartened that the page just appears to list things that they believe actions "could be mitigated by a Pause" (emphasis mine). [0]: https://pauseai.info/proposal https://pauseai.info/proposal [1]: https://pauseai.info/dangerous-capabilities https://pauseai.info/dangerous-capabilities
- rl3 9mo ago
  It's almost like there's enough people in the world that we can focus on and tackle multiple problems at once.
- ACCount37 9mo ago
  I'd rather the "AI safety" of the kind you want didn't exist. The catastrophic AI risk isn't "oh no, people can now generate pictures of women naked".
  - mossTechnician 9mo ago
    Why would you rather it not exist? In a vacuum, I agree with you that there's probably no harm in AI-generated nudes of fictional women per se; it's the rampant use to sexually harass real women and children[0], while "causing poor air quality and decreasing life expectancy" in Tennessee[1], that bothers me. [0]: https://arstechnica.com/tech-policy/2026/01/x-blames-users-for-grok-generated-csam-no-fixes-announced/ https://arstechnica.com/tech-policy/2026/01/x-blames-users-f... [1]: https://arstechnica.com/tech-policy/2025/04/elon-musks-xai-accused-of-lying-to-black-communities-about-harmful-pollution/ https://arstechnica.com/tech-policy/2025/04/elon-musks-xai-a...
    - ACCount37 9mo ago
      Because it's just a vessel for the puritans and the usual "cares more about feeling righteous than about being right" political activists. I have no love for either. The whole thing with "AI polluting the neighborhoods" falls apart on a closer examination. Because, as it turns out, xAI put its cluster in an industrial area that already has: a defunct coal power plant, an operational steel plant, and an operational 1 GW grid-scale natural gas power plant that powers the steel plant - that one being across the road from xAI's cluster. It's quite hard for me to imagine a world where it's the AI cluster that moves the needle on local pollution.
      - muwtyhg 9mo ago
        > Because it's just a vessel for the puritans and the usual "cares more about feeling righteous than about being right" political activists. I have no love for either. People are (or were, as of a couple of weeks ago) having nude images generated from pictures of them, and were posted publicly, by a company worth billions. You think the outage around that is just puritan hand-wringing? Not that real people are actively being harassed?
ltbarcly3 9mo ago
I think anyone who thinks that LLMs are not intelligent in any sense is simply living in denial. They might not be intelligent in the same way a human is intelligent, they might make mistakes a person wouldn't make, but that's not the question. Any standard of intelligence devised before LLMs is passed by LLMs relatively easily. They do things that 10 years ago people would have said are impossible for a computer to do. I can run claude code on my laptop with an instruction like "fix the sound card on this laptop" and it will analyze what my current settings are, determine what might be wrong, devise tests to have me gather information it can't gather itself, run commands to probe hardware for it's capabilities, and finally offer a menu of solutions, give the commands to implement the solution, and finally test that the solution works perfectly. Can you do that?
- exceptione 9mo ago
  In a way LLMs are intelligence tests indeed.
- dependency_2x 9mo ago
  I'm vibe coding now, after work. I am able to much more quickly explore the landscape of a problem, get into and out of dead ends in minutes instead of wasting an evening. At some point I need to go in and fix, but the benefit of the tool is there. It is like a electric screwdriver vs. normal one. Sometimes the normal one can do things the electric can't, but hell if you get an IKEA deliver you want the electric one.
  - SwoopsFromAbove 9mo ago
    And is the electric one intelligent? :p
    - dependency_2x 9mo ago
      Who cares!
  - hexbin010 9mo ago
    Got any recent specific examples of it saving you an entire evening?
schnitzelstoat 9mo ago
I agree that all the AI doomerism is silly (by which I mean those that are concerned about some Terminator-style machine uprising, the economic issues are quite real). But it's clear the LLM's have some real value, even if we always need a human-in-the-loop to prevent hallucinations it can still massively reduce the amount of human labour required for many tasks. NFT's felt like a con, and in retrospect were a con. The LLM's are clearly useful for many things.
- latexr 9mo ago
  Those aren’t mutually exclusive; something can be both useful and a con. When a con man sells you a cheap watch for an high price, what you get is still useful—a watch that tells the time—but you were also still conned, because what you paid for is not what was advertised. You overpaid because you were tricked about what you were buying. LLMs are useful for many things, but they’re also not nearly as beneficial and powerful as they’re being sold as. Sam Altman, while entirely ignoring the societal issues raised by the technology (such as the spread of misinformation and unhealthy dependencies), repeatedly claims it will cure all cancers and other kinds of diseases, eradicate poverty, solve the housing crisis, democracy… Those are bullshit, thus the con description applies. https://youtu.be/l0K4XPu3Qhg?t=60 https://youtu.be/l0K4XPu3Qhg?t=60
  - BoxOfRain 9mo ago
    I think the following things can both be true at the same time: * LLMs are a useful tool in a variety of circumstances. * Sam Altman is personally incentivised to spout a great deal of hyped-up rubbish about both what LLMs are capable of, and can be capable of.
    - latexr 9mo ago
      Yes, that’s the point I’m making. In the scenario you’re describing, that would make Sam Altman a con man. Alternatively, he could simply be delusional and/or stupid. But given his history of deceit with Loopt and Worldcoin, there is precedent for the former.
huflungdung 9mo ago
[dead]
baq 9mo ago
"People are falling in love with LLMs" and "P(Doom) is fearmongering" so close to each other is some cognitive dissonance. The 'are LLMs intelligent?' discussion should be retired at this point, too. It's academic, the answer doesn't matter for businesses and consumers; it matters for philosophers (which everyone is even a little bit). 'Are LLMs useful for a great variety of tasks?' is a resounding 'yes'.
leogao 9mo ago
> The purpose here is not to responsibly warn us of a real threat. If that were the aim there would be a lot more shutting down of data centres and a lot less selling of nuclear-weapon-level-dangerous chatbots. you're lumping together two very different groups of people and pointing out that their beliefs are incompatible. of course they are! the people who think there is a real threat are generally different people from the ones who want to push AI progress as fast as possible! the people who say both do so generally out of a need to compromise rather than there existing many people who simultaneously hold both views.
- BoxOfRain 9mo ago
  > nuclear-weapon-level-dangerous chatbots I feel this framing in general says more about our attitudes to nuclear weapons than it does about chatbots. The 'Peace Dividend' era which is rapidly drawing to a close has made people careless when they talk about the magnitude of effects a nuclear war would have. AI can be misused, but it can't be misused to the point an enormously depopulated humanity is forced back into subsistence agriculture to survive, spending centuries if not millennia to get back to where we are now.
lyu07282 9mo ago
I think it's interesting how gamers have developed a pretty healthy aversion to generative ai in video games. Steam and Itch both now make it mandatory that games disclose generative ai use and recently even beloved Larian Studios was under fire for using ai for concept art. Gamers hate that shit. I think that's good, but the whole "AI is literally not doing anything", that it's just some mass hallucination has to die. Gamers argue it takes jobs from artists away, programmers seem to have to argue it doesn't actually do anything for some reason. Isn't that telling?
- Chance-Device 9mo ago
  I think this is probably a trend that will erode with time, even now it’s probably just moved underground. How many human artists are using AI for concepts then laundering the results? Even if it’s just idea generation, that’s a part of the process. If it speeds up throughput, then maybe that’s fewer jobs in the long run. And if AI assisted products are cheaper, and are actually good, then people will have to vote with their wallets. I think we’ve learned that people aren’t very good at doing that with causes they claim to care about once they have to actually part with their money.
  - lyu07282 9mo ago
    Because voting with your wallet is nonsense, we can decide what society we want to live in we don't have to accept one in which human artists can't make a living. Capitalism isn't a force of nature we discovered like gravity, it's deliberate choices we made.
    - Chance-Device 9mo ago
      Which I assume is why you pay someone to hand-paint scenes from your holidays instead of taking photographs? And why you employ someone to wash your clothes on a scrubbing board instead of using a machine? Or would you prefer these things be outlawed to increase employment?
      - 9mo ago
krystofee 9mo ago
I disagree with the "confidence trick" framing completely. My belief in this tech isn't based on marketing hype or someone telling me it's good – it's based on cold reality of what I'm shipping daily. The productivity gains I'm seeing right now are unprecedented. Even a year ago this wouldn't have been possible, it really feels like an inflection point. I'm seeing legitimate 10x gains because I'm not writing code anymore – I'm thinking about code and reading code. The AI facilitates both. For context: I'm maintaining a well-structured enterprise codebase (100k+ lines Django). The reality is my input is still critically valuable. My insights guide the LLM, my code review is the guardrail. The AI doesn't replace the engineer, it amplifies the intent. Using Claude Code Opus 4.5 right now and it's insane. I love it. It's like being a writer after Gutenberg invented the printing press rather than the monk copying books by hand before it.
- ManuelKiessling 9mo ago
  This. By now I don’t understand how anyone can still argue in the abstract while it’s trivial to simply give it a try and collect cold, hard facts. It’s like arguing that the piano in the room is out of tune and not bothering to walk over to the piano and hit its keys.
  - satisfice 9mo ago
    I am hitting the keys, and I call bullshit. Yes, the technology is interesting and useful. No, it is not a “10x” miracle.
    - ozim 9mo ago
      I call "AGI" or "100x miracle" a bullshit but still existing stuff definitely is "10x miracle".
      - runarberg 9mo ago
        This is a known sales trick, called door-in-the face. First you introduce your victim to an outrageous claim, and then follow with a more modest and more reasonable sounding claim. In truth neither claims are reasonable, but because of the door in the face, the victim is more susceptible the the latter claim. Without the more outrageous claim it is unlikely the victim would have believed the latter claim. In reality, both "AGI" and "100x miracle" AND the "10x miracle" are all outrageous claims, and I call bullshit on all of them.
lxgr 9mo ago
Considerations around current events aside, what exactly is the supposed "confidence trick" of mechanical or electronic calculators? They're labor-saving devices, not arbiters of truth, and as far as I can tell, they're pretty good at saving a lot of labor.
mono442 9mo ago
I don't think it's true. It is probably overhyped but it is legitimately useful. Current agents can do around 70% of coding stuff I do at work with light supervision.
- latexr 9mo ago
  > It is probably overhyped That’s exactly what a con is: selling you something as being more than what it actually is. If you agree it’s overhyped by its sellers, you agree it’s a con. > Current agents can do around 70% of coding stuff I do LLMs are being sold as capable of significantly more than coding. Focusing on that singular aspect misses the point of the article.
Traubenfuchs 9mo ago
Yeah there is overhyped marketing, but at this point, AI has revolutionized software engineering and is writing the majority of code world wide whether you like it or not and is still improving.
self_awareness 9mo ago
> If your answer doesn’t match the calculator’s, you need to redo your work. Hm... is it wrong to think like this?
falcor84 9mo ago
> We should be afraid, they say, making very public comments about “P(Doom)” - the chance the technology somehow rises up and destroys us. > This has, of course, not happened. This is so incredibly shallow. I can't think of even a single doomer, who ever claimed that AI will destroy us by now. P(doom) is about the likelihood of it destroying us "eventually". And I haven't seen anything in this post or in any recent developments to make my reduce my own p(doom), which is not close to zero. Here are some representative values: https://pauseai.info/pdoom https://pauseai.info/pdoom
- Meneth 9mo ago
  > This has, of course, not happened. And that's the anthropic fallacy. In the worlds where it has happened, the author is dead.
  - falcor84 9mo ago
    A very good point too. Though I personally hope that we'll have enough of a warning to convince people that there is a problem and give us a fighting chance. I grew up on Terminator and would be really disappointed if the AI kills me in an impersonal way.
deleted 9mo ago
[deleted]
grumbel 9mo ago
> GPT-3 was supposedly so powerful OpenAI refused to release the trained model because of “concerns about malicious applications of the technology”. [...] This has, of course, not happened. What parallel world are they living in? Every single online platform has been flooded with AI generated content and had to enact counter measures, or went the other way, embraced it and replaced humans with AI. AI use in scams has also become common place. Everything they warned about with the release of GPT‑2 did in fact happen.
petesergeant 9mo ago
Reading AI-denier articles in 2026 is almost as boring as reading crypto-booster articles was 10 years ago. You may not like LLMs, you may not want LLMs, but pretending they're not doing anything clever or useful is bizarre, however flowery you make your language.
- latexr 9mo ago
  > Reading AI-denier articles in 2026 is almost as boring as reading crypto-booster articles was 10 years ago. That’s quite a funny take, because I bet you someone will have made that same argument to criticise “crypto-deniers”. > pretending they're not doing anything clever or useful That isn’t at all the argument of the article. No one is claiming LLMs are completely useless or that they aren’t interesting technology. The critique is they’re being sold as way more than what they are or could be, that that has tangible negative consequences we can already feel, and the benefits don’t offset it.
someuser484848 9mo ago
What? The comparison to the confidence trick from 400 years ago already stops at the second point? Why call this article that way if you are not going to bring up any parallels beyond... the extremely weak link of "building trust". You have not actually made clear how mechanical calculators were a scam. Ironically, this article feels like it was written by an LLM. Just a baseless opinion.
- pancsta 9mo ago
  You missed the point. > Simply put, these companies have fallen for a confidence trick. They have built on centuries of received wisdom about the efficacy and reliability of computers, and have been drawn in by highly effective salespeople selling scarcely-believable technological wonders. Calculators are ok, but LLMs are not calculators.
  - someuser484848 9mo ago
    I see what you mean - it wasn't intended to be a parallel to mechanical calculators. However, the title implies that they were a trick - otherwise why is the "confidence trick" 400 years old? I feel like this kind of imprecise use of language is what makes it difficult to interact with LLMs in a meaningful way - perhaps that is the reason the author seems to dismiss the value of them.
xtiansimon 9mo ago
> “…LLM vendors [are responsible for the message?] We should be afraid […] The purpose here is not to responsibly warn us of a real threat. If that were the aim there would be a lot more shutting down of data centres…” Let’s not forget these innovations are on the heels of COVID. Strong, swift action by government, industry, and individuals against a deadly pathogen is “controversial”. Even if killer AI was here, twice shy… I’m angry about a lot of things right now, but LLM “marketing” (and inadequate reporting which turns to science fiction instead of science) is not one of them. The LLM revolution is getting shoehorned into this Three Card Monte narrative, and I don’t see the utility. The criticisms of LLM promise and danger is part of the zeitgeist. If firms are playing off of anything I bet it’s that, and not an industry wide conspiracy to trick the public and customers. Advertising and marketing meets people where they’re at, and “imagines” where they want to go, all wrapped up with the product. It doesn’t make the product frightening. It’s the same for all manner of dangerous technologies—guns, nuclear energy, whatever. The product is the solution to the fear. > “The LLMs we have today are famously obsequious. The phrase “you’re absolutely right!” may never again be used in earnest.” Hard NO. I get it, the language patterns of LLMs are creepy, but it’s not bad usage. So, no. I can handle the cognitive dissonance of computer algorithms spewing out anthropomorphic phrasing and not decide that I, as a human being, can no longer in humility and honesty tell someone else they’re right, and i was wrong.
stavarotti 9mo ago
Most of what I've been reading on either side of the argument is reductive. It's possible to have a take based on one's perspective and experience but it's impossible (at this time) to generalize things more broadly. I think what most people feel is multi-faceted: efficiency expectations from "leaders", job change inevitability (perceived or real), economic impact (should things not go well), loss of identity (am I a programmer, engineer, manager of things?), and several others. The discussions on the multiplicative effect of LLMs are being framed as a false dichotomy when it's far more complicated and nuanced.
josefritzishere 9mo ago
This is spot on. LLMs are a money pit putting our whole economy at risk. There is no path to profitability so the trash fire continues to burn.
Havoc 9mo ago
The confidence trick seems to be pretty good at making me tools I want
deleted 9mo ago
[deleted]
GuestFAUniverse 9mo ago
A "confidence trick" doesn't generate a working program on demand -- I just did that yesterday. In a blink of an eye. It worked. It was clean (passed linter). It wasn't very complex, yet it would have taking me several minutes to type it myself. Again: two sentence description -> enter -> working program/script. Plus: with simple feedback the program is modified accordingly (and working). While there might be open issues with AI, those AI companies are providing *far* more value than null.
motbus3 9mo ago
"You're absolutely right!" Those models are geared towards continuing the text. I have my impression that without that, the model would disagree much more as a chat/conversation
erelong 9mo ago
More like LLMs are 400-year-long project we're confident "just works"?

117 points by Growtika 9mo ago

"AI safety" groups are part of what's described here: you might assume from the general "safety" label that organizations like PauseAI or ControlAI would focus things like data center pollution, the generation of sexual abuse material, causing mental harm, or many other things we can already observe. But they don't. Instead, "AI safety" organizations all appear to exclusively warn of unstoppable, apocalyptic, and unprovable harms that seem tuned exclusively to instill fear.

ltbarcly3 9mo ago
You are the masses. Are you afraid?
- Xss3 9mo ago
  Hn commenters are not representative
  - ltbarcly3 9mo ago
    Everyone thinks they are special right? Thinking you are special suggests you likely aren't that special (not saying this about you personally, but still).
- deleted 9mo ago
  [deleted]
- das_keyboard 9mo ago
  They don't need to instill fear in everyone, but only a critical mass and most importantly _regulators_. So there will be laws because not everyone can be trusted to host and use this "dangerous", new tech. And then you have a few "trusted" big tech firms forming an oligopoly of ai, with all of the drawbacks.
  - deleted 9mo ago
    [deleted]
iNic 9mo ago
We should do both and it makes sense that different orgs have different focuses. It makes no sense to berate one set of orgs for not working on the exact type of thing that you want. PauseAI and ControlAI have each received less than $1 million in funding. They are both very small organizations as far as these types of advocacy non-profits go.
- mossTechnician 9mo ago
  If it makes sense to handle all of these issues, then couldn't these organizations just acknowledge all of these issues? If reducing harm is the goal, I don't see a reason to totally segregate different issues, especially not by drawing a dividing line between the ones OpenAI already acknowledges and the ones it doesn't. I've never seen any self-described "AI safety" organizations that tackles any of the present-day issues AI companies cause.
  - iNic 9mo ago
    If you've never seen it then you haven't been paying attention. For example Anthropic (the biggest AI org which is "safety" aligned) released a big report last year on metal well being [1]. Also here is their page on societal impacts [2]. Here is PauseAI's list of risks [3], it has deepfakes as its second issue! The problem is not that no one is trying to solve the issues that you mentioned, but that it is really hard to solve them. You will probably have to bring large class action law suits, which is expensive and risky (if it fails it will be harder to sue again). Anthropic can make their own models safe, and PauseAI can organize some protests, but neither can easily stop grok from producing endless CSAM. [1] https://www.anthropic.com/news/protecting-well-being-of-users https://www.anthropic.com/news/protecting-well-being-of-user... [2] https://www.anthropic.com/research/team/societal-impacts https://www.anthropic.com/research/team/societal-impacts [3] https://pauseai.info/risks https://pauseai.info/risks
    - mossTechnician 9mo ago
      PauseAI's official proposal recommends[0]: "Only allow deployment of models after no dangerous capabilities are present." Their list of dangerous capabilities[1] does not include deepfakes, but it does include several unrealized ones that fit the description of this post here, including "a recursive loop of self-improvement, spinning rapidly out of control... called an intelligence explosion". I appreciate you pointing out the Risks page though, as it does disprove my hyperbole about ignoring present-day harms completely, although I was disheartened that the page just appears to list things that they believe actions "could be mitigated by a Pause" (emphasis mine). [0]: https://pauseai.info/proposal https://pauseai.info/proposal [1]: https://pauseai.info/dangerous-capabilities https://pauseai.info/dangerous-capabilities
rl3 9mo ago
It's almost like there's enough people in the world that we can focus on and tackle multiple problems at once.
ACCount37 9mo ago
I'd rather the "AI safety" of the kind you want didn't exist. The catastrophic AI risk isn't "oh no, people can now generate pictures of women naked".
- mossTechnician 9mo ago
  Why would you rather it not exist? In a vacuum, I agree with you that there's probably no harm in AI-generated nudes of fictional women per se; it's the rampant use to sexually harass real women and children[0], while "causing poor air quality and decreasing life expectancy" in Tennessee[1], that bothers me. [0]: https://arstechnica.com/tech-policy/2026/01/x-blames-users-for-grok-generated-csam-no-fixes-announced/ https://arstechnica.com/tech-policy/2026/01/x-blames-users-f... [1]: https://arstechnica.com/tech-policy/2025/04/elon-musks-xai-accused-of-lying-to-black-communities-about-harmful-pollution/ https://arstechnica.com/tech-policy/2025/04/elon-musks-xai-a...
  - ACCount37 9mo ago
    Because it's just a vessel for the puritans and the usual "cares more about feeling righteous than about being right" political activists. I have no love for either. The whole thing with "AI polluting the neighborhoods" falls apart on a closer examination. Because, as it turns out, xAI put its cluster in an industrial area that already has: a defunct coal power plant, an operational steel plant, and an operational 1 GW grid-scale natural gas power plant that powers the steel plant - that one being across the road from xAI's cluster. It's quite hard for me to imagine a world where it's the AI cluster that moves the needle on local pollution.
    - muwtyhg 9mo ago
      > Because it's just a vessel for the puritans and the usual "cares more about feeling righteous than about being right" political activists. I have no love for either. People are (or were, as of a couple of weeks ago) having nude images generated from pictures of them, and were posted publicly, by a company worth billions. You think the outage around that is just puritan hand-wringing? Not that real people are actively being harassed?

ltbarcly3 9mo ago

I think anyone who thinks that LLMs are not intelligent in any sense is simply living in denial. They might not be intelligent in the same way a human is intelligent, they might make mistakes a person wouldn't make, but that's not the question. Any standard of intelligence devised before LLMs is passed by LLMs relatively easily. They do things that 10 years ago people would have said are impossible for a computer to do. I can run claude code on my laptop with an instruction like "fix the sound card on this laptop" and it will analyze what my current settings are, determine what might be wrong, devise tests to have me gather information it can't gather itself, run commands to probe hardware for it's capabilities, and finally offer a menu of solutions, give the commands to implement the solution, and finally test that the solution works perfectly. Can you do that?

exceptione 9mo ago
In a way LLMs are intelligence tests indeed.
dependency_2x 9mo ago
I'm vibe coding now, after work. I am able to much more quickly explore the landscape of a problem, get into and out of dead ends in minutes instead of wasting an evening. At some point I need to go in and fix, but the benefit of the tool is there. It is like a electric screwdriver vs. normal one. Sometimes the normal one can do things the electric can't, but hell if you get an IKEA deliver you want the electric one.
- SwoopsFromAbove 9mo ago
  And is the electric one intelligent? :p
  - dependency_2x 9mo ago
    Who cares!
- hexbin010 9mo ago
  Got any recent specific examples of it saving you an entire evening?

schnitzelstoat 9mo ago

I agree that all the AI doomerism is silly (by which I mean those that are concerned about some Terminator-style machine uprising, the economic issues are quite real). But it's clear the LLM's have some real value, even if we always need a human-in-the-loop to prevent hallucinations it can still massively reduce the amount of human labour required for many tasks. NFT's felt like a con, and in retrospect were a con. The LLM's are clearly useful for many things.

latexr 9mo ago
Those aren’t mutually exclusive; something can be both useful and a con. When a con man sells you a cheap watch for an high price, what you get is still useful—a watch that tells the time—but you were also still conned, because what you paid for is not what was advertised. You overpaid because you were tricked about what you were buying. LLMs are useful for many things, but they’re also not nearly as beneficial and powerful as they’re being sold as. Sam Altman, while entirely ignoring the societal issues raised by the technology (such as the spread of misinformation and unhealthy dependencies), repeatedly claims it will cure all cancers and other kinds of diseases, eradicate poverty, solve the housing crisis, democracy… Those are bullshit, thus the con description applies. https://youtu.be/l0K4XPu3Qhg?t=60 https://youtu.be/l0K4XPu3Qhg?t=60
- BoxOfRain 9mo ago
  I think the following things can both be true at the same time: * LLMs are a useful tool in a variety of circumstances. * Sam Altman is personally incentivised to spout a great deal of hyped-up rubbish about both what LLMs are capable of, and can be capable of.
  - latexr 9mo ago
    Yes, that’s the point I’m making. In the scenario you’re describing, that would make Sam Altman a con man. Alternatively, he could simply be delusional and/or stupid. But given his history of deceit with Loopt and Worldcoin, there is precedent for the former.

huflungdung 9mo ago

[dead]

baq 9mo ago

"People are falling in love with LLMs" and "P(Doom) is fearmongering" so close to each other is some cognitive dissonance. The 'are LLMs intelligent?' discussion should be retired at this point, too. It's academic, the answer doesn't matter for businesses and consumers; it matters for philosophers (which everyone is even a little bit). 'Are LLMs useful for a great variety of tasks?' is a resounding 'yes'.

leogao 9mo ago

> The purpose here is not to responsibly warn us of a real threat. If that were the aim there would be a lot more shutting down of data centres and a lot less selling of nuclear-weapon-level-dangerous chatbots. you're lumping together two very different groups of people and pointing out that their beliefs are incompatible. of course they are! the people who think there is a real threat are generally different people from the ones who want to push AI progress as fast as possible! the people who say both do so generally out of a need to compromise rather than there existing many people who simultaneously hold both views.

BoxOfRain 9mo ago
> nuclear-weapon-level-dangerous chatbots I feel this framing in general says more about our attitudes to nuclear weapons than it does about chatbots. The 'Peace Dividend' era which is rapidly drawing to a close has made people careless when they talk about the magnitude of effects a nuclear war would have. AI can be misused, but it can't be misused to the point an enormously depopulated humanity is forced back into subsistence agriculture to survive, spending centuries if not millennia to get back to where we are now.

lyu07282 9mo ago

I think it's interesting how gamers have developed a pretty healthy aversion to generative ai in video games. Steam and Itch both now make it mandatory that games disclose generative ai use and recently even beloved Larian Studios was under fire for using ai for concept art. Gamers hate that shit. I think that's good, but the whole "AI is literally not doing anything", that it's just some mass hallucination has to die. Gamers argue it takes jobs from artists away, programmers seem to have to argue it doesn't actually do anything for some reason. Isn't that telling?

Chance-Device 9mo ago
I think this is probably a trend that will erode with time, even now it’s probably just moved underground. How many human artists are using AI for concepts then laundering the results? Even if it’s just idea generation, that’s a part of the process. If it speeds up throughput, then maybe that’s fewer jobs in the long run. And if AI assisted products are cheaper, and are actually good, then people will have to vote with their wallets. I think we’ve learned that people aren’t very good at doing that with causes they claim to care about once they have to actually part with their money.
- lyu07282 9mo ago
  Because voting with your wallet is nonsense, we can decide what society we want to live in we don't have to accept one in which human artists can't make a living. Capitalism isn't a force of nature we discovered like gravity, it's deliberate choices we made.
  - Chance-Device 9mo ago
    Which I assume is why you pay someone to hand-paint scenes from your holidays instead of taking photographs? And why you employ someone to wash your clothes on a scrubbing board instead of using a machine? Or would you prefer these things be outlawed to increase employment?
    - 9mo ago

krystofee 9mo ago

I disagree with the "confidence trick" framing completely. My belief in this tech isn't based on marketing hype or someone telling me it's good – it's based on cold reality of what I'm shipping daily. The productivity gains I'm seeing right now are unprecedented. Even a year ago this wouldn't have been possible, it really feels like an inflection point. I'm seeing legitimate 10x gains because I'm not writing code anymore – I'm thinking about code and reading code. The AI facilitates both. For context: I'm maintaining a well-structured enterprise codebase (100k+ lines Django). The reality is my input is still critically valuable. My insights guide the LLM, my code review is the guardrail. The AI doesn't replace the engineer, it amplifies the intent. Using Claude Code Opus 4.5 right now and it's insane. I love it. It's like being a writer after Gutenberg invented the printing press rather than the monk copying books by hand before it.

ManuelKiessling 9mo ago
This. By now I don’t understand how anyone can still argue in the abstract while it’s trivial to simply give it a try and collect cold, hard facts. It’s like arguing that the piano in the room is out of tune and not bothering to walk over to the piano and hit its keys.
- satisfice 9mo ago
  I am hitting the keys, and I call bullshit. Yes, the technology is interesting and useful. No, it is not a “10x” miracle.
  - ozim 9mo ago
    I call "AGI" or "100x miracle" a bullshit but still existing stuff definitely is "10x miracle".
    - runarberg 9mo ago
      This is a known sales trick, called door-in-the face. First you introduce your victim to an outrageous claim, and then follow with a more modest and more reasonable sounding claim. In truth neither claims are reasonable, but because of the door in the face, the victim is more susceptible the the latter claim. Without the more outrageous claim it is unlikely the victim would have believed the latter claim. In reality, both "AGI" and "100x miracle" AND the "10x miracle" are all outrageous claims, and I call bullshit on all of them.

lxgr 9mo ago

Considerations around current events aside, what exactly is the supposed "confidence trick" of mechanical or electronic calculators? They're labor-saving devices, not arbiters of truth, and as far as I can tell, they're pretty good at saving a lot of labor.

mono442 9mo ago

I don't think it's true. It is probably overhyped but it is legitimately useful. Current agents can do around 70% of coding stuff I do at work with light supervision.

latexr 9mo ago
> It is probably overhyped That’s exactly what a con is: selling you something as being more than what it actually is. If you agree it’s overhyped by its sellers, you agree it’s a con. > Current agents can do around 70% of coding stuff I do LLMs are being sold as capable of significantly more than coding. Focusing on that singular aspect misses the point of the article.

Traubenfuchs 9mo ago

Yeah there is overhyped marketing, but at this point, AI has revolutionized software engineering and is writing the majority of code world wide whether you like it or not and is still improving.

self_awareness 9mo ago

> If your answer doesn’t match the calculator’s, you need to redo your work. Hm... is it wrong to think like this?

falcor84 9mo ago

> We should be afraid, they say, making very public comments about “P(Doom)” - the chance the technology somehow rises up and destroys us. > This has, of course, not happened. This is so incredibly shallow. I can't think of even a single doomer, who ever claimed that AI will destroy us by now. P(doom) is about the likelihood of it destroying us "eventually". And I haven't seen anything in this post or in any recent developments to make my reduce my own p(doom), which is not close to zero. Here are some representative values: https://pauseai.info/pdoom https://pauseai.info/pdoom

Meneth 9mo ago
> This has, of course, not happened. And that's the anthropic fallacy. In the worlds where it has happened, the author is dead.
- falcor84 9mo ago
  A very good point too. Though I personally hope that we'll have enough of a warning to convince people that there is a problem and give us a fighting chance. I grew up on Terminator and would be really disappointed if the AI kills me in an impersonal way.

deleted 9mo ago

[deleted]

grumbel 9mo ago

> GPT-3 was supposedly so powerful OpenAI refused to release the trained model because of “concerns about malicious applications of the technology”. [...] This has, of course, not happened. What parallel world are they living in? Every single online platform has been flooded with AI generated content and had to enact counter measures, or went the other way, embraced it and replaced humans with AI. AI use in scams has also become common place. Everything they warned about with the release of GPT‑2 did in fact happen.

petesergeant 9mo ago

Reading AI-denier articles in 2026 is almost as boring as reading crypto-booster articles was 10 years ago. You may not like LLMs, you may not want LLMs, but pretending they're not doing anything clever or useful is bizarre, however flowery you make your language.

latexr 9mo ago
> Reading AI-denier articles in 2026 is almost as boring as reading crypto-booster articles was 10 years ago. That’s quite a funny take, because I bet you someone will have made that same argument to criticise “crypto-deniers”. > pretending they're not doing anything clever or useful That isn’t at all the argument of the article. No one is claiming LLMs are completely useless or that they aren’t interesting technology. The critique is they’re being sold as way more than what they are or could be, that that has tangible negative consequences we can already feel, and the benefits don’t offset it.

someuser484848 9mo ago

What? The comparison to the confidence trick from 400 years ago already stops at the second point? Why call this article that way if you are not going to bring up any parallels beyond... the extremely weak link of "building trust". You have not actually made clear how mechanical calculators were a scam. Ironically, this article feels like it was written by an LLM. Just a baseless opinion.

pancsta 9mo ago
You missed the point. > Simply put, these companies have fallen for a confidence trick. They have built on centuries of received wisdom about the efficacy and reliability of computers, and have been drawn in by highly effective salespeople selling scarcely-believable technological wonders. Calculators are ok, but LLMs are not calculators.
- someuser484848 9mo ago
  I see what you mean - it wasn't intended to be a parallel to mechanical calculators. However, the title implies that they were a trick - otherwise why is the "confidence trick" 400 years old? I feel like this kind of imprecise use of language is what makes it difficult to interact with LLMs in a meaningful way - perhaps that is the reason the author seems to dismiss the value of them.

xtiansimon 9mo ago

> “…LLM vendors [are responsible for the message?] We should be afraid […] The purpose here is not to responsibly warn us of a real threat. If that were the aim there would be a lot more shutting down of data centres…” Let’s not forget these innovations are on the heels of COVID. Strong, swift action by government, industry, and individuals against a deadly pathogen is “controversial”. Even if killer AI was here, twice shy… I’m angry about a lot of things right now, but LLM “marketing” (and inadequate reporting which turns to science fiction instead of science) is not one of them. The LLM revolution is getting shoehorned into this Three Card Monte narrative, and I don’t see the utility. The criticisms of LLM promise and danger is part of the zeitgeist. If firms are playing off of anything I bet it’s that, and not an industry wide conspiracy to trick the public and customers. Advertising and marketing meets people where they’re at, and “imagines” where they want to go, all wrapped up with the product. It doesn’t make the product frightening. It’s the same for all manner of dangerous technologies—guns, nuclear energy, whatever. The product is the solution to the fear. > “The LLMs we have today are famously obsequious. The phrase “you’re absolutely right!” may never again be used in earnest.” Hard NO. I get it, the language patterns of LLMs are creepy, but it’s not bad usage. So, no. I can handle the cognitive dissonance of computer algorithms spewing out anthropomorphic phrasing and not decide that I, as a human being, can no longer in humility and honesty tell someone else they’re right, and i was wrong.

stavarotti 9mo ago

Most of what I've been reading on either side of the argument is reductive. It's possible to have a take based on one's perspective and experience but it's impossible (at this time) to generalize things more broadly. I think what most people feel is multi-faceted: efficiency expectations from "leaders", job change inevitability (perceived or real), economic impact (should things not go well), loss of identity (am I a programmer, engineer, manager of things?), and several others. The discussions on the multiplicative effect of LLMs are being framed as a false dichotomy when it's far more complicated and nuanced.

josefritzishere 9mo ago

This is spot on. LLMs are a money pit putting our whole economy at risk. There is no path to profitability so the trash fire continues to burn.

Havoc 9mo ago

The confidence trick seems to be pretty good at making me tools I want

deleted 9mo ago

[deleted]

GuestFAUniverse 9mo ago

A "confidence trick" doesn't generate a working program on demand -- I just did that yesterday. In a blink of an eye. It worked. It was clean (passed linter). It wasn't very complex, yet it would have taking me several minutes to type it myself. Again: two sentence description -> enter -> working program/script. Plus: with simple feedback the program is modified accordingly (and working). While there might be open issues with AI, those AI companies are providing *far* more value than null.

motbus3 9mo ago

"You're absolutely right!" Those models are geared towards continuing the text. I have my impression that without that, the model would disagree much more as a chat/conversation

erelong 9mo ago

More like LLMs are 400-year-long project we're confident "just works"?