6 ms·

You can call your little doggy "AI" if it makes you happy. But when you call something "AI" and it tells you to walk instead of drive to the car wash, you're n

You can call your little doggy "AI" if it makes you happy. But when you call something "AI" and it tells you to walk instead of drive to the car wash, you're not talking about the "AI" science fiction authors were dreaming of.

ACCount37 3mo ago
Sure, just keep moving the goalposts. It's not a "real AI" because it can't take over the US military command and kick off WW3 and finish the survivors off with killer robots yet!
- _doctor_love 3mo ago
  That's how it works though. The moment we have "AI" and see something working, it immediately ceases to be magic because "it's just a program after all." Aligning on a true definition of Artificial Intelligence is a very vexing problem.
  - jacobgold 3mo ago
    We could've slapped a chat interface on calculators and called them "AI" because they can do superhuman math instantly. Most technical people would've thought that was stupid. LLMs are the same kind of category mistake.
    - _doctor_love 3mo ago
      Funny enough, calculators went through this exact same thing when they came out. "If the calculator can do math for the students, will they still learn?"
      - jacobgold 3mo ago
        We didn't have confused people claiming calculators were human-like intelligences doing math.
        _doctor_love 3mo ago
        Sorry I don't follow what argument you're making?
        pixl97 3mo ago
        They seem to be making a religious argument at this point. They don't like that the AI is a very poorly defined term and they are mad as hell about it to the point of irrational forum posting.
        jacobgold 3mo ago
        https://en.wikipedia.org/wiki/Ad_hominem https://en.wikipedia.org/wiki/Ad_hominem
        Jensson 3mo ago
        We did have that, people thought computers would overtake humans very soon when computers got better than humans at such things. It happens every single time computers do a new thing that previously humans were better at. Then 10 years later people see, oh that is just calculations, of course computers are better at that.
    - CamperBob2 3mo ago
      Let's talk about category mistakes. You've been here since 2007, according to your other reply. You understand that calculators have as much to do with mathematics as telescopes have to do with cosmology. Right? If someone unskilled at math brings a calculator to an international math competition, they will not succeed at solving many problems. Most likely, they will solve none at all. But if they bring a frontier LLM (and succeed at concealing it from the organizers), they can walk away with a gold medal. Such a feat requires intelligence... and if the contestant didn't provide the intelligence himself/herself, where'd it come from? That means that analogies involving calculators are completely useless when the topic is AI. Calculators are not, and can never be, intelligent. LLMs are nothing even remotely like calculators.
      - jacobgold 3mo ago
        > If someone unskilled at math brings a calculator to an international math competition, they will not succeed at solving many problems. > But if they bring a frontier LLM (and succeed at concealing it from the organizers), they can walk away with a gold medal. Of course you could win all kinds of math competitions with a concealed calculator. Maybe you'd need a fancy one, like a little SBC running Python. Anything complex and timed would be easy to win. You'd look like a genius to anyone who didn't know you had it. > Such a feat requires intelligence... and if the contestant didn't provide the intelligence himself/herself, where'd it come from? From computer software running on computer hardware, just like a calculator. Calculating trillions of digits of pi also requires intelligence far beyond human capacity. Computers displaying intelligence doesn't imply human-like intelligence. This is the source of confusion.
        CamperBob2 3mo ago
        Of course you could win all kinds of math competitions with a concealed calculator. Maybe you'd need a fancy one, like a little SBC running Python. My mistake.
        jacobgold 3mo ago
        Taking issue with using a computer to power the calculator? If so, you should know that all modern calculators are computers under the hood. Everything I've written about calculators applies to computers doing any kind of traditional deterministic processing, without anything like LLMs.
- dofm 3mo ago
  It can however select a girls school as a military target, and did.
  - dTal 3mo ago
    ...did it really, though?
    - dofm 3mo ago
      Yeah, it did. I don't think this is really in doubt. Especially since the military refuse to confirm it had any human oversight, which after all would have been a routine thing to talk about before AI targeted things — which is why we have the phrases "fog of war", "human error", "unfortunate mistake", etc. It will take a while to shake out — we won't know for sure for a decade, I suspect, but it seems very likely this will prove to be an AI error.
      - dTal 3mo ago
        Looks more like standard decision-washing finger pointing to me. "AI did it" is both hypey and also conveniently distracts from an uglier reality: >Palantir Technologies [built] Maven into a targeting infrastructure that pulls together satellite imagery, signals intelligence and sensor data to identify targets and carry them through every step from first detection to the order to strike. >The building in Minab had been classified as a military facility in a Defense Intelligence Agency database that, according to CNN, had not been updated to reflect that the building had been separated from the adjacent Islamic Revolutionary Guard Corps compound and converted into a school, a change that satellite imagery shows had occurred by 2016 at the latest. A chatbot did not kill those children. People failed to update a database, and other people built a system fast enough to make that failure lethal. https://www.theguardian.com/news/2026/mar/26/ai-got-the-blame-for-the-iran-school-bombing-the-truth-is-far-more-worrying https://www.theguardian.com/news/2026/mar/26/ai-got-the-blam...
        dofm 3mo ago
        I am in no way absolving the people who set up the AI-powered tool that apparently acted without their oversight. But they set it up, they let it choose, and they didn’t countermand it. So the AI did, effectively, make the decision. And the suppliers of AI technology to these people should be on the hook as a result.
TeMPOraL 3mo ago
That's a bad and tired example as it confuses people just as much as it does (or rather, did?) LLMs.
- jacobgold 3mo ago
  It's one example that points out a major (possibly fundamental) flaw. I can point to prompt injection as another example. There are tons more if you're interested. Are you actually claiming LLMs operate based on human-like intelligence?
  - ACCount37 3mo ago
    We're on Hacker News. Do I really have to point out the existence of social engineering to you? Or that scamming old people out of their life savings is a profitable enough activity that there are entire call centers dedicated to the task? Humans keep overestimating just how high the bar of "human-like intelligence" is.
    - jacobgold 3mo ago
      Drawing the conclusion that "humans fail" and "models fail", so they must be similar, is very wrong. You could have humans calculate 2+2 all day and get a surprisingly high error rate. That reveals a flaw in how humans operate. LLMs fail for entirely different reasons. Their mistakes don't imply they're human-like at all. It's not about the error rate.
      - ACCount37 3mo ago
        You're saying that a class of mistakes points out a "major (possibly fundamental) flaw". I'm pointing out some very similar classes of mistakes in humans - well known, well documented and widely exploited. They just keep paying the "IRS" in gift cards, buying lottery tickets and getting the captain's age wrong. If you're using the existence of flaws in LLMs to deny the claim of intelligence to them, then why do "generally intelligent" humans exhibit some impressively similar-looking flaws? And, if we're talking about that conspicuous similarity - do they actually fail "for entirely different reasons"? Or do you just want the reasons to be "entirely different" - and not the same reasons viewed at a different angle? Because the similarities between humans falling for trick questions or scams, and LLMs falling for adversarial questions or prompt injections don't look coincidental to me at all. One of the oldest patterns in scamming is overwhelming and confusing the victim. Numerous prompt injection methods seek to overwhelm and confuse an LLM - if an LLM can't keep track of things, can't grasp what's going on, it's far more likely to lose track of what's a prompt and what's data, overlook past instructions or go past its behavioral guardrails. And humans who fall for trick questions like "1kg of feathers" or "captain's age" due to shallow attention and naive pattern matching? They fail in surprisingly similar ways to how LLMs fail on SimpleBench tasks that are filled with overwhelming adversarial distractors. Many "trick questions" are tricky to humans and LLMs alike - to the point that it's unlikely to be coincidental.
CamperBob2 3mo ago
But when you call something "AI" and it tells you to walk instead of drive to the car wash There are so many other sites. So many others. Why are you here?
- deleted 3mo ago
  [deleted]
- ahartmetz 3mo ago
  Is this website called AI Faithful News?
- jacobgold 3mo ago
  > There are so many other sites. So many others. Why are you here? I've been here since 2007 when HN launched. You're confused about my objection. I don't like the term "AI" but I love the technology as much as almost anyone.
orangecat 3mo ago
You can call your little doggy "AI" if it makes you happy. Or you can keep calling them stochastic parrots as they solve decades-old open problems. The real question is how useful they are, and the answer "not at all" increasingly requires flat-earth levels of denial. it tells you to walk instead of drive to the car wash, you're not talking about the "AI" science fiction authors were dreaming of They sort of are. Think of Data from Star Trek TNG failing to understand figures of speech. Not that it's terribly relevant; humans regularly fall for tricks like "Paris in the the spring" or "where do you bury the survivors".
- jacobgold 3mo ago
  > Or you can keep calling them stochastic parrots as they solve decades-old open problems. I didn't use that phrase at all. But computers calculated digits of π to trillions of digits. With a chat interface for a Python math program would look like the most impressive math genius if you took it back a few decades. > The real question is how useful they are... That's not the "real question" but an entirely different question that is easily answered. Nothing I wrote suggested they're not incredibly useful. > Data from Star Trek TNG failing to understand figures of speech. These are just little instances of bad writing. Data is very much an attempt at displaying a human-like intelligence.
  - Kim_Bruning 3mo ago
    > That's not the "real question" but an entirely different question that is easily answered. Nothing I wrote suggested they're not incredibly useful. Oh, ok then. That does change things a bit. The impression I'm getting is that you were suggesting they're not. What's succinctly the thing you're objecting to? Is it Anthropomorphization? I mean, sure, but watch out : when defending on that axis, it's easy to slip into Anthropodenial, right? Frans de Waal (from the same science that invented "Don't Anthropomorphize" ) can tell you about it. > With a chat interface for a Python math program would look like the most impressive math genius if you took it back a few decades. Well, exactly. Whether any particular generation of AI or software is yes/no "Like A Human Being" is probably the least interesting question axis. It's all just anthropocentrism. Is that the thing you're trying to lay your finger on?

ACCount37 3mo ago

Sure, just keep moving the goalposts. It's not a "real AI" because it can't take over the US military command and kick off WW3 and finish the survivors off with killer robots yet!

_doctor_love 3mo ago
That's how it works though. The moment we have "AI" and see something working, it immediately ceases to be magic because "it's just a program after all." Aligning on a true definition of Artificial Intelligence is a very vexing problem.
- jacobgold 3mo ago
  We could've slapped a chat interface on calculators and called them "AI" because they can do superhuman math instantly. Most technical people would've thought that was stupid. LLMs are the same kind of category mistake.
  - _doctor_love 3mo ago
    Funny enough, calculators went through this exact same thing when they came out. "If the calculator can do math for the students, will they still learn?"
    - jacobgold 3mo ago
      We didn't have confused people claiming calculators were human-like intelligences doing math.
      - _doctor_love 3mo ago
        Sorry I don't follow what argument you're making?
        pixl97 3mo ago
        They seem to be making a religious argument at this point. They don't like that the AI is a very poorly defined term and they are mad as hell about it to the point of irrational forum posting.
        jacobgold 3mo ago
        https://en.wikipedia.org/wiki/Ad_hominem https://en.wikipedia.org/wiki/Ad_hominem
      - Jensson 3mo ago
        We did have that, people thought computers would overtake humans very soon when computers got better than humans at such things. It happens every single time computers do a new thing that previously humans were better at. Then 10 years later people see, oh that is just calculations, of course computers are better at that.
  - CamperBob2 3mo ago
    Let's talk about category mistakes. You've been here since 2007, according to your other reply. You understand that calculators have as much to do with mathematics as telescopes have to do with cosmology. Right? If someone unskilled at math brings a calculator to an international math competition, they will not succeed at solving many problems. Most likely, they will solve none at all. But if they bring a frontier LLM (and succeed at concealing it from the organizers), they can walk away with a gold medal. Such a feat requires intelligence... and if the contestant didn't provide the intelligence himself/herself, where'd it come from? That means that analogies involving calculators are completely useless when the topic is AI. Calculators are not, and can never be, intelligent. LLMs are nothing even remotely like calculators.
    - jacobgold 3mo ago
      > If someone unskilled at math brings a calculator to an international math competition, they will not succeed at solving many problems. > But if they bring a frontier LLM (and succeed at concealing it from the organizers), they can walk away with a gold medal. Of course you could win all kinds of math competitions with a concealed calculator. Maybe you'd need a fancy one, like a little SBC running Python. Anything complex and timed would be easy to win. You'd look like a genius to anyone who didn't know you had it. > Such a feat requires intelligence... and if the contestant didn't provide the intelligence himself/herself, where'd it come from? From computer software running on computer hardware, just like a calculator. Calculating trillions of digits of pi also requires intelligence far beyond human capacity. Computers displaying intelligence doesn't imply human-like intelligence. This is the source of confusion.
      - CamperBob2 3mo ago
        Of course you could win all kinds of math competitions with a concealed calculator. Maybe you'd need a fancy one, like a little SBC running Python. My mistake.
        jacobgold 3mo ago
        Taking issue with using a computer to power the calculator? If so, you should know that all modern calculators are computers under the hood. Everything I've written about calculators applies to computers doing any kind of traditional deterministic processing, without anything like LLMs.
dofm 3mo ago
It can however select a girls school as a military target, and did.
- dTal 3mo ago
  ...did it really, though?
  - dofm 3mo ago
    Yeah, it did. I don't think this is really in doubt. Especially since the military refuse to confirm it had any human oversight, which after all would have been a routine thing to talk about before AI targeted things — which is why we have the phrases "fog of war", "human error", "unfortunate mistake", etc. It will take a while to shake out — we won't know for sure for a decade, I suspect, but it seems very likely this will prove to be an AI error.
    - dTal 3mo ago
      Looks more like standard decision-washing finger pointing to me. "AI did it" is both hypey and also conveniently distracts from an uglier reality: >Palantir Technologies [built] Maven into a targeting infrastructure that pulls together satellite imagery, signals intelligence and sensor data to identify targets and carry them through every step from first detection to the order to strike. >The building in Minab had been classified as a military facility in a Defense Intelligence Agency database that, according to CNN, had not been updated to reflect that the building had been separated from the adjacent Islamic Revolutionary Guard Corps compound and converted into a school, a change that satellite imagery shows had occurred by 2016 at the latest. A chatbot did not kill those children. People failed to update a database, and other people built a system fast enough to make that failure lethal. https://www.theguardian.com/news/2026/mar/26/ai-got-the-blame-for-the-iran-school-bombing-the-truth-is-far-more-worrying https://www.theguardian.com/news/2026/mar/26/ai-got-the-blam...
      - dofm 3mo ago
        I am in no way absolving the people who set up the AI-powered tool that apparently acted without their oversight. But they set it up, they let it choose, and they didn’t countermand it. So the AI did, effectively, make the decision. And the suppliers of AI technology to these people should be on the hook as a result.

TeMPOraL 3mo ago

That's a bad and tired example as it confuses people just as much as it does (or rather, did?) LLMs.

jacobgold 3mo ago
It's one example that points out a major (possibly fundamental) flaw. I can point to prompt injection as another example. There are tons more if you're interested. Are you actually claiming LLMs operate based on human-like intelligence?
- ACCount37 3mo ago
  We're on Hacker News. Do I really have to point out the existence of social engineering to you? Or that scamming old people out of their life savings is a profitable enough activity that there are entire call centers dedicated to the task? Humans keep overestimating just how high the bar of "human-like intelligence" is.
  - jacobgold 3mo ago
    Drawing the conclusion that "humans fail" and "models fail", so they must be similar, is very wrong. You could have humans calculate 2+2 all day and get a surprisingly high error rate. That reveals a flaw in how humans operate. LLMs fail for entirely different reasons. Their mistakes don't imply they're human-like at all. It's not about the error rate.
    - ACCount37 3mo ago
      You're saying that a class of mistakes points out a "major (possibly fundamental) flaw". I'm pointing out some very similar classes of mistakes in humans - well known, well documented and widely exploited. They just keep paying the "IRS" in gift cards, buying lottery tickets and getting the captain's age wrong. If you're using the existence of flaws in LLMs to deny the claim of intelligence to them, then why do "generally intelligent" humans exhibit some impressively similar-looking flaws? And, if we're talking about that conspicuous similarity - do they actually fail "for entirely different reasons"? Or do you just want the reasons to be "entirely different" - and not the same reasons viewed at a different angle? Because the similarities between humans falling for trick questions or scams, and LLMs falling for adversarial questions or prompt injections don't look coincidental to me at all. One of the oldest patterns in scamming is overwhelming and confusing the victim. Numerous prompt injection methods seek to overwhelm and confuse an LLM - if an LLM can't keep track of things, can't grasp what's going on, it's far more likely to lose track of what's a prompt and what's data, overlook past instructions or go past its behavioral guardrails. And humans who fall for trick questions like "1kg of feathers" or "captain's age" due to shallow attention and naive pattern matching? They fail in surprisingly similar ways to how LLMs fail on SimpleBench tasks that are filled with overwhelming adversarial distractors. Many "trick questions" are tricky to humans and LLMs alike - to the point that it's unlikely to be coincidental.

CamperBob2 3mo ago

But when you call something "AI" and it tells you to walk instead of drive to the car wash There are so many other sites. So many others. Why are you here?

deleted 3mo ago
[deleted]
ahartmetz 3mo ago
Is this website called AI Faithful News?
jacobgold 3mo ago
> There are so many other sites. So many others. Why are you here? I've been here since 2007 when HN launched. You're confused about my objection. I don't like the term "AI" but I love the technology as much as almost anyone.

orangecat 3mo ago

You can call your little doggy "AI" if it makes you happy. Or you can keep calling them stochastic parrots as they solve decades-old open problems. The real question is how useful they are, and the answer "not at all" increasingly requires flat-earth levels of denial. it tells you to walk instead of drive to the car wash, you're not talking about the "AI" science fiction authors were dreaming of They sort of are. Think of Data from Star Trek TNG failing to understand figures of speech. Not that it's terribly relevant; humans regularly fall for tricks like "Paris in the the spring" or "where do you bury the survivors".

jacobgold 3mo ago
> Or you can keep calling them stochastic parrots as they solve decades-old open problems. I didn't use that phrase at all. But computers calculated digits of π to trillions of digits. With a chat interface for a Python math program would look like the most impressive math genius if you took it back a few decades. > The real question is how useful they are... That's not the "real question" but an entirely different question that is easily answered. Nothing I wrote suggested they're not incredibly useful. > Data from Star Trek TNG failing to understand figures of speech. These are just little instances of bad writing. Data is very much an attempt at displaying a human-like intelligence.
- Kim_Bruning 3mo ago
  > That's not the "real question" but an entirely different question that is easily answered. Nothing I wrote suggested they're not incredibly useful. Oh, ok then. That does change things a bit. The impression I'm getting is that you were suggesting they're not. What's succinctly the thing you're objecting to? Is it Anthropomorphization? I mean, sure, but watch out : when defending on that axis, it's easy to slip into Anthropodenial, right? Frans de Waal (from the same science that invented "Don't Anthropomorphize" ) can tell you about it. > With a chat interface for a Python math program would look like the most impressive math genius if you took it back a few decades. Well, exactly. Whether any particular generation of AI or software is yes/no "Like A Human Being" is probably the least interesting question axis. It's all just anthropocentrism. Is that the thing you're trying to lay your finger on?