3 ms·

All of this is accurate (and useful). But yeah, psychics do not prove theorems from frontier math and build working complex software.

deleted 6d ago
[deleted]
igortg 6d ago
On one side I agree that LLMs and Agents are not intelligent, but they present an illusion of intelligence given the shear amount of data they can process and act upon. But that cannot be used to discredit the fact that these are incredibly powerful tools that can get out of control and cause great damage.
- atemerev 6d ago
  What is "intelligence" then. I think the universalism of LLMs is now evident enough that they can be considered "intelligent", even in slightly different realization compared to humans. As of "great damage", I doubt it. They do not have self-preservation instinct (all "worrying" experiments are the attempts to initiate something resembling self-preservation from human initiative). The driving part is external - the query loop can always be turned off. So yes, a dangerous tool that can be exploited by humans (including governments, especially governments - which is why I am skeptical to government regulation proposals, particularly looking at what passes as governments in this era). But there are no inherent dangers from their own agency, as there is none.
  - alansaber 6d ago
    "Slightly different"- no, a completely different realisation of intelligence to humans. LLMs can definitely do cool stuff though.
    - pixl97 6d ago
      >completely different realisation of intelligence to humans. OK, so they are intelligent. I'm glad we agree. The problem with this word is people associate intelligence with the process they realize is occurring inside their head which is only a tiny fraction of what intelligence encompasses. Humans (without study) know very little about intelligence and they very gray boundaries it has.
  - chrisjj 5d ago
    > As of "great damage", I doubt it. They do not have self-preservation instinct The potential for great damage relies upon the lack of self-preservation instinct. That lack releases the so-called AI from an order under pain of death. Damage cannot be punished.
cmiles74 6d ago
In my opinion, this article is talking about people who get into long conversations and become convinced the model is “intelligent”, maybe even “AGI”. This seems to be a trap people are falling into, even in 2026.
- atemerev 6d ago
  Initial definitions of AGI (as of 2014) are long surpassed. Then, AGI was defined as something that is general enough to work in many fields and orient themselves. It did not include being smarter than humans or even having comparable intelligence to humans. By original standards, we have AGI already for some time. Current AGI definitions are intentionally vague.
- krupan 6d ago
  It's definitely talking about that, but it's also talking about all of us: - fawning over how amazing these tools are - believing everything OpenAI and Anthropic say about how powerful and dangerous their product is - minimizing the amount of human effort and involvement in every "AI" achievement
Madmallard 6d ago
Neither does LLMs. Not without a monumental amount of human oversight and expertise. Everyone not in a particular field asking LLM about said field is rolling bad dice.
- atemerev 6d ago
  I have published a LLM-assisted proof of a long-standing problem in convex geometry (70 years) while not being a specialist in convex geometry. I am a scientist and I have education and scientific experience in computer science and systems biology, but I was never a mathematician. So, well, you can definitely work across fields at least.
  - tripledry 6d ago
    Do you have a link? I would also be very interested in seeing the prompts, from what I've seen you still need to understand the field, maybe I'm wrong.
krupan 6d ago
Neither do LLMs. Humans using them do. Similar to how cops solve crimes using psychics.
- pixl97 6d ago
  Your statement is unclear, break it down a little further on how you think this works.
  - krupan 6d ago
    The original article already attempted to do that but I'll take another stab. Humans tend to believe what they want to believe. They gain confidence when a confident person (or LLM) they work with exudes confidence. Where they might have hesitated or got stuck in analysis paralysis alone, the aid of a con (short for confidence) artist will get them moving. They then attribute the solution and praise for it solely to the confident person (or LLM) that nudged them along. Did the "psychic" who understands human motivation better than the cops actually help? Very likely. Did the psychic end up getting far more credit than they deserve (it's magical! You'll never believe! Well worth the money I spent!)? Absolutely. The amount of real help vs. confidence/manipulation the LLM or con artist provides varies in each situation and scenario and mix of people (and LLMs) involved. It's something to strive to be aware and analytical about.
    - pixl97 6d ago
      There is also something else horizontally to this that occurs. There are numerous scientific discoveries that have been made because one human looked at all available data and turned off the assumptions other people in the field have been using for years, maybe hundreds of years. Once you delete the assumption and make a new one the answer is obvious any anyone from that point wonders how so many humans could have missed it for so much time. This can occur readily with LLMs as it can with people. It's very likely we will see this a lot as the causal connection in the available data will connected differently in their minds. Escaping from a local maximum can be very difficult as you have to climb uphill with a nearly infinite amount of freedom but no ability to see the horizon. Psychics, LLMs, or some guy name Bob walking in an providing a workable analogy for you to escape the local maxima and move closer to a global maxima are all the same. Moreso, if LLMs are AGI we should expect these "psychic" behaviors just as much as actual discovery because both continuums exist in the same problem space as human minds.
_superposition_ 6d ago
No they don't but psychics also don't have a verifiable feedback mechanism and unlimited retries. Give the psychic the ability to adjust answers based on tests and you'll have have a fair fight.

deleted 6d ago

[deleted]

igortg 6d ago

On one side I agree that LLMs and Agents are not intelligent, but they present an illusion of intelligence given the shear amount of data they can process and act upon. But that cannot be used to discredit the fact that these are incredibly powerful tools that can get out of control and cause great damage.

atemerev 6d ago
What is "intelligence" then. I think the universalism of LLMs is now evident enough that they can be considered "intelligent", even in slightly different realization compared to humans. As of "great damage", I doubt it. They do not have self-preservation instinct (all "worrying" experiments are the attempts to initiate something resembling self-preservation from human initiative). The driving part is external - the query loop can always be turned off. So yes, a dangerous tool that can be exploited by humans (including governments, especially governments - which is why I am skeptical to government regulation proposals, particularly looking at what passes as governments in this era). But there are no inherent dangers from their own agency, as there is none.
- alansaber 6d ago
  "Slightly different"- no, a completely different realisation of intelligence to humans. LLMs can definitely do cool stuff though.
  - pixl97 6d ago
    >completely different realisation of intelligence to humans. OK, so they are intelligent. I'm glad we agree. The problem with this word is people associate intelligence with the process they realize is occurring inside their head which is only a tiny fraction of what intelligence encompasses. Humans (without study) know very little about intelligence and they very gray boundaries it has.
- chrisjj 5d ago
  > As of "great damage", I doubt it. They do not have self-preservation instinct The potential for great damage relies upon the lack of self-preservation instinct. That lack releases the so-called AI from an order under pain of death. Damage cannot be punished.

cmiles74 6d ago

In my opinion, this article is talking about people who get into long conversations and become convinced the model is “intelligent”, maybe even “AGI”. This seems to be a trap people are falling into, even in 2026.

atemerev 6d ago
Initial definitions of AGI (as of 2014) are long surpassed. Then, AGI was defined as something that is general enough to work in many fields and orient themselves. It did not include being smarter than humans or even having comparable intelligence to humans. By original standards, we have AGI already for some time. Current AGI definitions are intentionally vague.
krupan 6d ago
It's definitely talking about that, but it's also talking about all of us: - fawning over how amazing these tools are - believing everything OpenAI and Anthropic say about how powerful and dangerous their product is - minimizing the amount of human effort and involvement in every "AI" achievement

Madmallard 6d ago

Neither does LLMs. Not without a monumental amount of human oversight and expertise. Everyone not in a particular field asking LLM about said field is rolling bad dice.

atemerev 6d ago
I have published a LLM-assisted proof of a long-standing problem in convex geometry (70 years) while not being a specialist in convex geometry. I am a scientist and I have education and scientific experience in computer science and systems biology, but I was never a mathematician. So, well, you can definitely work across fields at least.
- tripledry 6d ago
  Do you have a link? I would also be very interested in seeing the prompts, from what I've seen you still need to understand the field, maybe I'm wrong.

krupan 6d ago

Neither do LLMs. Humans using them do. Similar to how cops solve crimes using psychics.

pixl97 6d ago
Your statement is unclear, break it down a little further on how you think this works.
- krupan 6d ago
  The original article already attempted to do that but I'll take another stab. Humans tend to believe what they want to believe. They gain confidence when a confident person (or LLM) they work with exudes confidence. Where they might have hesitated or got stuck in analysis paralysis alone, the aid of a con (short for confidence) artist will get them moving. They then attribute the solution and praise for it solely to the confident person (or LLM) that nudged them along. Did the "psychic" who understands human motivation better than the cops actually help? Very likely. Did the psychic end up getting far more credit than they deserve (it's magical! You'll never believe! Well worth the money I spent!)? Absolutely. The amount of real help vs. confidence/manipulation the LLM or con artist provides varies in each situation and scenario and mix of people (and LLMs) involved. It's something to strive to be aware and analytical about.
  - pixl97 6d ago
    There is also something else horizontally to this that occurs. There are numerous scientific discoveries that have been made because one human looked at all available data and turned off the assumptions other people in the field have been using for years, maybe hundreds of years. Once you delete the assumption and make a new one the answer is obvious any anyone from that point wonders how so many humans could have missed it for so much time. This can occur readily with LLMs as it can with people. It's very likely we will see this a lot as the causal connection in the available data will connected differently in their minds. Escaping from a local maximum can be very difficult as you have to climb uphill with a nearly infinite amount of freedom but no ability to see the horizon. Psychics, LLMs, or some guy name Bob walking in an providing a workable analogy for you to escape the local maxima and move closer to a global maxima are all the same. Moreso, if LLMs are AGI we should expect these "psychic" behaviors just as much as actual discovery because both continuums exist in the same problem space as human minds.

_superposition_ 6d ago

No they don't but psychics also don't have a verifiable feedback mechanism and unlimited retries. Give the psychic the ability to adjust answers based on tests and you'll have have a fair fight.