13 ms·

This paper presents a theoretical proof that AGI systems will structurally collapse under certain semantic conditions — not due to lack of compute, but because

This paper presents a theoretical proof that AGI systems will structurally collapse under certain semantic conditions — not due to lack of compute, but because of how entropy behaves in heavy-tailed decision spaces. The idea is called IOpenER: Information Opens, Entropy Rises. It builds on Shannon’s information theory to show that in specific problem classes (those with α ≤ 1), adding information doesn’t reduce uncertainty — it increases it. The system can’t converge, because meaning itself keeps multiplying. The core concept — entropy divergence in these spaces — was already present in my earlier paper, uploaded to PhilArchive on June 1. This version formalizes it. Apple’s study, The Illusion of Thinking, was published a few days later. It shows that frontier reasoning models like Claude 3.7 and DeepSeek-R1 break down exactly when problem complexity increases — despite adequate inference budget. I didn’t write this paper in response to Apple’s work. But the alignment is striking. Their empirical findings seem to match what IOpenER predicts. Curious what this community thinks: is this a meaningful convergence, or just an interesting coincidence? Links: This paper (entropy + IOpenER): https://philarchive.org/archive/SCHAIM-14 https://philarchive.org/archive/SCHAIM-14 First paper (ICB + computability): https://philpapers.org/archive/SCHAII-17.pdf https://philpapers.org/archive/SCHAII-17.pdf Apple’s study: https://machinelearning.apple.com/research/illusion-of-thinking https://machinelearning.apple.com/research/illusion-of-think...

ben_w 1y ago
The mathematical proof, as you describe it, sounds like the "No Free Lunch theorem". Humans also can't generalise to learning such things. As you note in 2.1, there is widespread disagreement on what "AGI" means. I note that you list several definitions which are essentially "is human equivalent". As humans can be reduced to physics, and physics can be expressed as a computer program, obviously any such definition can be achieved by a sufficiently powerful computer. For 3.1, you assert: """ Now, let's observe what happens when an Al system - equipped with state-of-the-art natural language processing, sentiment analysis, and social reasoning - attempts to navigate this question. The Al begins its analysis: • Option 1: Truthful response based on biometric data → Calculates likely negative emotional impact → Adjusts for honesty parameter → But wait, what about relationship history? → Recalculating... • Option 2: Diplomatic deflection → Analyzing 10,000 successful deflection patterns → But tone matters → Analyzing micro-expressions needed → But timing matters → But past conversations matter → Still calculating... • Option 3: Affectionate redirect → Processing optimal sentiment → But what IS optimal here? The goal keeps shifting → Is it honesty? Harmony? Trust? → Parameters unstable → Still calculating... • Option n: .... Strange, isn't it? The Al hasn't crashed. It's still running. In fact, it's generating more and more nuanced analyses. Each additional factor may open ten new considerations. It's not getting closer to an answer - it's diverging. """ Which AI? ChatGPT just gives an answer. Your other supposed examples have similar issues in that it looks like you've *imagined* an AI rather than having tried asking an AI to seeing what it actually does or doesn't do. I'm not reading 47 pages to check for other similar issues.
- ICBTheory 1y ago
  1. I appreciate the comparison — but I’d argue this goes somewhat beyond the No Free Lunch theorem. NFL says: no optimizer performs best across all domains. But the core of this paper doesnt talk about performance variability, it’s about structural inaccessibility. Specifically, that some semanti spaces (e.g., heavy-tailed, frame-unstable, undecidable contexts) can’t be computed or resolved by any algorithmic policy — no matter how clever or powerful. The model does not underperform here, the point is that the problem itself collapses the computational frame. 2. OMG, lool. ... just to clarify, there’s been a major misunderstanding :) the “weight-question”-Part is NOT a transcript from my actual life... thankfully - I did not transcribe a live ChatGPT consult while navigating emotional landmines with my (perfectly slim) wife, then submit it to PhilPapers and now here… So - NOT a real thread, - NOT a real dialogue with my wife... - just an exemplary case... - No, I am not brain dead and/or categorically suicidal!! - And just to be clear: I dont write this while sitting in some marital counseling appointment, or in my lawyer's office, the ER, or in a coroners drawer --> It’s a stylized, composite example of a class of decision contexts that resist algorithmic resolution — where tone, timing, prior context, and social nuance create an uncomputably divergent response space. Again : No spouse was harmed in the making of that example. ;-))))
  - Dave_Wishengrad 1y ago
    [dead]
  - andoando 1y ago
    Just a layman here so Im not sure if Im understanding (probably not), but humans dont analyze every possible scenario ad infinitum, we go based on the accumulation of our positive/negative experiences from the past. We make decisions based on some self construed goal and beliefs as to what goes towards those goals, and these are arbitrary with no truth. Napolean for example conquered Europe perhaps simiply becuause he thought he was the best to rule it, not through a long chain of questions and self doubt We are generally intelligent only in the sense that our reasoning/modeling capabilities allow us to understand anything that happens in space-time.
  - 1y ago
vessenes 1y ago
Thanks for this - Looking forward to reading the full paper. That said, the most obvious objection that comes to mind about the title is that … well, I feel that I’m generally intelligent, and therefore general intelligence of some sort is clearly not impossible. Can you give a short précis as to how you are distinguishing humans and the “A” in artificial?
- rusk 1y ago
  Not the person asked, but in time honoured tradition I will venture forth that the key difference is billions of years of evolution. Innumerable blooms and culls. And a system that is vertically integrated to its core and self sustaining.
  - ben_w 1y ago
    AI can be, and often are, trained by simulated evolution.
    - rusk 1y ago
      Simulated.
      - ben_w 1y ago
        You have to say why you think that matters. It still culls the unfit.
        rusk 1y ago
        I don’t. You have boiled a process of billions of years down to a single sentence. You should ponder your absurdity.
        ben_w 1y ago
        That's the point of language, to abstract a complex thing to what is often as little as a single word or sentence. It's not like the idea represented by the words "simulated evolution" is itself as simple as those two words anyway. That it takes nature "billions of years" for natural evolution isn't even important here, because it's not like simulations have to run in real-time. If you run simulated evolution with mechanical parts and the reward function of things that function like clocks, you get the (design of) a thing that functions like a clock, and if you run the physics simulation of the design, you can tell the time with it. Do it with electronics and things that act like a radio, you get a radio. Do it with a CAD design and the goal of strength for minimum mass, you end up with something that looks bone-like. We also do it with AI, why should we expect it not to produce things in the general category of "minds"? Not necessarily human minds, even the biggest by parameter count are much smaller structures than our brains, but the general category.
WhitneyLand 1y ago
“This paper presents a theoretical proof that AGI systems will structurally collapse under certain semantic conditions…” No it doesn’t. Shannon entropy measures statistical uncertainty in data. It says nothing about whether an agent can invent new conceptual frames. Equating “frame changes” with rising entropy is a metaphor, not a theorem, so it doesn’t even make sense as a mathematical proof. This is philosophical musing at best.
- ICBTheory 1y ago
  Correct: Shannon entropy originally measures statistical uncertainty over a fixed symbol space. When the system is fed additional information/data, then entropy goes down, uncertainty falls. This is always true in situations where the possible outcomes are a) sufficiently limited and b)unequally distributed. In such cases, with enough input, the system can collapse the uncertainty function within a finite number of steps. But the paper doesn’t just restate Shannon. It extends this very formalism to semantic spaces where the symbol set itself becomes unstable. These situations arise when (a) entropy is calculated across interpretive layers (as in LLMs), and (b) the probability distribution follows a heavy-tailed regime (α ≤ 1). Under these conditions, entropy divergence becomes mathematically provable. This is far from being metaphorical: it’s backed by formal Coq-style proofs (see Appendix C in he paper). AND: it is exactly the mechanism that can explain the Apple-Papers' results
  - int_19h 1y ago
    Your paper only claims that those Coq snippets constitute a "constructive proof sketch". Have those formalizations actually been verified, and if so, why not include the results in the paper? Separately from that, your entire argument wrt Shannon hinges on this notion that it is applicable to "semantic spaces", but it is not clear on what basis this jump is made.
  - Llamamoe 1y ago
    This sounds like a good argument why making the optimal decisions in every single case is undecidable, but not why an AGI should be unable to exist.
gremlinsinc 1y ago
does this include if the AI can devise new components and use drones and things essentially to build a new iteration of itself more capable to compute a thing and keep repeating this going out into the universe as needed for resources and using von Neumann probes.. etc?
yodon 1y ago
I'm wondering if you may have rediscovered the concept of "Wicked Problems", which have been studied in system analysis and sociology since the 1970's (I'd cite the Wikipedia page, but I've never been particularly fond of Wikipedia's write up on them). They may be worth reading up on if you're not familiar with them.
- ICBTheory 1y ago
  Wow, that is a great advice. Never heard of them - and they seem to fit perfectly into the whole concept THANK YOU! :-)
- Agraillo 1y ago
  It's interesting. The question from the paper "Darling, please be honest: have I gained weight?" assumes that the "socially acceptability" of the answer should be taken into account. In this case the problem fits the "Wickedness" (Wikipedia's quote is "Classic examples of wicked problems include economic, environmental, and political issues"). But taken formally, and with the ability for LLM to ask questions in return to decrease formal uncertainty ("Please, give me several full photos of yourself from the past year to evaluate"), it is not "wicked" at all. This example alone makes the topic very uncertain in itself
AndrewKemendo 1y ago
In your paper it states: AGI as commonly defined However I don’t see where you go on to give a formalization of “AGI” or what the common definition is. can you do that in a mathematically rigorous way such that it’s a testable hypothesis?
- fc417fc802 1y ago
  I don't think it exists. We can't even seem to agree on a standard criteria for "intelligence" when assessing humans let alone a rigorous mathematical definition. In turn, my understanding of the commonly accepted definition for AGI (as opposed to AI or ML) has always been "vaguely human or better". Unless the marketing department is involved in which case all bets are off.
  - viraptor 1y ago
    It can exist for the purpose of the paper. As in "when I write AGI, I mean ...". Otherwise what's the point in any rigour if we're just going by "you know what I mean" vibes.
vidarh 1y ago
Unless you can prove that humans exceed the Turing computable, the headline is nonsense unless you can also show that the Church-Turing thesis isn't true. Since you don't even appear to have dealt with this, there is no reason to consider the rest of the paper.
- haneul 1y ago
  > In plain language: > No matter how sophisticated, the system MUST fail on some inputs. Well, no person is immune to propaganda and stupididty, so I don't see it as a huge issue.
  - amelius 1y ago
    But what then is the relevance of the study?
    - haneul 1y ago
      I suppose it disproves embodied, fully meat-space god if sound?
      - amelius 1y ago
        I'm looking at the title again and it seems wrong, because AGI ~ human intelligence. Unless human intelligence has non physical components to it.
  - vidarh 1y ago
    I have no idea how you believe this relates to the comment you replied to.
    - harimau777 1y ago
      If I'm understanding correctly, they are arguing that the paper only requires that an intelligent system will fail for some inputs and suggest that things like propaganda are inputs for which the human intelligent system fails. Therefore, they are suggesting that the human intelligent system does not necessarily refute the paper's argument.
ccppurcell 1y ago
I am sympathetic to the kind of claims made by your paper. I like impossibility results and I could believe that for some definition of AGI there is at least a plausible argument that entropy is a problem. Scalable quantum computing is a good point of comparison. But your paper is throwing up crank red flags left and right. If you have a strong argument for such a bold claim, you should put it front and centre: give your definition of AGI, give your proof, let it stand on its own. Some discussion of the definition is useful. Discussion of your personal life and Kant is really not. Skimming through your paper, your argument seems to boil down to "there must be some questions AGI gets wrong". Well since the definition includes that AGI is algorithmic, this is already clear thanks to the halting problem.
afiori 1y ago
> specific problem classes (those with α ≤ 1), For the layman, what does α mean here?
- 317070 1y ago
  I'm sure this is a reference to alpha stable distributions: https://en.m.wikipedia.org/wiki/Stable_distribution https://en.m.wikipedia.org/wiki/Stable_distribution Most of these don't have finite moments and are hard to do inference on with standard statistical tools. Nassim Taleb's work (Black Swan, etc.) is around these distributions. But I think the argument of OP in this section doesn't hold.
coderenegade 1y ago
Apple's paper sets up a bit of a straw man in my opinion. It's unreasonable to expect that an LLM not trained on what are essentially complex algorithmic tasks is just going to discover the solution on the spot. Most people can solve simple cases of the tower of Hanoi, and almost none of us can solve complex cases. In general, the ones who can have trained to be able to do so.

ben_w 1y ago

The mathematical proof, as you describe it, sounds like the "No Free Lunch theorem". Humans also can't generalise to learning such things. As you note in 2.1, there is widespread disagreement on what "AGI" means. I note that you list several definitions which are essentially "is human equivalent". As humans can be reduced to physics, and physics can be expressed as a computer program, obviously any such definition can be achieved by a sufficiently powerful computer. For 3.1, you assert: """ Now, let's observe what happens when an Al system - equipped with state-of-the-art natural language processing, sentiment analysis, and social reasoning - attempts to navigate this question. The Al begins its analysis: • Option 1: Truthful response based on biometric data → Calculates likely negative emotional impact → Adjusts for honesty parameter → But wait, what about relationship history? → Recalculating... • Option 2: Diplomatic deflection → Analyzing 10,000 successful deflection patterns → But tone matters → Analyzing micro-expressions needed → But timing matters → But past conversations matter → Still calculating... • Option 3: Affectionate redirect → Processing optimal sentiment → But what IS optimal here? The goal keeps shifting → Is it honesty? Harmony? Trust? → Parameters unstable → Still calculating... • Option n: .... Strange, isn't it? The Al hasn't crashed. It's still running. In fact, it's generating more and more nuanced analyses. Each additional factor may open ten new considerations. It's not getting closer to an answer - it's diverging. """ Which AI? ChatGPT just gives an answer. Your other supposed examples have similar issues in that it looks like you've *imagined* an AI rather than having tried asking an AI to seeing what it actually does or doesn't do. I'm not reading 47 pages to check for other similar issues.

ICBTheory 1y ago
1. I appreciate the comparison — but I’d argue this goes somewhat beyond the No Free Lunch theorem. NFL says: no optimizer performs best across all domains. But the core of this paper doesnt talk about performance variability, it’s about structural inaccessibility. Specifically, that some semanti spaces (e.g., heavy-tailed, frame-unstable, undecidable contexts) can’t be computed or resolved by any algorithmic policy — no matter how clever or powerful. The model does not underperform here, the point is that the problem itself collapses the computational frame. 2. OMG, lool. ... just to clarify, there’s been a major misunderstanding :) the “weight-question”-Part is NOT a transcript from my actual life... thankfully - I did not transcribe a live ChatGPT consult while navigating emotional landmines with my (perfectly slim) wife, then submit it to PhilPapers and now here… So - NOT a real thread, - NOT a real dialogue with my wife... - just an exemplary case... - No, I am not brain dead and/or categorically suicidal!! - And just to be clear: I dont write this while sitting in some marital counseling appointment, or in my lawyer's office, the ER, or in a coroners drawer --> It’s a stylized, composite example of a class of decision contexts that resist algorithmic resolution — where tone, timing, prior context, and social nuance create an uncomputably divergent response space. Again : No spouse was harmed in the making of that example. ;-))))
- Dave_Wishengrad 1y ago
  [dead]
- andoando 1y ago
  Just a layman here so Im not sure if Im understanding (probably not), but humans dont analyze every possible scenario ad infinitum, we go based on the accumulation of our positive/negative experiences from the past. We make decisions based on some self construed goal and beliefs as to what goes towards those goals, and these are arbitrary with no truth. Napolean for example conquered Europe perhaps simiply becuause he thought he was the best to rule it, not through a long chain of questions and self doubt We are generally intelligent only in the sense that our reasoning/modeling capabilities allow us to understand anything that happens in space-time.
- 1y ago

vessenes 1y ago

Thanks for this - Looking forward to reading the full paper. That said, the most obvious objection that comes to mind about the title is that … well, I feel that I’m generally intelligent, and therefore general intelligence of some sort is clearly not impossible. Can you give a short précis as to how you are distinguishing humans and the “A” in artificial?

rusk 1y ago
Not the person asked, but in time honoured tradition I will venture forth that the key difference is billions of years of evolution. Innumerable blooms and culls. And a system that is vertically integrated to its core and self sustaining.
- ben_w 1y ago
  AI can be, and often are, trained by simulated evolution.
  - rusk 1y ago
    Simulated.
    - ben_w 1y ago
      You have to say why you think that matters. It still culls the unfit.
      - rusk 1y ago
        I don’t. You have boiled a process of billions of years down to a single sentence. You should ponder your absurdity.
        ben_w 1y ago
        That's the point of language, to abstract a complex thing to what is often as little as a single word or sentence. It's not like the idea represented by the words "simulated evolution" is itself as simple as those two words anyway. That it takes nature "billions of years" for natural evolution isn't even important here, because it's not like simulations have to run in real-time. If you run simulated evolution with mechanical parts and the reward function of things that function like clocks, you get the (design of) a thing that functions like a clock, and if you run the physics simulation of the design, you can tell the time with it. Do it with electronics and things that act like a radio, you get a radio. Do it with a CAD design and the goal of strength for minimum mass, you end up with something that looks bone-like. We also do it with AI, why should we expect it not to produce things in the general category of "minds"? Not necessarily human minds, even the biggest by parameter count are much smaller structures than our brains, but the general category.

WhitneyLand 1y ago

“This paper presents a theoretical proof that AGI systems will structurally collapse under certain semantic conditions…” No it doesn’t. Shannon entropy measures statistical uncertainty in data. It says nothing about whether an agent can invent new conceptual frames. Equating “frame changes” with rising entropy is a metaphor, not a theorem, so it doesn’t even make sense as a mathematical proof. This is philosophical musing at best.

ICBTheory 1y ago
Correct: Shannon entropy originally measures statistical uncertainty over a fixed symbol space. When the system is fed additional information/data, then entropy goes down, uncertainty falls. This is always true in situations where the possible outcomes are a) sufficiently limited and b)unequally distributed. In such cases, with enough input, the system can collapse the uncertainty function within a finite number of steps. But the paper doesn’t just restate Shannon. It extends this very formalism to semantic spaces where the symbol set itself becomes unstable. These situations arise when (a) entropy is calculated across interpretive layers (as in LLMs), and (b) the probability distribution follows a heavy-tailed regime (α ≤ 1). Under these conditions, entropy divergence becomes mathematically provable. This is far from being metaphorical: it’s backed by formal Coq-style proofs (see Appendix C in he paper). AND: it is exactly the mechanism that can explain the Apple-Papers' results
- int_19h 1y ago
  Your paper only claims that those Coq snippets constitute a "constructive proof sketch". Have those formalizations actually been verified, and if so, why not include the results in the paper? Separately from that, your entire argument wrt Shannon hinges on this notion that it is applicable to "semantic spaces", but it is not clear on what basis this jump is made.
- Llamamoe 1y ago
  This sounds like a good argument why making the optimal decisions in every single case is undecidable, but not why an AGI should be unable to exist.

gremlinsinc 1y ago

does this include if the AI can devise new components and use drones and things essentially to build a new iteration of itself more capable to compute a thing and keep repeating this going out into the universe as needed for resources and using von Neumann probes.. etc?

yodon 1y ago

I'm wondering if you may have rediscovered the concept of "Wicked Problems", which have been studied in system analysis and sociology since the 1970's (I'd cite the Wikipedia page, but I've never been particularly fond of Wikipedia's write up on them). They may be worth reading up on if you're not familiar with them.

ICBTheory 1y ago
Wow, that is a great advice. Never heard of them - and they seem to fit perfectly into the whole concept THANK YOU! :-)
Agraillo 1y ago
It's interesting. The question from the paper "Darling, please be honest: have I gained weight?" assumes that the "socially acceptability" of the answer should be taken into account. In this case the problem fits the "Wickedness" (Wikipedia's quote is "Classic examples of wicked problems include economic, environmental, and political issues"). But taken formally, and with the ability for LLM to ask questions in return to decrease formal uncertainty ("Please, give me several full photos of yourself from the past year to evaluate"), it is not "wicked" at all. This example alone makes the topic very uncertain in itself

AndrewKemendo 1y ago

In your paper it states: AGI as commonly defined However I don’t see where you go on to give a formalization of “AGI” or what the common definition is. can you do that in a mathematically rigorous way such that it’s a testable hypothesis?

fc417fc802 1y ago
I don't think it exists. We can't even seem to agree on a standard criteria for "intelligence" when assessing humans let alone a rigorous mathematical definition. In turn, my understanding of the commonly accepted definition for AGI (as opposed to AI or ML) has always been "vaguely human or better". Unless the marketing department is involved in which case all bets are off.
- viraptor 1y ago
  It can exist for the purpose of the paper. As in "when I write AGI, I mean ...". Otherwise what's the point in any rigour if we're just going by "you know what I mean" vibes.

vidarh 1y ago

Unless you can prove that humans exceed the Turing computable, the headline is nonsense unless you can also show that the Church-Turing thesis isn't true. Since you don't even appear to have dealt with this, there is no reason to consider the rest of the paper.

haneul 1y ago
> In plain language: > No matter how sophisticated, the system MUST fail on some inputs. Well, no person is immune to propaganda and stupididty, so I don't see it as a huge issue.
- amelius 1y ago
  But what then is the relevance of the study?
  - haneul 1y ago
    I suppose it disproves embodied, fully meat-space god if sound?
    - amelius 1y ago
      I'm looking at the title again and it seems wrong, because AGI ~ human intelligence. Unless human intelligence has non physical components to it.
- vidarh 1y ago
  I have no idea how you believe this relates to the comment you replied to.
  - harimau777 1y ago
    If I'm understanding correctly, they are arguing that the paper only requires that an intelligent system will fail for some inputs and suggest that things like propaganda are inputs for which the human intelligent system fails. Therefore, they are suggesting that the human intelligent system does not necessarily refute the paper's argument.

ccppurcell 1y ago

I am sympathetic to the kind of claims made by your paper. I like impossibility results and I could believe that for some definition of AGI there is at least a plausible argument that entropy is a problem. Scalable quantum computing is a good point of comparison. But your paper is throwing up crank red flags left and right. If you have a strong argument for such a bold claim, you should put it front and centre: give your definition of AGI, give your proof, let it stand on its own. Some discussion of the definition is useful. Discussion of your personal life and Kant is really not. Skimming through your paper, your argument seems to boil down to "there must be some questions AGI gets wrong". Well since the definition includes that AGI is algorithmic, this is already clear thanks to the halting problem.

afiori 1y ago

> specific problem classes (those with α ≤ 1), For the layman, what does α mean here?

317070 1y ago
I'm sure this is a reference to alpha stable distributions: https://en.m.wikipedia.org/wiki/Stable_distribution https://en.m.wikipedia.org/wiki/Stable_distribution Most of these don't have finite moments and are hard to do inference on with standard statistical tools. Nassim Taleb's work (Black Swan, etc.) is around these distributions. But I think the argument of OP in this section doesn't hold.

coderenegade 1y ago

Apple's paper sets up a bit of a straw man in my opinion. It's unreasonable to expect that an LLM not trained on what are essentially complex algorithmic tasks is just going to discover the solution on the spot. Most people can solve simple cases of the tower of Hanoi, and almost none of us can solve complex cases. In general, the ones who can have trained to be able to do so.