7 ms·

> We don't know if AGI is even possible outside of a biological construct yet. This is key. A discovery that AGI is impossible in principle to implement in an

• epiccoleman 1y ago
  Would that really be a physics discovery? I mean I guess everything ultimately is. But it seems like maybe consciousness could be understood in terms of "higher level" sciences - somewhere on the chain of neurology->biology->chemistry->physics.
  • mmoskal 1y ago
    Consciousness (subjective experience) is possibly orthogonal to intelligence (ability to achieve complex goals). We definitely have a better handle on what intelligence is than consciousness.
    • epiccoleman 1y ago
      That does make sense, reminds me of Blindsight, where one central idea is that conscious experience might not even be necessary for intelligence (and possibly even maladaptive).
  • marcosdumay 1y ago
    > Would that really be a physics discovery? No, it could be something that proves all of our fundamental mathematics wrong. The GP just gave the more conservative option.
    • tshaddox 1y ago
      I’m not sure what you mean. This new discovery in mathematics would also necessarily tell us something new about what is computable, which is physics.
      • marcosdumay 1y ago
        It would impact physics, yes. And literally every other natural science.
  • tshaddox 1y ago
    That sounds like you’re describing AGI as being impractical to implement in an electronic computer, not impossible in principle.
    • epiccoleman 1y ago
      Yeah, I guess I'm not taking a stance on that above, just wondering where in that chain holds the most explanatory power for intelligence and/or consciousness. I don't think there's any real reason to think intelligence depends on "meat" as its substrate, so AGI seems in principle possible to me. Not that my opinion counts for much on this topic, since I don't really have any relevant education on the topic. But my half baked instinct is that LLMs in and of themselves will never constitute true AGI. The biggest thing that seems to be missing from what we currently call AI is memory - and it's very interesting to see how their behavior changes if you hook up LLMs to any of the various "memory MCP" implementations out there. Even experimenting with those sorts of things has left me feeling there's still something (or many somethings) missing to take us from what is currently called "AI" to "AGI" or so-called super intelligence.
      • kelnos 1y ago
        > I don't think there's any real reason to think intelligence depends on "meat" as its substrate This made me think of... ok, so let's say that we discover that intelligence does indeed depend on "meat". Could we then engineer a sort of organic computer that has general intelligence? But could we also claim that this organic computer isn't a computer at all, but is actually a new genetically engineered life form?
      • mindcrime 1y ago
        But my half baked instinct is that LLMs in and of themselves will never constitute true AGI. I agree. But... LLM's are not the only game in town. They are just one approach to AI that is currently being pursued. The current dominant approach by investment dollars, attention, and hype, to be sure. But still far from the only thing around.
• deleted 1y ago
  [deleted]
• throwaway31131 1y ago
  We would also need a definition of AGI that is provable or disprovable. We don’t even have a workable definition, never mind a machine.
  • tshaddox 1y ago
    We don’t need such a definition of general intelligence to conclude that biological humans have it, so I’m not sure why we’d such a definition for AGI.
    • bastawhiz 1y ago
      Well general intelligence in humans already exists, whereas general intelligence doesn't yet exist in machines. How do we know when we have it? You can't even simply compare it to humans and ask "is it able to do the same things?" because your answer depends on what you define those things to be. Surely you wouldn't say that someone who can't remember names or navigate without GPS lacks general intelligence, so it's necessary to define what criteria are absolutely required.
      • tshaddox 1y ago
        > You can't even simply compare it to humans and ask "is it able to do the same things?" because your answer depends on what you define those things to be. Right, but you can’t compare two different humans either. You don’t test each new human to see if they have it. Somehow we conclude that humans have it without doing either of those things.
        • Jensson 1y ago
          > You don’t test each new human to see if they have it We do, its called school and we label some humans with different learning disabilities. Some of those learning disabilities are grave enough that they can't learn to do tasks we expect humans to be able to learn, such humans can be argued to not posses the general intelligence we expect from humans. Interacting with an LLM today is like interacting with an Alzheimer patient, they can do things they already learned well but poke at it and it all falls apart and they start repeating themselves, they can't learn.
• aorloff 1y ago
  A question which will be trivial to answer once you properly define what you mean by "brain" Presumably "brains" do not do many of the things that you will measure AGI by, and your brain is having trouble understanding the idea that "brain" is not well understood by brains. Does it make it any easier if we simplify the problem to: what is the human doing that makes (him) intelligent ? If you know your historical context, no. This is not a solved problem.
  • tshaddox 1y ago
    > Does it make it any easier if we simplify the problem to: what is the human doing that makes (him) intelligent ? Sure, it doesn’t have to be literally just the brain, but my point is you’d need very new physics to answer the question “how does a biological human have general intelligence?”
    • aorloff 1y ago
      Suppose dogs invent their own idea of intelligence but they say only dogs have it. Do we think new physics would be required to validate dog intelligence ?
      • tshaddox 1y ago
        The claim that only dogs have intelligence is open for criticism, just like every other claim. I’m not sure what your point is, because the source of the claim is irrelevant anyway. The reason I think that humans have general intelligence is not that humans say that they have it.
• lll-o-lll 1y ago
  It’s not really “what is the brain doing”; that path leads to “quantum mysticism”. What we lack is a good theoretical framework about complex emergence. More maths in this space please. Intelligence is an emergent phenomenon; all the interesting stuff happens at the boundary of order and disorder but we don’t have good tools in this space.
• ysavir 1y ago
  Seems the opposite way round to me. We couldn't conclusively say that AGI is possible in principle until some physics (or rather biology) discovery explains how it would be possible. Until then, anything we engineer is an approximation as best.
• AIPedant 1y ago
  It is vacuously true that a Turing machine can implement human intelligence: simply solve the Schrödinger equation for every atom in the human body and local environment. Obviously this is cost-prohibitive and we don’t have even 0.1% of the data required to make the simulation. Maybe we could simulate every single neuron instead, but again it’ll take many decades to gather the data in living human brains, and it would still be extremely expensive computationally since we would need to simulate every protein and mRNA molecule across billions of neurons and glial cells. So the question is whether human intelligence has higher-level primitives that can be implemented more efficiently - sort of akin to solving differential equations, is there a “symbolic solution” or are we forced to go “numerically” no matter how clever we are?
  • tshaddox 1y ago
    > It is vacuously true that a Turing machine can implement human intelligence: simply solve the Schrödinger equation for every atom in the human body and local environment. Yes, that is the bluntest, lowest level version of what I mean. To discover that this wouldn’t work in principle would be to discover that quantum mechanics is false. Which, hey, quantum mechanics probably is false! But discovering the theory which both replaces quantum mechanics and shows that AGI in an electronic computer is physically impossible is definitely a tall order.
    • card_zero 1y ago
      There's that aphorism that goes: people who thought the epitome of technology was a steam engine pictured the brain as pipes and connecting rods, people who thought the epitome of technology was a telephone exchange pictured the brain as wires and relays... and now we have computers, and the fact that they can in principle simulate anything at all is a red herring, because we can't actually make them simulate things we don't understand, and we can't always make them simulate things we do understand, either, when it comes down to it. We still need to know what the thing is that the brain does, it's still a hard question, and maybe it would even be a kind of revolution in physics, just not in fundamental physics.
      • 1y ago
• add-sub-mul-div 1y ago
  It doesn't have to be impossible in principle, just impossible given how little we understand consciousness or will anytime in the next century. Impossible for all intents and purposes for anyone living today.
  • tshaddox 1y ago
    > Impossible for all intents and purposes for anyone living today. Sure, but tons of things which are obviously physically possible are also out of reach for anyone living today.
• bilsbie 1y ago
  Not necessarily. It could simply be a question of scale. Being analog and molecular means that brain could be doing enormously more than any foreseeable computer. For a simple example what if every neuron is doing trillions of calculations. (I’m not saying it is, just that it’s possible)
  • tshaddox 1y ago
    I think you’re merely referring to what is feasible in practice to computer with our current or near-future computers. I was referring to what is computable in principle.
    • bilsbie 1y ago
      Right. That’s what I was responding to. OP wrote: > We don't know if AGI is even possible outside of a biological construct yet And you replied that means it’s impossible in principle. I’m correcting you in saying that it can be impossible in ways other than principle.
• manquer 1y ago
  Not necessarily , for a given definition of AGI you could have mathematical proof that it is incomputable similar to how Gödel incompleteness theorems work . It need not even be incomputable, it could be NP hard and practically be incomputable, or it could be undecidable I.e. a version of the halting problem. There are any number of ways our current models of mathematics or computation can in theory could be shown as not capable of expressing AGI without needing a fundamental change in physics
  • jibal 1y ago
    This is not a logically valid argument. If AGI isn't possible for any of the reasons you suggest, then human cognition isn't possible either. And you're making numerous category mistakes ... an AGI can't be "incomputable" or "undecidable" or "NP hard". A problem that we put to an AGI might be NP hard, but neither AGIs nor humans need to solve the entire class of problems, only instances of them, and they don't have to solve them optimally. Thus salesmen are able to travel. To quote ChatGPT on this: "Could cognition be NP-hard? Strictly speaking, no—if human brains were literally solving NP-hard problems in their general form, we wouldn’t be able to think at all. Does cognition involve NP-hard problems? Yes—in theory, many of the domains we reason about are NP-hard in the worst case. What’s really happening? Human cognition relies on heuristics, approximations, and exploiting real-world regularities, so we almost never hit the formal “worst cases” that define NP-hardness."
• slashdave 1y ago
  That question is not a physics question