8 ms·

I have a personal belief that I can’t quite articulate in rigorous scientific terms that there is some information-theoretic barrier for us to understand the “t

I have a personal belief that I can’t quite articulate in rigorous scientific terms that there is some information-theoretic barrier for us to understand the “true nature” or “essence” of our own intelligence, and so if we can’t get to that, we’ll never be able to model it (notwithstanding “all models are wrong”). The belief that we can get to AGI comes off as religion to me. It is a substitute for something we can’t really understand, and it will continue to shift the more we learn, yet always remain out of reach. There will be some true believers, and some people simply gunning for power. Might as well call AGI Nirvana.

deleted 3y ago
[deleted]
jameshart 3y ago
The idea that there’s some ‘ information-theoretic barrier for us to understand the “true nature” or “essence” of our own intelligence’ sounds more religious to me. If evolution can cross that barrier just by banging molecules together and seeing which ones work, it seems unlikely there’s some causal disconnect that makes it impossible for us to get there by thinking about it.
- RcouF1uZ4gsC 3y ago
  > If evolution can cross that barrier just by banging molecules together and seeing which ones work, it seems unlikely there’s some causal disconnect that makes it impossible for us to get there by thinking about it. Evolution also crossed the flight barrier by banging molecules together. I don't think banging molecules together without having an understanding of physics and the forces involved would have been a viable means for us to get to flight.
  - jameshart 3y ago
    That’s… not a refutation of my point. AI/flight analogies are tired, but the OPs argument amounts to the equivalent of, before the Wright Brothers, proclaiming ‘there’s an inherent inability for humans to ever conceive of a way to engineer heavier than air flight’. It’s a ‘man was never mean to fly, therefore heavier than air flight is impossible’ argument.
  - shawnz 3y ago
    But wasn't it through experimentation that we developed that understanding? And in the earliest phases of that experimentation, that's when our understanding was at its weakest, right? You almost seem to be implying that we had the understanding before ever attempting the experiments.
- mejutoco 3y ago
  I dont have a strong opinion about it, but when I read parents post I immediately thought of Gödel s works and how there are limits of what you can know within a system (and in Mathematics no less). Thinking that something similar exists in intelligence does not seem so far fetched. As a hypothesis, of course.
  - jameshart 3y ago
    And yet we can use mathematics to know that that limit exists. It’s a powerful thing, mathematics. Gödel doesn’t say ‘mathematics cannot contemplate itself’. Quite the opposite.
    - mejutoco 3y ago
      It is an analogy to mention that is not a far fetched idea that there are limits. As far as I recall, one of the results of Gödel is that a system capable of Arithmetic cannot prove certain things about itself. Aka there are limits within a system. My claim was about limits existing.
  - ncallaway 3y ago
    The reason it seems far fetched is we have one example of a mechanistic, physics based machine that we know can perform the computations necessary to produce intelligence. I could understand the hypothesis that AGI is not computable, if we didn’t have an existing example of a machine that can produce AGI. Since we do have an existing machine that can produce AGI, we would have to suppose it does something: - outside of physics to achieve its results - performs some operation that is impossible for us to understand or replicate Both of those seem… unlikely to me.
    - jprete 3y ago
      It seems like an article of faith to believe that the universe is a computer in the first place. Without that assumption, we don't even know that intelligence is computation.
- paganel 3y ago
  This is a very mechanic-ist way of looking at things.
  - anon84873628 3y ago
    Yes, which seems to be the way the universe works. Duality is a religious belief.
    - paganel 3y ago
      As far as I know we haven't figured it out how the universe works, hence the unified theory debacle and all that.
  - ncallaway 3y ago
    Sure, and we have pretty good evidence of the mechanistic operation of the universe from quantum mechanics up, and we haven’t found any evidence of anything outside of that. So, to date, a mechanistic view of the universe seems like one that is the most supported be evidence at the moment. Have we ruled out meta-physics or some kind of duality? No, of course we haven’t. As long as there’s unknown physics, it’s a possibility. But I think if you’re reasoning based on the evidence we’ve been able to collect to date, then you should absolutely favor a mechanistic universe to a dual universe.
HuShifang 3y ago
Not an endorsement, but here's an argument that AGI is computationally intractable (edit: at least via ML). https://irisvanrooijcogsci.com/2023/09/17/debunking-agi-inevitability-claims/ https://irisvanrooijcogsci.com/2023/09/17/debunking-agi-inev...
- ceejayoz 3y ago
  "AI is impossible" is a tough argument, given the existence of non-A I. "Machine learning isn't the route to AI" is something you could more reasonably argue, but that's a drastically narrower claim.
- jameshart 3y ago
  Intractable and impossible are very different claims. NP Hardness is a statement about the asymptotic difficulty of solving a problem at ever larger scales. It says ‘if you have a way to solve this problem at size n, that way will scale worse than polynomially when you try to apply it to a problem at size 2n’. Which might place limits on the practicable scale of how big an n your approach gets to work for. But if your approach works practically for a big enough n to make AGI then your approach works - NP Hardness doesn’t matter. And since we know that finite mass lumps of finite numbers of gray cells are capable of GI, we have a reasonable expectation that there is some n for which AGI might be possible.
johnwheeler 3y ago
Yes, If you look at the power efficiency of the brain relative to LLMs, you can see nature has done something special, and I don’t think we should be so quick to call it game over. It might be that LLMs are just some atom or building block of a much larger and more complex thing.
- jofla_net 3y ago
  Yes, this is huge. Of all the places to shim in my ¢2, as the topic could go on forever. I remember talking to a robotics specialist and his take on efficiency was that despite the obvious speed and strength that machines have over us, you need to factor in just how little power is consumed when we move our extremities compared to what is needed for actuators and motors of even the best bots. Now this was years ago and things could have taken a huge turn, but I think given the recent statements by Altman about the looming power crush drives it home, that even if we do manage something truly monumental, it'll take a fusion reaction to pull off.
  - kaba0 3y ago
    I mean, I wouldn’t bet on that either. We built the “computing” empire on silicon, but we might be able to very significantly improve on this with a single discovery/invention. All the computational stuff we built/figured out will move out without trouble.
    - johnwheeler 3y ago
      Yes, that might be too. You just never know.
  - johnwheeler 3y ago
    Maybe Berkshire Hathaway Energy is the smart play
Lerc 3y ago
As a counterpoint, I feel like the belief that we can't get to AGI comes off as religion. It presupposes an ineffable quality that we posses that machines cannot. The argument that we have to fully understand something to build it doesn't hold water for me. I have made plenty of things where understanding the depths of what I had made took far more time and effort than building the thing in the first place. It's always hard to predict the rate of progress, Most of the current optimism comes from how radically wrong predictions were for the capabilities of AI today. 10 years ago a lot of people would have put current AI capability as arriving well after 2050. The jump in progress may not be sustained, but it definitely places doubt on people confidently predicting slow progress.
- mjburgess 3y ago
  Does gold have an 'ineffable' quality that hydrogen does not? Do cells, after 1bn years of evolution have a quality that gold does not? Does reality work one way, and not another? Are not all things unequal? A "machine" is what, exactly? Do we take it to be an abstraction? Or is it an electrical field oscillating over silicon? Either way, you're in trouble. Abstractions have no physical properties, and electrified sand seems hardly to possess any interesting properties. The ability for animals to adapt to their environments, by growing into them, by establishing plastic causal connections in their very bodies, grown by their environemtns... able to almost instantly move from protein expression in 1trn 1-bn-yr cellular supercomputers in each of our bodies to macro sensory-motor representation --- and back again Is this ineffable? Or is this extremely effable. Is rather, not the superstitious view that "everything is anything" ? All extant, knowing, studied intelligent systems have organic properties; and radically so. Insofar as this is "ineffable" you should take that up with the animal kingdom. I find the contrary supersitious, magical, religious, ineffable... that mere abstract patterns in arbiatrily chosen aspects of our bodies are necessary and sufficient conditions for anything at all. This would be the only case in all science. The only physical property instantiated by mere arrangement at any level. Upload our consciousness? Make it out of wood why not! Nonesense. When I am hungry, I dream of food, when I dream of food I plan to get some, and I am angry without it. This will not be made out of sand.
  - 3y ago
ncallaway 3y ago
Honestly, yours sounds more like a religious claim that the other. It seems less a religious claim to assert that the human brain is mechanistic, and simply operates using physics to perform complex operations than to assert that there is something “unknowable”, or “outside of physics” that would prevent us from being able to build a similar machine. If there were 0 examples in the universe, I think your point would be a good one. But, given there is 1 example, I think it stands to reason that there could be more. I could accept an argument that one might expect it to be hideously complicated, and not something we’ll be able to accomplish for a long time. But the claim that there’s an information theoretic reason that would absolutely prevent it, would seem to make the claim that there is something metaphysical about the operation of the brain, which seems like a quasi religious claim to me.
- axlee 3y ago
  I think one of the biggest barriers is that the human brain isn't running on binary code, and attempting to replicate it on our current technology model is doomed to encounter emulation issues.
  - ncallaway 3y ago
    That’s not an “information theoretic barrier” argument. That’s not an impossibility argument. That’s a “hideously complicated, and not something we’ll be able to accomplish for a long time” argument, which I happily concede may be the case. I see a very very large gulf between: “Impossible in all scenarios” and “totally impractical with current technology”. I was refuting only the former argument.
- mjburgess 3y ago
  Almost everything is "outside of physics", this is mainstream physics. All of physics, as formulated, is uncomputable. And what is computable is a radical approximation. Physics cannot formulate descriptions of climates, or even, many cases of turbulence, or 4 bodies in a gravitational field. Leaving "physics" is trivial, it occurs whenever you take any of the current toy models and add one layer of reality to them. The whole edifice of formal science is a little like children's block toys.
karaterobot 3y ago
Reminds me of the old line about how, if the brain were so simple we could understand it, we'd be so stupid that we couldn't. Also reminds me of an older book I read about AI (I think it was On Intelligence by Jeff Hawkins?) where I first became aware of the idea we had been scrambling to create AI without first having a good definition of intelligence or deep understanding of how it works in our own brains. And when I ask myself or other people how they define intelligence, it always comes down to some variation on "the ability to solve problems", which feels deeply beside the point and likely to never produce something that "feels" intelligent. But I don't necessarily agree that there is this special case of human intelligence that makes it impossible to understand or model. I would really like to believe it, personally, because I don't want AGI. I just don't buy that that's the explanation for our failure to do so up to now. It seems like we ought to be able to do it, but that we're muddling in the wrong direction, coming up with an exceptionally clever implementation of an approach which cannot produce intelligence that satisfies our intuition about what intelligence is. To tie it back to the article, I keyed in on the word 'design' in LeCun's statement that, "contrary to what you might hear from some people, we do not have a design for an intelligent system that would reach human intelligence." In other words, that it's not just a quantitative difference (more parameters, more data) but that a different approach than what we are taking would be necessary.
logicchains 3y ago
>we’ll never be able to model it That's completely false. Neural networks have been mathematically proven to be universal approximators (https://www.deep-mind.org/2023/03/26/the-universal-approximation-theorem/ https://www.deep-mind.org/2023/03/26/the-universal-approxima... ), i.e. a sufficiently large neural networks can approximate any given mathematical function to an arbitrary level of precision. Given any programs can be modelled as a mathematical function, neural networks can hence approximate any arbitrary program. Unless we assume something supernatural, human intelligence is just a program.
- belter 3y ago
  Not really...It has limitations. Does not specify how large the neural network must be to achieve the desired level of approximation, does not address the computational resources required, ignores non continuous functions and so on... "The Truth About the [Not So] Universal Approximation Theorem" - https://www.lifeiscomputation.com/the-truth-about-the-not-so-universal-approximation-theorem/ https://www.lifeiscomputation.com/the-truth-about-the-not-so...
foobarian 3y ago
I usually take AGI to mean what it says literally - simply an AI with very general training. I think that description already applies to the latest crop of LLM models. Taking it to mean some nebulous sentient entity with a sense of self, I doubt we'll see in our lifetimes. We don't even understand how our own "souls" work.
- idontpost 3y ago
  [dead]
- maleldil 3y ago
  > I think that description already applies to the latest crop of LLM models I think you're using a completely different definition from everyone else.
- jacquesm 3y ago
  > I usually take AGI to mean what it says literally - simply an AI with very general training. That's clever but it doesn't help to start redefining terms that are the basis for a discussion with others who use that term in a different and more generally accepted way. General doesn't mean 'generally trained' it specifically means that it does not require training for a particular task in order to figure it out. That implies that it may not require training at all and that if it is trained the training isn't necessarily general but that the AI can extract useful patterns from training on completely different subjects and apply them to the problem at hand. This is subtly but crucially importantly different from 'AI with very general training'.
  - foobarian 3y ago
    > This is subtly but crucially importantly different from 'AI with very general training'. I am not being as precise on the Internet as I would be if this was a paper perhaps, and for that I apologize :-) However, just reading the top comment chain on this story, and other discussions elsewhere, I think there is a lot of cross-talk where AGI is being confused for sapient AI. I don't get the sense that there is a definition as generally accepted as we would like.
trevyn 3y ago
Yes, people tend to regard things that they don’t personally understand as magic. I understand my own intelligence, and newsflash: AGI is already here.
ryandvm 3y ago
We have to understand it to build it? That seems absurd. Did evolution understand it? I'll grant you that maybe we should understand it before building it, but it's absolutely not a requirement. The reality is that your consciousness sits at the end of a gradient of intelligence that nature simply brute forced. Your conscious experience is more sophisticated than a dog's, which surpasses a hamster's, which surpasses a goldfish, insect, etc. There is no magic to it, there is nothing but more and more and more. We will get to AGI eventually. We probably won't understand it. We won't apply it judiciously. And we'll probably argue for decades about whether or not it's really AGI, but it will happen.
- ryandvm 3y ago
  The scariest part to me is the realization that not only will we probably crack AGI in my lifetime, if we keep throwing resources at it, it will almost certainly have a richer more sophisticated conscious experience than myself. What the hell are we supposed to do when we've created an intelligence that is better at being "human" than humans?
  - eastbound 3y ago
    Will you have the choice? More seriously, we create objects for us. We don’t have to cater for the feelings of a bigger AI. And maybe that bigger AI will have time to speak with those who need a human presence (to everyone using TV as a background noise, or watching Youtube out of solitude).
    - kaba0 3y ago
      I think the ramifications of what a true AI would bring are impossible to predict. Especially if it can improve itself, it can quickly give rise to many sci-fi stories.
adroniser 3y ago
At the very least, when compute passes a certain threshold we are just gonna be able to simulate human brains.
- dkersten 3y ago
  We have to understand our brains better first. Real neurons are substantially more complex than what even our most “biologically inspired” models are greatly simplified.
Upvoter33 3y ago
This seems highly hand wavy to me. We have biological proof that intelligence can be created, mostly through the neocortex, which in and of itself is actually a fairly simple structure repeated at great scale. Why wouldn't we be able to build this eventually, once we understand it a bit more?
- lanstin 3y ago
  90 T neurons, quadrillion interconnections, lots of glia cells with synapse altering action, plus chemical signals diffusing around and going thru the bloodstream. In principle, it is comprehensible in some sense tho not by one human, but the architecture is quite different than LLMs and we don’t know what parts of the architecture are needed for sentience. Given the gap between what genetics starts going and what we end up with by age 25, it is likely that certain routine experiences of childhood and of human social groups are also requirements for sentience.
tim333 3y ago
I think a barrier to understanding our own intelligence meaning no AGI is false. It's like saying not understanding exactly how birds fly would mean we can never build machines that will fly faster than them. We'll probably make machines that think generally and are smarter than us long before we really understand our own minds.
aantix 3y ago
Our most recent gains in AI come in terms of programming and informational retrieval efficiency. We’re accelerating.
Keyframe 3y ago
You got challenged, as if we have all the answers there are. Thing is, path to AGI and beyond inevitably leads to those hard questions from the beginning which we have zero idea about how to even define, or at least some proxy definition exists, such as what is and what isn't, what is consciousness, interplay between layers of psyche and how it translates to intelligence and everything else, biological ship of theseus, etc.. it becomes philosophical quite fast and there are more questions there than answers.
- anon84873628 3y ago
  These arguments are so bewildering to me. Intelligence is very easy to define and has nothing to do with consciousness. It is simply the ability to model an external reality and then take action according to inputs. After combining enough specialized modules together with a management layer, we will essentially have AGI. Of course it won't be human, because human intelligence is tightly coupled to human perception.
  - Keyframe 3y ago
    These arguments are so bewildering to me. My friend, that's because you stopped reading once you had your framework set in and decided to pontificate. Let me highlight the point for you, it might be easier to grasp: path to AGI and beyond inevitably leads to those hard questions... ... how it translates to intelligence and everything else there is nothing simply about it, there is nothing guaranteeing anything about it and there is everything about it once we start pursuing things AGI - AND BEYOND. Notice the keywords.
beepbooptheory 3y ago
One just has to design and build the house of the room they're trapped in.
- bongodongobob 3y ago
  That's a pretty lazy take. You could look out the windows and see the reflections of the house. You could build some kind of RF gun to probe the house. You could use sub or supersonic echolocation. You could build a camera and stick it out the window etc.
  - deleted 3y ago
    [deleted]
nobodyandproud 3y ago
If I may hazard a guess: The term and concept you’re grasping for is “emergence.” It’s an area I’m both interested in but completely unfamiliar with. I wish there were a crash-course for dummies, because I feel it’s a deep topic that we’ve only scratched the surface of; and that philosophy seems to be the most rich resource means there’s low hanging fruit to discover. https://plato.stanford.edu/entries/properties-emergent/ https://plato.stanford.edu/entries/properties-emergent/ https://link.springer.com/chapter/10.1007/978-981-15-9297-3_4 https://link.springer.com/chapter/10.1007/978-981-15-9297-3_...
layer8 3y ago
Well, we don’t even understand what “understand” means, so there’s really little basis for anything. On the other hand, we also don’t understand (and didn’t predict) how LLMs work so well, so understanding isn’t a precondition for bringing about.
gfodor 3y ago
Probably not going to pan out well https://arxiv.org/html/2401.10020v1 https://arxiv.org/html/2401.10020v1
great_psy 3y ago
Would you also have that belief about how much weight a human can lift ? Just because a human can lift 500lb, we will never build a machine that can lift more than that ? Why is it so crazy to think that we, the tool building animals that we are, could also build a machine that things better than us, just how we built a crane ?
sanroot99 3y ago
Biologically achieving agi seems easier, just clone human brain,or more evil organization can just use human captive and use there brain. As depicted in psycho pass anime.
kaba0 3y ago
Do we have to understand something to recreate it? I think that’s already a fundamentally false-y assumption. For example, many branches of science operate on a more “discover” over “invent” mindset, quite successfully.

deleted 3y ago

[deleted]

jameshart 3y ago

The idea that there’s some ‘ information-theoretic barrier for us to understand the “true nature” or “essence” of our own intelligence’ sounds more religious to me. If evolution can cross that barrier just by banging molecules together and seeing which ones work, it seems unlikely there’s some causal disconnect that makes it impossible for us to get there by thinking about it.

RcouF1uZ4gsC 3y ago
> If evolution can cross that barrier just by banging molecules together and seeing which ones work, it seems unlikely there’s some causal disconnect that makes it impossible for us to get there by thinking about it. Evolution also crossed the flight barrier by banging molecules together. I don't think banging molecules together without having an understanding of physics and the forces involved would have been a viable means for us to get to flight.
- jameshart 3y ago
  That’s… not a refutation of my point. AI/flight analogies are tired, but the OPs argument amounts to the equivalent of, before the Wright Brothers, proclaiming ‘there’s an inherent inability for humans to ever conceive of a way to engineer heavier than air flight’. It’s a ‘man was never mean to fly, therefore heavier than air flight is impossible’ argument.
- shawnz 3y ago
  But wasn't it through experimentation that we developed that understanding? And in the earliest phases of that experimentation, that's when our understanding was at its weakest, right? You almost seem to be implying that we had the understanding before ever attempting the experiments.
mejutoco 3y ago
I dont have a strong opinion about it, but when I read parents post I immediately thought of Gödel s works and how there are limits of what you can know within a system (and in Mathematics no less). Thinking that something similar exists in intelligence does not seem so far fetched. As a hypothesis, of course.
- jameshart 3y ago
  And yet we can use mathematics to know that that limit exists. It’s a powerful thing, mathematics. Gödel doesn’t say ‘mathematics cannot contemplate itself’. Quite the opposite.
  - mejutoco 3y ago
    It is an analogy to mention that is not a far fetched idea that there are limits. As far as I recall, one of the results of Gödel is that a system capable of Arithmetic cannot prove certain things about itself. Aka there are limits within a system. My claim was about limits existing.
- ncallaway 3y ago
  The reason it seems far fetched is we have one example of a mechanistic, physics based machine that we know can perform the computations necessary to produce intelligence. I could understand the hypothesis that AGI is not computable, if we didn’t have an existing example of a machine that can produce AGI. Since we do have an existing machine that can produce AGI, we would have to suppose it does something: - outside of physics to achieve its results - performs some operation that is impossible for us to understand or replicate Both of those seem… unlikely to me.
  - jprete 3y ago
    It seems like an article of faith to believe that the universe is a computer in the first place. Without that assumption, we don't even know that intelligence is computation.
paganel 3y ago
This is a very mechanic-ist way of looking at things.
- anon84873628 3y ago
  Yes, which seems to be the way the universe works. Duality is a religious belief.
  - paganel 3y ago
    As far as I know we haven't figured it out how the universe works, hence the unified theory debacle and all that.
- ncallaway 3y ago
  Sure, and we have pretty good evidence of the mechanistic operation of the universe from quantum mechanics up, and we haven’t found any evidence of anything outside of that. So, to date, a mechanistic view of the universe seems like one that is the most supported be evidence at the moment. Have we ruled out meta-physics or some kind of duality? No, of course we haven’t. As long as there’s unknown physics, it’s a possibility. But I think if you’re reasoning based on the evidence we’ve been able to collect to date, then you should absolutely favor a mechanistic universe to a dual universe.

HuShifang 3y ago

Not an endorsement, but here's an argument that AGI is computationally intractable (edit: at least via ML). https://irisvanrooijcogsci.com/2023/09/17/debunking-agi-inevitability-claims/ https://irisvanrooijcogsci.com/2023/09/17/debunking-agi-inev...

ceejayoz 3y ago
"AI is impossible" is a tough argument, given the existence of non-A I. "Machine learning isn't the route to AI" is something you could more reasonably argue, but that's a drastically narrower claim.
jameshart 3y ago
Intractable and impossible are very different claims. NP Hardness is a statement about the asymptotic difficulty of solving a problem at ever larger scales. It says ‘if you have a way to solve this problem at size n, that way will scale worse than polynomially when you try to apply it to a problem at size 2n’. Which might place limits on the practicable scale of how big an n your approach gets to work for. But if your approach works practically for a big enough n to make AGI then your approach works - NP Hardness doesn’t matter. And since we know that finite mass lumps of finite numbers of gray cells are capable of GI, we have a reasonable expectation that there is some n for which AGI might be possible.

johnwheeler 3y ago

Yes, If you look at the power efficiency of the brain relative to LLMs, you can see nature has done something special, and I don’t think we should be so quick to call it game over. It might be that LLMs are just some atom or building block of a much larger and more complex thing.

jofla_net 3y ago
Yes, this is huge. Of all the places to shim in my ¢2, as the topic could go on forever. I remember talking to a robotics specialist and his take on efficiency was that despite the obvious speed and strength that machines have over us, you need to factor in just how little power is consumed when we move our extremities compared to what is needed for actuators and motors of even the best bots. Now this was years ago and things could have taken a huge turn, but I think given the recent statements by Altman about the looming power crush drives it home, that even if we do manage something truly monumental, it'll take a fusion reaction to pull off.
- kaba0 3y ago
  I mean, I wouldn’t bet on that either. We built the “computing” empire on silicon, but we might be able to very significantly improve on this with a single discovery/invention. All the computational stuff we built/figured out will move out without trouble.
  - johnwheeler 3y ago
    Yes, that might be too. You just never know.
- johnwheeler 3y ago
  Maybe Berkshire Hathaway Energy is the smart play

Lerc 3y ago

As a counterpoint, I feel like the belief that we can't get to AGI comes off as religion. It presupposes an ineffable quality that we posses that machines cannot. The argument that we have to fully understand something to build it doesn't hold water for me. I have made plenty of things where understanding the depths of what I had made took far more time and effort than building the thing in the first place. It's always hard to predict the rate of progress, Most of the current optimism comes from how radically wrong predictions were for the capabilities of AI today. 10 years ago a lot of people would have put current AI capability as arriving well after 2050. The jump in progress may not be sustained, but it definitely places doubt on people confidently predicting slow progress.

mjburgess 3y ago
Does gold have an 'ineffable' quality that hydrogen does not? Do cells, after 1bn years of evolution have a quality that gold does not? Does reality work one way, and not another? Are not all things unequal? A "machine" is what, exactly? Do we take it to be an abstraction? Or is it an electrical field oscillating over silicon? Either way, you're in trouble. Abstractions have no physical properties, and electrified sand seems hardly to possess any interesting properties. The ability for animals to adapt to their environments, by growing into them, by establishing plastic causal connections in their very bodies, grown by their environemtns... able to almost instantly move from protein expression in 1trn 1-bn-yr cellular supercomputers in each of our bodies to macro sensory-motor representation --- and back again Is this ineffable? Or is this extremely effable. Is rather, not the superstitious view that "everything is anything" ? All extant, knowing, studied intelligent systems have organic properties; and radically so. Insofar as this is "ineffable" you should take that up with the animal kingdom. I find the contrary supersitious, magical, religious, ineffable... that mere abstract patterns in arbiatrily chosen aspects of our bodies are necessary and sufficient conditions for anything at all. This would be the only case in all science. The only physical property instantiated by mere arrangement at any level. Upload our consciousness? Make it out of wood why not! Nonesense. When I am hungry, I dream of food, when I dream of food I plan to get some, and I am angry without it. This will not be made out of sand.
- 3y ago

ncallaway 3y ago

Honestly, yours sounds more like a religious claim that the other. It seems less a religious claim to assert that the human brain is mechanistic, and simply operates using physics to perform complex operations than to assert that there is something “unknowable”, or “outside of physics” that would prevent us from being able to build a similar machine. If there were 0 examples in the universe, I think your point would be a good one. But, given there is 1 example, I think it stands to reason that there could be more. I could accept an argument that one might expect it to be hideously complicated, and not something we’ll be able to accomplish for a long time. But the claim that there’s an information theoretic reason that would absolutely prevent it, would seem to make the claim that there is something metaphysical about the operation of the brain, which seems like a quasi religious claim to me.

axlee 3y ago
I think one of the biggest barriers is that the human brain isn't running on binary code, and attempting to replicate it on our current technology model is doomed to encounter emulation issues.
- ncallaway 3y ago
  That’s not an “information theoretic barrier” argument. That’s not an impossibility argument. That’s a “hideously complicated, and not something we’ll be able to accomplish for a long time” argument, which I happily concede may be the case. I see a very very large gulf between: “Impossible in all scenarios” and “totally impractical with current technology”. I was refuting only the former argument.
mjburgess 3y ago
Almost everything is "outside of physics", this is mainstream physics. All of physics, as formulated, is uncomputable. And what is computable is a radical approximation. Physics cannot formulate descriptions of climates, or even, many cases of turbulence, or 4 bodies in a gravitational field. Leaving "physics" is trivial, it occurs whenever you take any of the current toy models and add one layer of reality to them. The whole edifice of formal science is a little like children's block toys.

karaterobot 3y ago

Reminds me of the old line about how, if the brain were so simple we could understand it, we'd be so stupid that we couldn't. Also reminds me of an older book I read about AI (I think it was On Intelligence by Jeff Hawkins?) where I first became aware of the idea we had been scrambling to create AI without first having a good definition of intelligence or deep understanding of how it works in our own brains. And when I ask myself or other people how they define intelligence, it always comes down to some variation on "the ability to solve problems", which feels deeply beside the point and likely to never produce something that "feels" intelligent. But I don't necessarily agree that there is this special case of human intelligence that makes it impossible to understand or model. I would really like to believe it, personally, because I don't want AGI. I just don't buy that that's the explanation for our failure to do so up to now. It seems like we ought to be able to do it, but that we're muddling in the wrong direction, coming up with an exceptionally clever implementation of an approach which cannot produce intelligence that satisfies our intuition about what intelligence is. To tie it back to the article, I keyed in on the word 'design' in LeCun's statement that, "contrary to what you might hear from some people, we do not have a design for an intelligent system that would reach human intelligence." In other words, that it's not just a quantitative difference (more parameters, more data) but that a different approach than what we are taking would be necessary.

logicchains 3y ago

>we’ll never be able to model it That's completely false. Neural networks have been mathematically proven to be universal approximators (https://www.deep-mind.org/2023/03/26/the-universal-approximation-theorem/ https://www.deep-mind.org/2023/03/26/the-universal-approxima... ), i.e. a sufficiently large neural networks can approximate any given mathematical function to an arbitrary level of precision. Given any programs can be modelled as a mathematical function, neural networks can hence approximate any arbitrary program. Unless we assume something supernatural, human intelligence is just a program.

belter 3y ago
Not really...It has limitations. Does not specify how large the neural network must be to achieve the desired level of approximation, does not address the computational resources required, ignores non continuous functions and so on... "The Truth About the [Not So] Universal Approximation Theorem" - https://www.lifeiscomputation.com/the-truth-about-the-not-so-universal-approximation-theorem/ https://www.lifeiscomputation.com/the-truth-about-the-not-so...

foobarian 3y ago

I usually take AGI to mean what it says literally - simply an AI with very general training. I think that description already applies to the latest crop of LLM models. Taking it to mean some nebulous sentient entity with a sense of self, I doubt we'll see in our lifetimes. We don't even understand how our own "souls" work.

idontpost 3y ago
[dead]
maleldil 3y ago
> I think that description already applies to the latest crop of LLM models I think you're using a completely different definition from everyone else.
jacquesm 3y ago
> I usually take AGI to mean what it says literally - simply an AI with very general training. That's clever but it doesn't help to start redefining terms that are the basis for a discussion with others who use that term in a different and more generally accepted way. General doesn't mean 'generally trained' it specifically means that it does not require training for a particular task in order to figure it out. That implies that it may not require training at all and that if it is trained the training isn't necessarily general but that the AI can extract useful patterns from training on completely different subjects and apply them to the problem at hand. This is subtly but crucially importantly different from 'AI with very general training'.
- foobarian 3y ago
  > This is subtly but crucially importantly different from 'AI with very general training'. I am not being as precise on the Internet as I would be if this was a paper perhaps, and for that I apologize :-) However, just reading the top comment chain on this story, and other discussions elsewhere, I think there is a lot of cross-talk where AGI is being confused for sapient AI. I don't get the sense that there is a definition as generally accepted as we would like.

trevyn 3y ago

Yes, people tend to regard things that they don’t personally understand as magic. I understand my own intelligence, and newsflash: AGI is already here.

ryandvm 3y ago

We have to understand it to build it? That seems absurd. Did evolution understand it? I'll grant you that maybe we should understand it before building it, but it's absolutely not a requirement. The reality is that your consciousness sits at the end of a gradient of intelligence that nature simply brute forced. Your conscious experience is more sophisticated than a dog's, which surpasses a hamster's, which surpasses a goldfish, insect, etc. There is no magic to it, there is nothing but more and more and more. We will get to AGI eventually. We probably won't understand it. We won't apply it judiciously. And we'll probably argue for decades about whether or not it's really AGI, but it will happen.

ryandvm 3y ago
The scariest part to me is the realization that not only will we probably crack AGI in my lifetime, if we keep throwing resources at it, it will almost certainly have a richer more sophisticated conscious experience than myself. What the hell are we supposed to do when we've created an intelligence that is better at being "human" than humans?
- eastbound 3y ago
  Will you have the choice? More seriously, we create objects for us. We don’t have to cater for the feelings of a bigger AI. And maybe that bigger AI will have time to speak with those who need a human presence (to everyone using TV as a background noise, or watching Youtube out of solitude).
  - kaba0 3y ago
    I think the ramifications of what a true AI would bring are impossible to predict. Especially if it can improve itself, it can quickly give rise to many sci-fi stories.

adroniser 3y ago

At the very least, when compute passes a certain threshold we are just gonna be able to simulate human brains.

dkersten 3y ago
We have to understand our brains better first. Real neurons are substantially more complex than what even our most “biologically inspired” models are greatly simplified.

Upvoter33 3y ago

This seems highly hand wavy to me. We have biological proof that intelligence can be created, mostly through the neocortex, which in and of itself is actually a fairly simple structure repeated at great scale. Why wouldn't we be able to build this eventually, once we understand it a bit more?

lanstin 3y ago
90 T neurons, quadrillion interconnections, lots of glia cells with synapse altering action, plus chemical signals diffusing around and going thru the bloodstream. In principle, it is comprehensible in some sense tho not by one human, but the architecture is quite different than LLMs and we don’t know what parts of the architecture are needed for sentience. Given the gap between what genetics starts going and what we end up with by age 25, it is likely that certain routine experiences of childhood and of human social groups are also requirements for sentience.

tim333 3y ago

I think a barrier to understanding our own intelligence meaning no AGI is false. It's like saying not understanding exactly how birds fly would mean we can never build machines that will fly faster than them. We'll probably make machines that think generally and are smarter than us long before we really understand our own minds.

aantix 3y ago

Our most recent gains in AI come in terms of programming and informational retrieval efficiency. We’re accelerating.

Keyframe 3y ago

You got challenged, as if we have all the answers there are. Thing is, path to AGI and beyond inevitably leads to those hard questions from the beginning which we have zero idea about how to even define, or at least some proxy definition exists, such as what is and what isn't, what is consciousness, interplay between layers of psyche and how it translates to intelligence and everything else, biological ship of theseus, etc.. it becomes philosophical quite fast and there are more questions there than answers.

anon84873628 3y ago
These arguments are so bewildering to me. Intelligence is very easy to define and has nothing to do with consciousness. It is simply the ability to model an external reality and then take action according to inputs. After combining enough specialized modules together with a management layer, we will essentially have AGI. Of course it won't be human, because human intelligence is tightly coupled to human perception.
- Keyframe 3y ago
  These arguments are so bewildering to me. My friend, that's because you stopped reading once you had your framework set in and decided to pontificate. Let me highlight the point for you, it might be easier to grasp: path to AGI and beyond inevitably leads to those hard questions... ... how it translates to intelligence and everything else there is nothing simply about it, there is nothing guaranteeing anything about it and there is everything about it once we start pursuing things AGI - AND BEYOND. Notice the keywords.

beepbooptheory 3y ago

One just has to design and build the house of the room they're trapped in.

bongodongobob 3y ago
That's a pretty lazy take. You could look out the windows and see the reflections of the house. You could build some kind of RF gun to probe the house. You could use sub or supersonic echolocation. You could build a camera and stick it out the window etc.
- deleted 3y ago
  [deleted]

nobodyandproud 3y ago

If I may hazard a guess: The term and concept you’re grasping for is “emergence.” It’s an area I’m both interested in but completely unfamiliar with. I wish there were a crash-course for dummies, because I feel it’s a deep topic that we’ve only scratched the surface of; and that philosophy seems to be the most rich resource means there’s low hanging fruit to discover. https://plato.stanford.edu/entries/properties-emergent/ https://plato.stanford.edu/entries/properties-emergent/ https://link.springer.com/chapter/10.1007/978-981-15-9297-3_4 https://link.springer.com/chapter/10.1007/978-981-15-9297-3_...

layer8 3y ago

Well, we don’t even understand what “understand” means, so there’s really little basis for anything. On the other hand, we also don’t understand (and didn’t predict) how LLMs work so well, so understanding isn’t a precondition for bringing about.

gfodor 3y ago

Probably not going to pan out well https://arxiv.org/html/2401.10020v1 https://arxiv.org/html/2401.10020v1

great_psy 3y ago

Would you also have that belief about how much weight a human can lift ? Just because a human can lift 500lb, we will never build a machine that can lift more than that ? Why is it so crazy to think that we, the tool building animals that we are, could also build a machine that things better than us, just how we built a crane ?

sanroot99 3y ago

Biologically achieving agi seems easier, just clone human brain,or more evil organization can just use human captive and use there brain. As depicted in psycho pass anime.

kaba0 3y ago

Do we have to understand something to recreate it? I think that’s already a fundamentally false-y assumption. For example, many branches of science operate on a more “discover” over “invent” mindset, quite successfully.