5 ms·

We don't know if AGI is even possible outside of a biological construct yet. This is key. Can we land on AGI without some clear indication of possibility (aka C

We don't know if AGI is even possible outside of a biological construct yet. This is key. Can we land on AGI without some clear indication of possibility (aka Chappie style)? Possibly, but the likelihood is low. Quite low. It's essentially groping in the dark. A good contrast is quantum computing. We know that's possible, even feasible, and now are trying to overcome the engineering hurdles. And people still think that's vaporware.

sixo 1y ago
On the contrary, we have one working example of general intelligence (humans) and zero of quantum computing.
- deleted 1y ago
  [deleted]
- bee_rider 1y ago
  Do we have a specific enough definition of general intelligence that we can exclude all non-human animals?
  - mattnewton 1y ago
    Why does it need to exclude all non human animals? Could it not be a difference of degree rather than of kind?
    - bee_rider 1y ago
      The post I was responding to had > On the contrary, we have one working example of general intelligence (humans) I think some animals probably have what most people would informally call general intelligence, but maybe there’s some technical definition that makes me wrong.
      - mitthrowaway2 1y ago
        Their point is not in any way weakened if you read "one working example" as "at least one working example".
        bee_rider 1y ago
        Oh, good point, I hadn’t noticed the alternative reading. That makes sense, then.
      - dsubburam 1y ago
        I do not know how "general intelligence" is defined, but there are a set of features we humans have that other animals mostly don't, as per the philosopher Roger Scruton[1], that I am reproducing from memory (errors mine): 1. Animals have desires, but do not make choices We can choose to do what we do not desire, and choose not to do what we desire. For animals, one does not need to make this distinction to explain their behavior (Occam's razor)--they simply do what they desire. 2. Animals "live in a world of perception" (Schopenhauer) They only engage with things as they are. They do not reminisce about the past, plan for the future, or fantasize about the impossible. They do not ask "what if?" or "why?". They lack imagination. 3. Animals do not have the higher emotions that require a conceptual repertoire such as regret, gratitude, shame, pride, guilt, etc. 4. Animals do not form complex relationships with others Because it requires the higher emotions like gratitude and resentment, and concepts such as rights and responsibilities. 5. Animals do not get art or music We can pay disinterested attention to a work of art (or nature) for its own sake, taking pleasure from the exercise of our rational faculties thereof. 6. Animals do not laugh I do not know if the science/philosophy of laughter is settled, but it appears to me to be some kind of phenomenon that depends on civil society. 7. Animals lack language in the full sense of being able to engage in reason-giving dialogue with others, justifying your actions and explaining your intentions. Scruton believed that all of the above arise together. I know this is perhaps a little OT, but I seldom if ever see these issues mentioned in discussions about AGI. Maybe less applicable to super-intelligence, but certainly applicable to the "artificial human" part of the equation. [1] Philosophy: Principles and Problems. Roger Scruton
      - jibal 1y ago
    - Fricken 1y ago
      We're fixated on human intelligence but a computer cannot even emulate the intelligence of a honeybee or an ant.
      - fastball 1y ago
        How do you mean? AFAICT computers can definitely do that. Sure, it won't be the size of an ant, but we definitely have models running on computers that have much more complexity than the life of an ant.
        Fricken 1y ago
        Having aptitude in mathematics was once considered the highest form of human intelligence, yet a simple pocket calculator can beat the pants off most humans at arithmetic tasks. Conversely, something we regard as simple, such as selecting a key from a keychain and using to unlock a door not previously encountered is beyond the current abilities of any machine. I suspect you might be underestimating the real complexity of what bees and ants do. Self-driving cars as well seemed like a simpler problem before concerted efforts were made to build one.
        dragonwriter 1y ago
        > Having aptitude in mathematics was once considered the highest form of human intelligence, yet a simple pocket calculator can beat the pants off most humans at arithmetic tasks. Mathematics has been a lot more than arithmetic for... a very long time.
        Jensson 1y ago
        But arithmetics was seen as requiring intelligence, as did chess.
        Jensson 1y ago
        > Sure, it won't be the size of an ant, but we definitely have models running on computers that have much more complexity than the life of an ant. Do we? Where is the model that can run an ant and navigate a 3d environment, parse visuals and different senses to orient itself, figure out where it can climb to get to where it needs to go. Then put that in an average forest and navigate trees and other insects and try to cooperate with other ants and find its way back. Or build an anthill, an ant can build an anthill, full of tunnels everywhere that doesn't collapse without using a plan. Do we have such a model? I don't think we have anything that can do that yet. Waymo is trying to solve a much simpler problem and they still struggle, so I am pretty sure we still can't run anything even remotely as complex as an ant. Maybe a simple worm, but not an ant.
  - jibal 1y ago
    No one said "exclusively humans", and that's not relevant.
- glitchc 1y ago
  That's covered in the biological construct part. And no, we definitely do have quantum computers. They're just not practical yet.
- adastra22 1y ago
  There are many working quantum computers…
  - sixo 1y ago
    ah, I mean, working in the sense of OP: that a system which overcomes the "engineering hurdles" is actually feasible and will be successful. To be blocked merely by "engineering hurdles" puts QC in approximately the same place as fusion.
    - adastra22 1y ago
      There are working quantum computers that are not only feasible, but exist, can be rented on the cloud, and are people pay money to use. Whether these are a commercial success at this point in time is missing the forest for the trees. A LOT of money has been put into getting as far as we have, and the limited market for using these machines at the moment means that getting a return on investment right now is difficult. But this is/has been true of every new technology. And quantum computers are getting better & more energy efficient year-by-year.
tshaddox 1y ago
> We don't know if AGI is even possible outside of a biological construct yet. This is key. A discovery that AGI is impossible in principle to implement in an electronic computer would require a major fundamental discovery in physics that answers the question “what is the brain doing in order to implement general intelligence?”
- epiccoleman 1y ago
  Would that really be a physics discovery? I mean I guess everything ultimately is. But it seems like maybe consciousness could be understood in terms of "higher level" sciences - somewhere on the chain of neurology->biology->chemistry->physics.
  - mmoskal 1y ago
    Consciousness (subjective experience) is possibly orthogonal to intelligence (ability to achieve complex goals). We definitely have a better handle on what intelligence is than consciousness.
    - epiccoleman 1y ago
      That does make sense, reminds me of Blindsight, where one central idea is that conscious experience might not even be necessary for intelligence (and possibly even maladaptive).
  - marcosdumay 1y ago
    > Would that really be a physics discovery? No, it could be something that proves all of our fundamental mathematics wrong. The GP just gave the more conservative option.
    - tshaddox 1y ago
      I’m not sure what you mean. This new discovery in mathematics would also necessarily tell us something new about what is computable, which is physics.
singpolyma3 1y ago
This makes no sense. If you believe in eg a mind or soul then maybe it's possible we cannot make AGI. But if we are purely biological then obviously it's possible to replicate that in principle.
- DrewADesign 1y ago
  That doesn’t contradict what they said. We may one day design a biological computing system that is capable of it. We don’t entirely understand how neurons work; it’s reasonable to posit that the differences that many AGI boosters assert don’t matter do matter— just not in ways we’ve discovered yet.
  - kelnos 1y ago
    I mentioned this in another thread, but I do wonder if we engineer a sort of biological computer, will it really be a computer at all, and not a new kind of life itself?
    - jakeydus 1y ago
      > not a new kind of life itself? In my opinion, this is more a philosophical question than an engineering one. Is something alive because it’s conscious? Is it alive because it’s intelligent? Is a virus alive, or a bacteria, or an LLM? Beats me.
    - DrewADesign 1y ago
      Maybe — though we’d still have engineered it, which is the point I was trying to make.
  - slashdave 1y ago
    We understand how neurons work to quite a bit of detail.
    - DrewADesign 1y ago
      The Allen Institute doesn’t seem to think so. We don’t even know how the brain of a roundworm ticks and it’s only got 302 neurons— all of which are mapped, along with their connections.
root_axis 1y ago
The practical feasibility of quantum computing is definitely still an open research question.
deleted 1y ago
[deleted]
jibal 1y ago
It's not "key"; it's not even relevant ... the proof will be in the pudding. Proving a priori that some outcome is possible plays no role in achieving it. And you slid, motte-and-bailey-like, from "know" to "some clear indication of possibility" -- we have extremely clear indications that it's possible, since there's no reason other than a belief in magic to think that "biological" is a necessity. Whether is feasible or practical or desirable to achieve AGI is another matter, but the OP lays out multiple problem areas to tackle.
slashdave 1y ago
> We don't know if AGI is even possible outside of a biological construct yet Of course it is. A brain is just a machine like any other.
- glitchc 1y ago
  Except we don't understand how the brain actually works and have yet to build a machine that behaves like it.
DrewADesign 1y ago
Sometimes I think we’re like cats that learned how to make mirrors without really understanding them, and are so close to making one good enough that the other cat becomes sentient.

sixo 1y ago

On the contrary, we have one working example of general intelligence (humans) and zero of quantum computing.

deleted 1y ago
[deleted]
bee_rider 1y ago
Do we have a specific enough definition of general intelligence that we can exclude all non-human animals?
- mattnewton 1y ago
  Why does it need to exclude all non human animals? Could it not be a difference of degree rather than of kind?
  - bee_rider 1y ago
    The post I was responding to had > On the contrary, we have one working example of general intelligence (humans) I think some animals probably have what most people would informally call general intelligence, but maybe there’s some technical definition that makes me wrong.
    - mitthrowaway2 1y ago
      Their point is not in any way weakened if you read "one working example" as "at least one working example".
      - bee_rider 1y ago
        Oh, good point, I hadn’t noticed the alternative reading. That makes sense, then.
    - dsubburam 1y ago
      I do not know how "general intelligence" is defined, but there are a set of features we humans have that other animals mostly don't, as per the philosopher Roger Scruton[1], that I am reproducing from memory (errors mine): 1. Animals have desires, but do not make choices We can choose to do what we do not desire, and choose not to do what we desire. For animals, one does not need to make this distinction to explain their behavior (Occam's razor)--they simply do what they desire. 2. Animals "live in a world of perception" (Schopenhauer) They only engage with things as they are. They do not reminisce about the past, plan for the future, or fantasize about the impossible. They do not ask "what if?" or "why?". They lack imagination. 3. Animals do not have the higher emotions that require a conceptual repertoire such as regret, gratitude, shame, pride, guilt, etc. 4. Animals do not form complex relationships with others Because it requires the higher emotions like gratitude and resentment, and concepts such as rights and responsibilities. 5. Animals do not get art or music We can pay disinterested attention to a work of art (or nature) for its own sake, taking pleasure from the exercise of our rational faculties thereof. 6. Animals do not laugh I do not know if the science/philosophy of laughter is settled, but it appears to me to be some kind of phenomenon that depends on civil society. 7. Animals lack language in the full sense of being able to engage in reason-giving dialogue with others, justifying your actions and explaining your intentions. Scruton believed that all of the above arise together. I know this is perhaps a little OT, but I seldom if ever see these issues mentioned in discussions about AGI. Maybe less applicable to super-intelligence, but certainly applicable to the "artificial human" part of the equation. [1] Philosophy: Principles and Problems. Roger Scruton
    - jibal 1y ago
  - Fricken 1y ago
    We're fixated on human intelligence but a computer cannot even emulate the intelligence of a honeybee or an ant.
    - fastball 1y ago
      How do you mean? AFAICT computers can definitely do that. Sure, it won't be the size of an ant, but we definitely have models running on computers that have much more complexity than the life of an ant.
      - Fricken 1y ago
        Having aptitude in mathematics was once considered the highest form of human intelligence, yet a simple pocket calculator can beat the pants off most humans at arithmetic tasks. Conversely, something we regard as simple, such as selecting a key from a keychain and using to unlock a door not previously encountered is beyond the current abilities of any machine. I suspect you might be underestimating the real complexity of what bees and ants do. Self-driving cars as well seemed like a simpler problem before concerted efforts were made to build one.
        dragonwriter 1y ago
        > Having aptitude in mathematics was once considered the highest form of human intelligence, yet a simple pocket calculator can beat the pants off most humans at arithmetic tasks. Mathematics has been a lot more than arithmetic for... a very long time.
        Jensson 1y ago
        But arithmetics was seen as requiring intelligence, as did chess.
      - Jensson 1y ago
        > Sure, it won't be the size of an ant, but we definitely have models running on computers that have much more complexity than the life of an ant. Do we? Where is the model that can run an ant and navigate a 3d environment, parse visuals and different senses to orient itself, figure out where it can climb to get to where it needs to go. Then put that in an average forest and navigate trees and other insects and try to cooperate with other ants and find its way back. Or build an anthill, an ant can build an anthill, full of tunnels everywhere that doesn't collapse without using a plan. Do we have such a model? I don't think we have anything that can do that yet. Waymo is trying to solve a much simpler problem and they still struggle, so I am pretty sure we still can't run anything even remotely as complex as an ant. Maybe a simple worm, but not an ant.
- jibal 1y ago
  No one said "exclusively humans", and that's not relevant.
glitchc 1y ago
That's covered in the biological construct part. And no, we definitely do have quantum computers. They're just not practical yet.
adastra22 1y ago
There are many working quantum computers…
- sixo 1y ago
  ah, I mean, working in the sense of OP: that a system which overcomes the "engineering hurdles" is actually feasible and will be successful. To be blocked merely by "engineering hurdles" puts QC in approximately the same place as fusion.
  - adastra22 1y ago
    There are working quantum computers that are not only feasible, but exist, can be rented on the cloud, and are people pay money to use. Whether these are a commercial success at this point in time is missing the forest for the trees. A LOT of money has been put into getting as far as we have, and the limited market for using these machines at the moment means that getting a return on investment right now is difficult. But this is/has been true of every new technology. And quantum computers are getting better & more energy efficient year-by-year.

tshaddox 1y ago

> We don't know if AGI is even possible outside of a biological construct yet. This is key. A discovery that AGI is impossible in principle to implement in an electronic computer would require a major fundamental discovery in physics that answers the question “what is the brain doing in order to implement general intelligence?”

epiccoleman 1y ago
Would that really be a physics discovery? I mean I guess everything ultimately is. But it seems like maybe consciousness could be understood in terms of "higher level" sciences - somewhere on the chain of neurology->biology->chemistry->physics.
- mmoskal 1y ago
  Consciousness (subjective experience) is possibly orthogonal to intelligence (ability to achieve complex goals). We definitely have a better handle on what intelligence is than consciousness.
  - epiccoleman 1y ago
    That does make sense, reminds me of Blindsight, where one central idea is that conscious experience might not even be necessary for intelligence (and possibly even maladaptive).
- marcosdumay 1y ago
  > Would that really be a physics discovery? No, it could be something that proves all of our fundamental mathematics wrong. The GP just gave the more conservative option.
  - tshaddox 1y ago
    I’m not sure what you mean. This new discovery in mathematics would also necessarily tell us something new about what is computable, which is physics.

singpolyma3 1y ago

This makes no sense. If you believe in eg a mind or soul then maybe it's possible we cannot make AGI. But if we are purely biological then obviously it's possible to replicate that in principle.

DrewADesign 1y ago
That doesn’t contradict what they said. We may one day design a biological computing system that is capable of it. We don’t entirely understand how neurons work; it’s reasonable to posit that the differences that many AGI boosters assert don’t matter do matter— just not in ways we’ve discovered yet.
- kelnos 1y ago
  I mentioned this in another thread, but I do wonder if we engineer a sort of biological computer, will it really be a computer at all, and not a new kind of life itself?
  - jakeydus 1y ago
    > not a new kind of life itself? In my opinion, this is more a philosophical question than an engineering one. Is something alive because it’s conscious? Is it alive because it’s intelligent? Is a virus alive, or a bacteria, or an LLM? Beats me.
  - DrewADesign 1y ago
    Maybe — though we’d still have engineered it, which is the point I was trying to make.
- slashdave 1y ago
  We understand how neurons work to quite a bit of detail.
  - DrewADesign 1y ago
    The Allen Institute doesn’t seem to think so. We don’t even know how the brain of a roundworm ticks and it’s only got 302 neurons— all of which are mapped, along with their connections.

root_axis 1y ago

The practical feasibility of quantum computing is definitely still an open research question.

deleted 1y ago

[deleted]

jibal 1y ago

It's not "key"; it's not even relevant ... the proof will be in the pudding. Proving a priori that some outcome is possible plays no role in achieving it. And you slid, motte-and-bailey-like, from "know" to "some clear indication of possibility" -- we have extremely clear indications that it's possible, since there's no reason other than a belief in magic to think that "biological" is a necessity. Whether is feasible or practical or desirable to achieve AGI is another matter, but the OP lays out multiple problem areas to tackle.

slashdave 1y ago

> We don't know if AGI is even possible outside of a biological construct yet Of course it is. A brain is just a machine like any other.

glitchc 1y ago
Except we don't understand how the brain actually works and have yet to build a machine that behaves like it.

DrewADesign 1y ago

Sometimes I think we’re like cats that learned how to make mirrors without really understanding them, and are so close to making one good enough that the other cat becomes sentient.