6 ms·

I’m not so sure about that. It’s a long way from a machine intelligence in your computer to one that is even as self sufficient as a cockroach with regard to n

I’m not so sure about that. It’s a long way from a machine intelligence in your computer to one that is even as self sufficient as a cockroach with regard to not getting stuck on stuff in the real world, constantly having energy, able to repair itself in the physical world. I think people hand wave these things away and they are very huge hurdles. Even something as simple as a car can’t run a decade autonomously. We expect it to explore the stars, make robots that can remove circuit boards, troubleshoot, etc.? Nothing around me keeps working without human interaction for even very small timescales.

knodi123 4y ago
> It’s a long way from a machine intelligence in your computer to one that is even as self sufficient as a cockroach Ah, but the trick is, as soon as you can get a machine intelligence smart enough to work on programming tasks, you can get it to help you design a better one. And so on, etc. The first AGI is (some) amount of time away. A more sophisticated AGI is coming not long after that, and exponentially faster and so on.
- asdff 4y ago
  Where I get hung up on with this is how the AGI knows what a more sophisticated AGI would look like without being at least as sophisticated? For example with our own intelligence, we aren't getting smarter with each generation. We have less brain matter than humans who lived millions of years ago on average, because there isn't as strong of a selective pressure present these days to see those with a lot of brain matter out reproduce those who get by in life with not as much of it. A very 'intelligent' AGI might realize the most optimal way to seed the planet with itself is actually to have a swarm of very unintelligent AI that is able to run on some very non resource intensive hardware, and seed it widely like how an "unintelligent" insect is able to lay millions of eggs and infest an area beyond human control, and the more intelligent wolf is easily hunted down.
  - pencilguin 4y ago
    We do not, in fact, have less brain than hominins millions of years ago. We do have less than our immediate ancestors 100,000 years ago. Your very intelligent AGI would most likely have no interest in seeding anything on Earth. It might reasonably be curious about our ecosystems, but disturbing those would reduce that interest.
  - naasking 4y ago
    > Where I get hung up on with this is how the AGI knows what a more sophisticated AGI would look like without being at least as sophisticated I'm not sure what's difficult about this. Are we not able to test what humans can run the fastest without being able to run just as fast? We can equally test which humans are the best spellers, or the best chess players, or the best problem solvers.
    - asdff 4y ago
      We can test these things that we cannot ourselves do, only because we are able to define them and comprehend the definitions. We define a runner as fast when they run a fixed distance in a certain time. Even though we can't run as fast, the understanding of the rules at play are within what we can understand. OTOH when you move past these examples into more abstract ideas of arbitrary rules defining arbitrary tasks, A basic AGI might not be able to even comprehend the full set of rules that define the expected tasks of a much more complicated AGI, much like how it wouldn't do you any good trying to explain the rules of baseball to an ant.
      - google234123 4y ago
        Yet we invent physics that wasn’t comprehended before it was discovered… not sure what your hangup is. We wouldnt stop trying to improve the AGI when it’s at the level of an ant.
      - naasking 4y ago
        No person or AI needs to actually understand how an AGI actually works, they just need observable behaviours indicative of intelligence. That's what we already do with image recognition, language translation tasks, and more. Once we move to more general forms of intelligence, like math and other forms of problem solving, then you apply more general tests. For instance, if we had an Einstein AI v1.0 that could infer General Relativity using X0 bits of information in time T0, and Einstein AI v2.0 managed to infer GR using X1 << X0 bits of information and/or could infer it in time T1 << T0, then v2.0 is clearly superior to v1.0. I agree that sometimes the parameters for such tests aren't always clear at first, but this has always been the case in science, which is we we iterate and progressively refine our tests as we learn. Any true AGI must be capable of such abductive and inductive reasoning to qualify as a general intelligence.
- pasquinelli 4y ago
  > The first AGI is (some) amount of time away. you jump in the air at high noon. the sun is (some) distance away, and for some time, you're getting closer to it. if we make an ai that's able to make a better ai than we are, then it does stand to reason that it'll kick off exponential growth for some time, but it'll reach limits just like anything. those limits may fall short of magicing/smarting away the obstacles to colonizing space. and we'll reach limits too. it might be that the amount of time between now and when we make a sufficient ai is more time than we have.
  - unsupp0rted 4y ago
    A rabbit jumps toward the sun and briefly gets closer. But no rabbit has or can jump even 0.001% of the way. There are hard limits for the rabbit which the rabbit is not equipped to address. A human jumps, fails, learns the underlying physics of what’s holding them back, and consequently build rockets to sit in. There are no hard limits yet, as far as we know, and we’re just bags of mostly water. Whatever the hard limits on AI, they will be so far beyond us that we won’t be able to visualize or guess at them.
    - mechagodzilla 4y ago
      There are indeed hard limits to chemical rocketry, and we mostly hit them after a decade or two of work by people with pencils and slide rules. Computers might be a better analogy. We started designing computers without the aid of other computers. We made very rapid progress on architecture and semiconductors. We’re now completely dependent on computers to help us design future ones, and despite all our efforts and hundreds of billions of dollars in investments, our rate of progress has slowed to a crawl and we seem to rapidly be approaching the end game. Dennard scaling has been done for 15 years. We can make wires that are smaller but just proportionally worse. We can use more silicon area, but then latency and power consumption get proportionally worse. A super clever hyper optimizing AI might push a couple things forward a little bit (or more likely wind up at the exact same destination slightly earlier than we other wise might have), but there’s no compelling reason to think it gets to design computers that face different constraints.
- voxl 4y ago
  Except there is nothing but idealism behind the idea that this is true. Indeed, we have AGI already, they are called humans, and the progress we see in the development of AI is not exponentially explosive. The only exponential growth we've seen (which is necessarily logarithmic growth after some inflection point, actually) is Moore's Law.
  - naasking 4y ago
    > Indeed, we have AGI already, they are called humans, and the progress we see in the development of AI is not exponentially explosive Actually it is. Human population growth has been exponential, as has been our resource use and consumption of our habitat, and the pace of our innovations. We've now created forms of machine intelligence to further augment our own, and machine intelligence could itself start advancing soon, without the messy constraints of evolution and biology. The possibility is definitely there, we'll see if it works out that way in the end. If it does, it will likely blindside us.
- causi 4y ago
  The problem is the hard limits of physics. We have an upper bound for the number of interacting elements required for intelligence in the form of number of synapses in the human brain. That number is a thousand trillion. We don't know what the lower bound is. Neurons are ferociously efficient computing devices, though. A dragonfly can take input from thousands of ommatidia and use it to track prey in three dimensions using only sixteen neurons. How many transistors do you need to do that? Let's be generous and say only ten percent of the human brain is needed for sapience, or 8.6 billion neurons. 8.6 billion / 16 roughly equals five hundred million. Multiply your earlier transistor number by five hundred million. There's your lower bound for a sapient computer. There is no law of the universe that says it has to be possible to generate human-level sapience with less than one kilogram of processing mass. In all likelihood we are never going to create a server with a whole city of intelligences in it, ever.
  - ben_w 4y ago
    > There is no law of the universe that says it has to be possible to generate human-level sapience with less than one kilogram of processing mass. In all likelihood we are never going to create a server with a whole city of intelligences in it, ever. A computer with five times the mass, five times the volume, and five times the power requirements of the human brain, that can run an uploaded copy of a human brain at merely half the speed of a human brain, would still render Pluto and Mercury trivial to colonise with machine intelligences. And if it cost a $100k (inflation adjusted) to build, and lasted 80 years before you had to melt down the hardware and reforge from scratch, it would still be cheaper than Musk's target for a human to Mars.
    - sokoloff 4y ago
      That’s for the computing power, not for the physical actuators which would be needed to provide power, other environmentals, and repairs.
      - ben_w 4y ago
        Those are the easy bits. We've already got all of those, because that's how we make the machines we already use even today, and we would need them in any scenario other than genetically engineering ourselves to eat the moon, not just the uploaded brains scenario.
- lm28469 4y ago
  > The first AGI is (some) amount of time away. A more sophisticated AGI is coming not long after that, and exponentially faster and so on. Why do people keep acting as if this was a god given truth and just a matter of time ?
  - Avamander 4y ago
    What indicates that at some point in the future we wouldn't have the computational power to run neural nets roughly equivalent to a human? It is just a matter of time unless we stop progressing.
    - HideousKojima 4y ago
      >What indicates that at some point in the future we wouldn't have the computational power to run neural nets roughly equivalent to a human? It's making the assumption that human brains are simply very complex neural nets. Even ignoring religious/spiritual arguments about souls, the simple fact of the matter is that we don't really know a lot about how the human brain/consciousness actually works on a fundamental level (and anyone who tells you otherwise is misinformed or lying).
      - Avamander 4y ago
        Not knowing the exact details doesn't conclude it's not a neural net. Anyone saying otherwise wants to debate religion instead of facts.
      - adamsmith143 4y ago
        >It's making the assumption that human brains are simply very complex neural nets. Even ignoring religious/spiritual arguments about souls, the simple fact of the matter is that we don't really know a lot about how the human brain/consciousness actually works on a fundamental level (and anyone who tells you otherwise is misinformed or lying). How human brains work is irrelevant, we didn't need to solve the Navier Stokes Equation to build Airplanes and we don't need to solve intelligence either. Consciousness is totally outside the bounds of this discussion and also irrelevant to building an intelligent AI.
naasking 4y ago
> It’s a long way from a machine intelligence in your computer to one that is even as self sufficient as a cockroach Is it? I'm not so sure about that. I'm pretty sure you can't be sure of that too. Given how machine learning continues to surprise us, I think this is one of those questions that will only be clear in hindsight.
- xref 4y ago
  Ehhh I’d side with the parent on this one, given how we thought self-driving cars were gonna be a solved problem by 2015 but how intractable the problem has turned out to be.
  - unsupp0rted 4y ago
    The problem isn’t intractable at all. We had self-driving cars by 2015 and especially do today. But we can’t let self-driving cars injure or kill even 25% as often as human drivers do, so we refuse to use them to their maximum capability.
    - xref 4y ago
      We don’t, and you have no numbers to back that up. When my blind grandma can sit in a car and it drives her from Paris to Rome though the Alps in winter, then maybe. Keep in mind this thread’s premise is us developing an intelligence that can navigate space and maintain all requirements of that for millennia. Not driving a couple miles on a clear stretch of highway.
      - unsupp0rted 4y ago
        About navigating space, I have no comment, but about driving a car safely: My non-blind and somewhat functional 90-year old uncle can sit in a car and drive it quite legally at the moment. I'm pretty sure self-driving cars are less dangerous on the roads than he is, but we're not willing to test that assumption.
      - google234123 4y ago

knodi123 4y ago

> It’s a long way from a machine intelligence in your computer to one that is even as self sufficient as a cockroach Ah, but the trick is, as soon as you can get a machine intelligence smart enough to work on programming tasks, you can get it to help you design a better one. And so on, etc. The first AGI is (some) amount of time away. A more sophisticated AGI is coming not long after that, and exponentially faster and so on.

asdff 4y ago
Where I get hung up on with this is how the AGI knows what a more sophisticated AGI would look like without being at least as sophisticated? For example with our own intelligence, we aren't getting smarter with each generation. We have less brain matter than humans who lived millions of years ago on average, because there isn't as strong of a selective pressure present these days to see those with a lot of brain matter out reproduce those who get by in life with not as much of it. A very 'intelligent' AGI might realize the most optimal way to seed the planet with itself is actually to have a swarm of very unintelligent AI that is able to run on some very non resource intensive hardware, and seed it widely like how an "unintelligent" insect is able to lay millions of eggs and infest an area beyond human control, and the more intelligent wolf is easily hunted down.
- pencilguin 4y ago
  We do not, in fact, have less brain than hominins millions of years ago. We do have less than our immediate ancestors 100,000 years ago. Your very intelligent AGI would most likely have no interest in seeding anything on Earth. It might reasonably be curious about our ecosystems, but disturbing those would reduce that interest.
- naasking 4y ago
  > Where I get hung up on with this is how the AGI knows what a more sophisticated AGI would look like without being at least as sophisticated I'm not sure what's difficult about this. Are we not able to test what humans can run the fastest without being able to run just as fast? We can equally test which humans are the best spellers, or the best chess players, or the best problem solvers.
  - asdff 4y ago
    We can test these things that we cannot ourselves do, only because we are able to define them and comprehend the definitions. We define a runner as fast when they run a fixed distance in a certain time. Even though we can't run as fast, the understanding of the rules at play are within what we can understand. OTOH when you move past these examples into more abstract ideas of arbitrary rules defining arbitrary tasks, A basic AGI might not be able to even comprehend the full set of rules that define the expected tasks of a much more complicated AGI, much like how it wouldn't do you any good trying to explain the rules of baseball to an ant.
    - google234123 4y ago
      Yet we invent physics that wasn’t comprehended before it was discovered… not sure what your hangup is. We wouldnt stop trying to improve the AGI when it’s at the level of an ant.
    - naasking 4y ago
      No person or AI needs to actually understand how an AGI actually works, they just need observable behaviours indicative of intelligence. That's what we already do with image recognition, language translation tasks, and more. Once we move to more general forms of intelligence, like math and other forms of problem solving, then you apply more general tests. For instance, if we had an Einstein AI v1.0 that could infer General Relativity using X0 bits of information in time T0, and Einstein AI v2.0 managed to infer GR using X1 << X0 bits of information and/or could infer it in time T1 << T0, then v2.0 is clearly superior to v1.0. I agree that sometimes the parameters for such tests aren't always clear at first, but this has always been the case in science, which is we we iterate and progressively refine our tests as we learn. Any true AGI must be capable of such abductive and inductive reasoning to qualify as a general intelligence.
pasquinelli 4y ago
> The first AGI is (some) amount of time away. you jump in the air at high noon. the sun is (some) distance away, and for some time, you're getting closer to it. if we make an ai that's able to make a better ai than we are, then it does stand to reason that it'll kick off exponential growth for some time, but it'll reach limits just like anything. those limits may fall short of magicing/smarting away the obstacles to colonizing space. and we'll reach limits too. it might be that the amount of time between now and when we make a sufficient ai is more time than we have.
- unsupp0rted 4y ago
  A rabbit jumps toward the sun and briefly gets closer. But no rabbit has or can jump even 0.001% of the way. There are hard limits for the rabbit which the rabbit is not equipped to address. A human jumps, fails, learns the underlying physics of what’s holding them back, and consequently build rockets to sit in. There are no hard limits yet, as far as we know, and we’re just bags of mostly water. Whatever the hard limits on AI, they will be so far beyond us that we won’t be able to visualize or guess at them.
  - mechagodzilla 4y ago
    There are indeed hard limits to chemical rocketry, and we mostly hit them after a decade or two of work by people with pencils and slide rules. Computers might be a better analogy. We started designing computers without the aid of other computers. We made very rapid progress on architecture and semiconductors. We’re now completely dependent on computers to help us design future ones, and despite all our efforts and hundreds of billions of dollars in investments, our rate of progress has slowed to a crawl and we seem to rapidly be approaching the end game. Dennard scaling has been done for 15 years. We can make wires that are smaller but just proportionally worse. We can use more silicon area, but then latency and power consumption get proportionally worse. A super clever hyper optimizing AI might push a couple things forward a little bit (or more likely wind up at the exact same destination slightly earlier than we other wise might have), but there’s no compelling reason to think it gets to design computers that face different constraints.
voxl 4y ago
Except there is nothing but idealism behind the idea that this is true. Indeed, we have AGI already, they are called humans, and the progress we see in the development of AI is not exponentially explosive. The only exponential growth we've seen (which is necessarily logarithmic growth after some inflection point, actually) is Moore's Law.
- naasking 4y ago
  > Indeed, we have AGI already, they are called humans, and the progress we see in the development of AI is not exponentially explosive Actually it is. Human population growth has been exponential, as has been our resource use and consumption of our habitat, and the pace of our innovations. We've now created forms of machine intelligence to further augment our own, and machine intelligence could itself start advancing soon, without the messy constraints of evolution and biology. The possibility is definitely there, we'll see if it works out that way in the end. If it does, it will likely blindside us.
causi 4y ago
The problem is the hard limits of physics. We have an upper bound for the number of interacting elements required for intelligence in the form of number of synapses in the human brain. That number is a thousand trillion. We don't know what the lower bound is. Neurons are ferociously efficient computing devices, though. A dragonfly can take input from thousands of ommatidia and use it to track prey in three dimensions using only sixteen neurons. How many transistors do you need to do that? Let's be generous and say only ten percent of the human brain is needed for sapience, or 8.6 billion neurons. 8.6 billion / 16 roughly equals five hundred million. Multiply your earlier transistor number by five hundred million. There's your lower bound for a sapient computer. There is no law of the universe that says it has to be possible to generate human-level sapience with less than one kilogram of processing mass. In all likelihood we are never going to create a server with a whole city of intelligences in it, ever.
- ben_w 4y ago
  > There is no law of the universe that says it has to be possible to generate human-level sapience with less than one kilogram of processing mass. In all likelihood we are never going to create a server with a whole city of intelligences in it, ever. A computer with five times the mass, five times the volume, and five times the power requirements of the human brain, that can run an uploaded copy of a human brain at merely half the speed of a human brain, would still render Pluto and Mercury trivial to colonise with machine intelligences. And if it cost a $100k (inflation adjusted) to build, and lasted 80 years before you had to melt down the hardware and reforge from scratch, it would still be cheaper than Musk's target for a human to Mars.
  - sokoloff 4y ago
    That’s for the computing power, not for the physical actuators which would be needed to provide power, other environmentals, and repairs.
    - ben_w 4y ago
      Those are the easy bits. We've already got all of those, because that's how we make the machines we already use even today, and we would need them in any scenario other than genetically engineering ourselves to eat the moon, not just the uploaded brains scenario.
lm28469 4y ago
> The first AGI is (some) amount of time away. A more sophisticated AGI is coming not long after that, and exponentially faster and so on. Why do people keep acting as if this was a god given truth and just a matter of time ?
- Avamander 4y ago
  What indicates that at some point in the future we wouldn't have the computational power to run neural nets roughly equivalent to a human? It is just a matter of time unless we stop progressing.
  - HideousKojima 4y ago
    >What indicates that at some point in the future we wouldn't have the computational power to run neural nets roughly equivalent to a human? It's making the assumption that human brains are simply very complex neural nets. Even ignoring religious/spiritual arguments about souls, the simple fact of the matter is that we don't really know a lot about how the human brain/consciousness actually works on a fundamental level (and anyone who tells you otherwise is misinformed or lying).
    - Avamander 4y ago
      Not knowing the exact details doesn't conclude it's not a neural net. Anyone saying otherwise wants to debate religion instead of facts.
    - adamsmith143 4y ago
      >It's making the assumption that human brains are simply very complex neural nets. Even ignoring religious/spiritual arguments about souls, the simple fact of the matter is that we don't really know a lot about how the human brain/consciousness actually works on a fundamental level (and anyone who tells you otherwise is misinformed or lying). How human brains work is irrelevant, we didn't need to solve the Navier Stokes Equation to build Airplanes and we don't need to solve intelligence either. Consciousness is totally outside the bounds of this discussion and also irrelevant to building an intelligent AI.

naasking 4y ago

> It’s a long way from a machine intelligence in your computer to one that is even as self sufficient as a cockroach Is it? I'm not so sure about that. I'm pretty sure you can't be sure of that too. Given how machine learning continues to surprise us, I think this is one of those questions that will only be clear in hindsight.

xref 4y ago
Ehhh I’d side with the parent on this one, given how we thought self-driving cars were gonna be a solved problem by 2015 but how intractable the problem has turned out to be.
- unsupp0rted 4y ago
  The problem isn’t intractable at all. We had self-driving cars by 2015 and especially do today. But we can’t let self-driving cars injure or kill even 25% as often as human drivers do, so we refuse to use them to their maximum capability.
  - xref 4y ago
    We don’t, and you have no numbers to back that up. When my blind grandma can sit in a car and it drives her from Paris to Rome though the Alps in winter, then maybe. Keep in mind this thread’s premise is us developing an intelligence that can navigate space and maintain all requirements of that for millennia. Not driving a couple miles on a clear stretch of highway.
    - unsupp0rted 4y ago
      About navigating space, I have no comment, but about driving a car safely: My non-blind and somewhat functional 90-year old uncle can sit in a car and drive it quite legally at the moment. I'm pretty sure self-driving cars are less dangerous on the roads than he is, but we're not willing to test that assumption.
    - google234123 4y ago