3 ms·

> The first AGI is (some) amount of time away. you jump in the air at high noon. the sun is (some) distance away, and for some time, you're getting closer to i

> The first AGI is (some) amount of time away. you jump in the air at high noon. the sun is (some) distance away, and for some time, you're getting closer to it. if we make an ai that's able to make a better ai than we are, then it does stand to reason that it'll kick off exponential growth for some time, but it'll reach limits just like anything. those limits may fall short of magicing/smarting away the obstacles to colonizing space. and we'll reach limits too. it might be that the amount of time between now and when we make a sufficient ai is more time than we have.

unsupp0rted 4y ago
A rabbit jumps toward the sun and briefly gets closer. But no rabbit has or can jump even 0.001% of the way. There are hard limits for the rabbit which the rabbit is not equipped to address. A human jumps, fails, learns the underlying physics of what’s holding them back, and consequently build rockets to sit in. There are no hard limits yet, as far as we know, and we’re just bags of mostly water. Whatever the hard limits on AI, they will be so far beyond us that we won’t be able to visualize or guess at them.
- mechagodzilla 4y ago
  There are indeed hard limits to chemical rocketry, and we mostly hit them after a decade or two of work by people with pencils and slide rules. Computers might be a better analogy. We started designing computers without the aid of other computers. We made very rapid progress on architecture and semiconductors. We’re now completely dependent on computers to help us design future ones, and despite all our efforts and hundreds of billions of dollars in investments, our rate of progress has slowed to a crawl and we seem to rapidly be approaching the end game. Dennard scaling has been done for 15 years. We can make wires that are smaller but just proportionally worse. We can use more silicon area, but then latency and power consumption get proportionally worse. A super clever hyper optimizing AI might push a couple things forward a little bit (or more likely wind up at the exact same destination slightly earlier than we other wise might have), but there’s no compelling reason to think it gets to design computers that face different constraints.
  - adamsmith143 4y ago
    This sounds like a child explaining to other children that Adults can't be that much smarter and probably won't be able to make any progress on putting the square peg into the right hole. The child can't comprehend the things Adults can think about.
    - simiones 4y ago
      We have a pretty good understanding of physics in this area, and one that has stood many significant tests. While it's of course always possible that we're missing something, it's entirely likely right now that there is no new physics to be discovered at the scale of microprocessors. Given that, we have a pretty good idea of how much more could be done in principle to improve processors (if we assumed we could overcome all engineering challenges), and it's not that much more.
- lm28469 4y ago
  > Whatever the hard limits on AI, they will be so far beyond us that we won’t be able to visualize or guess at them. Can you share your crystal ball? I need to check next week lottery numbers
knodi123 4y ago
> those limits may fall short of magicing/smarting away the obstacles to colonizing space. They may, or they may not, which is why I would not opine on the subject. I was only addressing one narrow part of the comment I replied to.
adamsmith143 4y ago
>those limits may fall short of magicing/smarting away the obstacles to colonizing space. We have no idea what that limit is so it seems crazy to make any assumptions about where it is. At minimum it lies above Human Intelligence and even imagining what a civilization could do with millions of beings only as intelligent as say John von Neumann is frightening. >those limits may fall short of magicing/smarting away the obstacles to colonizing space. We can basically already colonize our Solar System with current tech. You don't need FTL or anything close to that to eventually colonize the entire Galaxy.

unsupp0rted 4y ago

A rabbit jumps toward the sun and briefly gets closer. But no rabbit has or can jump even 0.001% of the way. There are hard limits for the rabbit which the rabbit is not equipped to address. A human jumps, fails, learns the underlying physics of what’s holding them back, and consequently build rockets to sit in. There are no hard limits yet, as far as we know, and we’re just bags of mostly water. Whatever the hard limits on AI, they will be so far beyond us that we won’t be able to visualize or guess at them.

mechagodzilla 4y ago
There are indeed hard limits to chemical rocketry, and we mostly hit them after a decade or two of work by people with pencils and slide rules. Computers might be a better analogy. We started designing computers without the aid of other computers. We made very rapid progress on architecture and semiconductors. We’re now completely dependent on computers to help us design future ones, and despite all our efforts and hundreds of billions of dollars in investments, our rate of progress has slowed to a crawl and we seem to rapidly be approaching the end game. Dennard scaling has been done for 15 years. We can make wires that are smaller but just proportionally worse. We can use more silicon area, but then latency and power consumption get proportionally worse. A super clever hyper optimizing AI might push a couple things forward a little bit (or more likely wind up at the exact same destination slightly earlier than we other wise might have), but there’s no compelling reason to think it gets to design computers that face different constraints.
- adamsmith143 4y ago
  This sounds like a child explaining to other children that Adults can't be that much smarter and probably won't be able to make any progress on putting the square peg into the right hole. The child can't comprehend the things Adults can think about.
  - simiones 4y ago
    We have a pretty good understanding of physics in this area, and one that has stood many significant tests. While it's of course always possible that we're missing something, it's entirely likely right now that there is no new physics to be discovered at the scale of microprocessors. Given that, we have a pretty good idea of how much more could be done in principle to improve processors (if we assumed we could overcome all engineering challenges), and it's not that much more.
lm28469 4y ago
> Whatever the hard limits on AI, they will be so far beyond us that we won’t be able to visualize or guess at them. Can you share your crystal ball? I need to check next week lottery numbers

knodi123 4y ago

> those limits may fall short of magicing/smarting away the obstacles to colonizing space. They may, or they may not, which is why I would not opine on the subject. I was only addressing one narrow part of the comment I replied to.

adamsmith143 4y ago

>those limits may fall short of magicing/smarting away the obstacles to colonizing space. We have no idea what that limit is so it seems crazy to make any assumptions about where it is. At minimum it lies above Human Intelligence and even imagining what a civilization could do with millions of beings only as intelligent as say John von Neumann is frightening. >those limits may fall short of magicing/smarting away the obstacles to colonizing space. We can basically already colonize our Solar System with current tech. You don't need FTL or anything close to that to eventually colonize the entire Galaxy.