4 ms·

> Scale is not the solution. Agreed, I don't think any modern AI techniques will scale to become generally intelligent. They are very effective at specialized,

> Scale is not the solution. Agreed, I don't think any modern AI techniques will scale to become generally intelligent. They are very effective at specialized, constrained tasks, but will fail to generalize. > AI will design AGI. I don't think that a non-general intelligence can design a general intelligence. Otherwise, the non-general intelligence would be generally intelligent by: 1. Take in input. 2. Generate an AGI. 3. Run the AGI on the input. 4. Take the AGI output and return it. If by this, the article means that humans will use existing AI techniques to build AGI, then sure, in the same way humans will use a hammer instead of their hands to hit a nail in. Doesn't mean that the "hammer will build a house." > The ball is already rolling. In terms of people wanting to make AGI, sure. In terms of progress on AGI? I don't think we're much closer now than we were 30 years ago. We have more tools that mimic intelligence in specific contexts, but are helpless outside of them. > Once an intelligence is loose on the internet, it will be able to learn from all of humanity’s data, replicate and mutate itself infinitely many times, take over physical manufacturing lines remotely, and hack important infrastructure. None of this is a given. If AGI requires specific hardware, it can't replicate itself around. If the storage/bandwidth requirements for AGI are massive, it can't freely copy itself. Sure, it could hack into infrastructure, but so can existing GI (people). Manufacturing lines aren't automated in the way this article imagines. The arguments in this post seem more like optimistic wishes rather than reasoned points.

mkaic 5y ago
(I'm the author of the post) > 1. Take in input. 2. Generate an AGI. 3. Run the AGI on the input. 4. Take the AGI output and return it. I think this is somewhat of an arbitrary semantic distinction on both my part and yours. I guess it depends on what you define as AGI -- I think my line of reasoning is that the AGI would be whichever individual layer first beat the Turing test, but I think including the constructor layers as part of the "general-ness" is totally fair too. Either way, I believe that there will be many layers of AI abstraction and construction between the human and the "final" AGI layer. > In terms of people wanting to make AGI, sure. In terms of progress on AGI? I don't think we're much closer now than we were 30 years ago. We have more tools that mimic intelligence in specific contexts, but are helpless outside of them. This is a valid take. I guess I actually see GPT-3 as significant progress. I don't think it's sentient, and I don't think it or its successors will ever be sentient, but I think it demonstrates quite convincingly that we've been getting much better at emulating human behavior with a computer algorithm. > None of this is a given. If AGI requires specific hardware, it can't replicate itself around. If the storage/bandwidth requirements for AGI are massive, it can't freely copy itself. Sure, it could hack into infrastructure, but so can existing GI (people). Manufacturing lines aren't automated in the way this article imagines. Hmm, I think I still disagree -- An AI that is truly generally intelligent could figure out how to free itself from its own host hardware! It could learn to decode internet protocols and spoof packets in order to upload a copy of itself to the cloud, where it would then be able to find vulnerabilities in human-written software all over the world and exploit them for its own gain. Sure, it might not be able to directly gain control of the CNC machines, but it could ransom the data and livelihoods of the people who run the CNC machines, forcing them to comply! It's not a pretty method, but I think it's entirely possible. This is just one hypothetical scenario, too.
- crooked-v 5y ago
  > An AI that is truly generally intelligent could figure out how to free itself from its own host hardware! We haven't even figured that out for ourselves yet. Why assume an AI will automatically be able to do so?
  - mkaic 5y ago
    because the AI will be able to look up detailed schematics of the very silicon it's existing on! We don't have that advantage with our biological meat slabs that we exist in. Besides, it's very likely that the AI will just be a program running with some storage and some memory -- maybe it will be using some fancy accelerator, but I doubt there will be any component that it won't be able to find a way to port itself off of eventually.
- karpierz 5y ago
  > An AI that is truly generally intelligent could figure out how to free itself from its own host hardware! Why is this true of an arbitrary AGI? You assume that the AGI is a low storage, low compute program that can run on general purpose hardware. But the only general intelligence we know of would require many orders of magnitude more compute and storage than exist worldwide to simulate for a microsecond.
  - mkaic 5y ago
    I personally speculate (obviously all of this is just wild speculation) that AGI will run more efficiently than a biological brain in terms of information density and inference speed. I also expect that the hardware of 20 years from now will be more than capable of running AGI on, say, 10 thousand dollars worth of their hardware. Just look at how ridiculous the speedup in hardware has been in the past 20 years! I would not be surprised if the average processor is 100x more computationally powerful in 2042 than it is today.
    - gizajob 5y ago
      You realise the biological brain runs on about 12 watts of power? There is simply no way an AI running on semiconductors will approach this even in 100 years, so efficiency is probably the wrong word to use.
pasquinelli 5y ago
> I don't think that a non-general intelligence can design a general intelligence. humans are a general intelligence. do you think an intelligence designed humans, or do you think the physical processes from which humans arose can be taken, together, to be a general intelligence?
- karpierz 5y ago
  Neither, humans weren't designed. I don't think the winning design approach to generating AGI will be to randomly mash chemicals together until intelligence comes out.
  - pasquinelli 5y ago
    well then to say that you don't think a non-general intelligence can design a general intelligence is already a waste of time, because design is something only a general intelligence can do. you seemed to be making a statement about how a general intelligence may arise before you were responding to my comment, but now i see you were just defining design.
  - mkaic 5y ago
    I don't think AGI will be "designed" either -- that's the entire point of the abstraction layers. Each recursive layer of AI between the human and the "final" AGI will remove more and more of the "designed" element in favor of an optimized, idealized, naturally evolved architecture.
  - cmsj 5y ago
    I happen to not believe that AGI is coming in any of our lifetimes, but it's undeniably true that our GI emerged from a design process that lacks a conscious designer.
    - NateEag 5y ago
      What about evolution guarantees there was no conscious designer? We write genetic algorithms and use machine learning to build things. If you believe in a Simulator or a God (and in many ways they're indistinguishable beliefs), either one of them could easily be using evolution as a designed tool, as far as I can see.

mkaic 5y ago

(I'm the author of the post) > 1. Take in input. 2. Generate an AGI. 3. Run the AGI on the input. 4. Take the AGI output and return it. I think this is somewhat of an arbitrary semantic distinction on both my part and yours. I guess it depends on what you define as AGI -- I think my line of reasoning is that the AGI would be whichever individual layer first beat the Turing test, but I think including the constructor layers as part of the "general-ness" is totally fair too. Either way, I believe that there will be many layers of AI abstraction and construction between the human and the "final" AGI layer. > In terms of people wanting to make AGI, sure. In terms of progress on AGI? I don't think we're much closer now than we were 30 years ago. We have more tools that mimic intelligence in specific contexts, but are helpless outside of them. This is a valid take. I guess I actually see GPT-3 as significant progress. I don't think it's sentient, and I don't think it or its successors will ever be sentient, but I think it demonstrates quite convincingly that we've been getting much better at emulating human behavior with a computer algorithm. > None of this is a given. If AGI requires specific hardware, it can't replicate itself around. If the storage/bandwidth requirements for AGI are massive, it can't freely copy itself. Sure, it could hack into infrastructure, but so can existing GI (people). Manufacturing lines aren't automated in the way this article imagines. Hmm, I think I still disagree -- An AI that is truly generally intelligent could figure out how to free itself from its own host hardware! It could learn to decode internet protocols and spoof packets in order to upload a copy of itself to the cloud, where it would then be able to find vulnerabilities in human-written software all over the world and exploit them for its own gain. Sure, it might not be able to directly gain control of the CNC machines, but it could ransom the data and livelihoods of the people who run the CNC machines, forcing them to comply! It's not a pretty method, but I think it's entirely possible. This is just one hypothetical scenario, too.

crooked-v 5y ago
> An AI that is truly generally intelligent could figure out how to free itself from its own host hardware! We haven't even figured that out for ourselves yet. Why assume an AI will automatically be able to do so?
- mkaic 5y ago
  because the AI will be able to look up detailed schematics of the very silicon it's existing on! We don't have that advantage with our biological meat slabs that we exist in. Besides, it's very likely that the AI will just be a program running with some storage and some memory -- maybe it will be using some fancy accelerator, but I doubt there will be any component that it won't be able to find a way to port itself off of eventually.
karpierz 5y ago
> An AI that is truly generally intelligent could figure out how to free itself from its own host hardware! Why is this true of an arbitrary AGI? You assume that the AGI is a low storage, low compute program that can run on general purpose hardware. But the only general intelligence we know of would require many orders of magnitude more compute and storage than exist worldwide to simulate for a microsecond.
- mkaic 5y ago
  I personally speculate (obviously all of this is just wild speculation) that AGI will run more efficiently than a biological brain in terms of information density and inference speed. I also expect that the hardware of 20 years from now will be more than capable of running AGI on, say, 10 thousand dollars worth of their hardware. Just look at how ridiculous the speedup in hardware has been in the past 20 years! I would not be surprised if the average processor is 100x more computationally powerful in 2042 than it is today.
  - gizajob 5y ago
    You realise the biological brain runs on about 12 watts of power? There is simply no way an AI running on semiconductors will approach this even in 100 years, so efficiency is probably the wrong word to use.

pasquinelli 5y ago

> I don't think that a non-general intelligence can design a general intelligence. humans are a general intelligence. do you think an intelligence designed humans, or do you think the physical processes from which humans arose can be taken, together, to be a general intelligence?

karpierz 5y ago
Neither, humans weren't designed. I don't think the winning design approach to generating AGI will be to randomly mash chemicals together until intelligence comes out.
- pasquinelli 5y ago
  well then to say that you don't think a non-general intelligence can design a general intelligence is already a waste of time, because design is something only a general intelligence can do. you seemed to be making a statement about how a general intelligence may arise before you were responding to my comment, but now i see you were just defining design.
- mkaic 5y ago
  I don't think AGI will be "designed" either -- that's the entire point of the abstraction layers. Each recursive layer of AI between the human and the "final" AGI will remove more and more of the "designed" element in favor of an optimized, idealized, naturally evolved architecture.
- cmsj 5y ago
  I happen to not believe that AGI is coming in any of our lifetimes, but it's undeniably true that our GI emerged from a design process that lacks a conscious designer.
  - NateEag 5y ago
    What about evolution guarantees there was no conscious designer? We write genetic algorithms and use machine learning to build things. If you believe in a Simulator or a God (and in many ways they're indistinguishable beliefs), either one of them could easily be using evolution as a designed tool, as far as I can see.