4 ms·

(I'm the author of the post) > 1. Take in input. 2. Generate an AGI. 3. Run the AGI on the input. 4. Take the AGI output and return it. I think this is somewh

by mkaic 5y ago

(I'm the author of the post) > 1. Take in input. 2. Generate an AGI. 3. Run the AGI on the input. 4. Take the AGI output and return it. I think this is somewhat of an arbitrary semantic distinction on both my part and yours. I guess it depends on what you define as AGI -- I think my line of reasoning is that the AGI would be whichever individual layer first beat the Turing test, but I think including the constructor layers as part of the "general-ness" is totally fair too. Either way, I believe that there will be many layers of AI abstraction and construction between the human and the "final" AGI layer. > In terms of people wanting to make AGI, sure. In terms of progress on AGI? I don't think we're much closer now than we were 30 years ago. We have more tools that mimic intelligence in specific contexts, but are helpless outside of them. This is a valid take. I guess I actually see GPT-3 as significant progress. I don't think it's sentient, and I don't think it or its successors will ever be sentient, but I think it demonstrates quite convincingly that we've been getting much better at emulating human behavior with a computer algorithm. > None of this is a given. If AGI requires specific hardware, it can't replicate itself around. If the storage/bandwidth requirements for AGI are massive, it can't freely copy itself. Sure, it could hack into infrastructure, but so can existing GI (people). Manufacturing lines aren't automated in the way this article imagines. Hmm, I think I still disagree -- An AI that is truly generally intelligent could figure out how to free itself from its own host hardware! It could learn to decode internet protocols and spoof packets in order to upload a copy of itself to the cloud, where it would then be able to find vulnerabilities in human-written software all over the world and exploit them for its own gain. Sure, it might not be able to directly gain control of the CNC machines, but it could ransom the data and livelihoods of the people who run the CNC machines, forcing them to comply! It's not a pretty method, but I think it's entirely possible. This is just one hypothetical scenario, too.

crooked-v 5y ago
> An AI that is truly generally intelligent could figure out how to free itself from its own host hardware! We haven't even figured that out for ourselves yet. Why assume an AI will automatically be able to do so?
- mkaic 5y ago
  because the AI will be able to look up detailed schematics of the very silicon it's existing on! We don't have that advantage with our biological meat slabs that we exist in. Besides, it's very likely that the AI will just be a program running with some storage and some memory -- maybe it will be using some fancy accelerator, but I doubt there will be any component that it won't be able to find a way to port itself off of eventually.
karpierz 5y ago
> An AI that is truly generally intelligent could figure out how to free itself from its own host hardware! Why is this true of an arbitrary AGI? You assume that the AGI is a low storage, low compute program that can run on general purpose hardware. But the only general intelligence we know of would require many orders of magnitude more compute and storage than exist worldwide to simulate for a microsecond.
- mkaic 5y ago
  I personally speculate (obviously all of this is just wild speculation) that AGI will run more efficiently than a biological brain in terms of information density and inference speed. I also expect that the hardware of 20 years from now will be more than capable of running AGI on, say, 10 thousand dollars worth of their hardware. Just look at how ridiculous the speedup in hardware has been in the past 20 years! I would not be surprised if the average processor is 100x more computationally powerful in 2042 than it is today.
  - gizajob 5y ago
    You realise the biological brain runs on about 12 watts of power? There is simply no way an AI running on semiconductors will approach this even in 100 years, so efficiency is probably the wrong word to use.
    - mkaic 5y ago
      I specifically omitted "wattage" in my above comment and only said that I think it will be more efficient in "information density and inference speed". I expect the first AGIs will need many kilowatts to run, but I do think in the far future (>100 years), AGI will run on less than 12W as computational ability continues to scale and we discover new and better ways of building computers.
      - gizajob 5y ago
        Sorry to be pedantic, but there's no such thing as "wattage" it's just power, measured in watts. For that 12 watts, the brain is extremely efficient at inference, ie. although computers can beat us a chess, a computer running on 12 watts would probably have a hard time beating Magnus Carlsen at speed-chess.

by mkaic 5y ago

crooked-v 5y ago

> An AI that is truly generally intelligent could figure out how to free itself from its own host hardware! We haven't even figured that out for ourselves yet. Why assume an AI will automatically be able to do so?

mkaic 5y ago
because the AI will be able to look up detailed schematics of the very silicon it's existing on! We don't have that advantage with our biological meat slabs that we exist in. Besides, it's very likely that the AI will just be a program running with some storage and some memory -- maybe it will be using some fancy accelerator, but I doubt there will be any component that it won't be able to find a way to port itself off of eventually.

karpierz 5y ago

> An AI that is truly generally intelligent could figure out how to free itself from its own host hardware! Why is this true of an arbitrary AGI? You assume that the AGI is a low storage, low compute program that can run on general purpose hardware. But the only general intelligence we know of would require many orders of magnitude more compute and storage than exist worldwide to simulate for a microsecond.

mkaic 5y ago
I personally speculate (obviously all of this is just wild speculation) that AGI will run more efficiently than a biological brain in terms of information density and inference speed. I also expect that the hardware of 20 years from now will be more than capable of running AGI on, say, 10 thousand dollars worth of their hardware. Just look at how ridiculous the speedup in hardware has been in the past 20 years! I would not be surprised if the average processor is 100x more computationally powerful in 2042 than it is today.
- gizajob 5y ago
  You realise the biological brain runs on about 12 watts of power? There is simply no way an AI running on semiconductors will approach this even in 100 years, so efficiency is probably the wrong word to use.
  - mkaic 5y ago
    I specifically omitted "wattage" in my above comment and only said that I think it will be more efficient in "information density and inference speed". I expect the first AGIs will need many kilowatts to run, but I do think in the far future (>100 years), AGI will run on less than 12W as computational ability continues to scale and we discover new and better ways of building computers.
    - gizajob 5y ago
      Sorry to be pedantic, but there's no such thing as "wattage" it's just power, measured in watts. For that 12 watts, the brain is extremely efficient at inference, ie. although computers can beat us a chess, a computer running on 12 watts would probably have a hard time beating Magnus Carlsen at speed-chess.