5 ms·
People always say "AGI" and they mean different things but I am guessing most now mean something like a digital living human/simulation of human when they say t
by ilaksh 3y ago
People always say "AGI" and they mean different things but I am guessing most now mean something like a digital living human/simulation of human when they say that. Mixed in with some blurry connotation of it automatically becoming godlike in power.
That stuff is very speculative. But we don't need AI to get to that level for it to be very dangerous. Just imagine that we have an open source GPT that is something like 33% smarter than GPT-4 and less brittle. Make the output 50 times faster than human thought and suppose that we can run it very inexpensively, in a "swarm" of agents cooperating.
Then you have a type of superintelligence that does not require any really speculative AI advance -- hyperspeed reasoning -- based on the current technology.
If that is widely deployed for military and industrial decision-making then something like a computer virus could create existential risk.
Also, realize that all these systems really need to emulate some of the core functional aspects of animals, like a self-preservation instinct or desire to control resources, is the right instruction and relaxation of guardrails. So when they can reason a bit better and at hyperspeed with collaboration between them, they don't need to be alive or anything to be dangerous.
Look at the history of increases in computing efficiency. It is easy to imagine 50 or whatever times output speed increase in less than 5 years. And even though they are not human level now, GPT-4 has proven that these types of systems can have strong problem solving ability.
One more thing to add is that as the AI performance increases and surpasses human decision-making ability, the hyperspeed will push competiting companies and countries to give them broader and broader goals and more autonomy. Because waiting overnight for human input means your competitor's AIs race ahead the equivalent of weeks.
I think we need to set a limit for the performance of new hardware at some point. Also we need people to understand the dangers of full autonomy and imitating animals in the context of hyperspeed or superintelligent AI. We will need criminal penalties and maybe some type of cooperative digital immune system.
- LatteLazy 3y agoI think you also need to draw a line between different types of danger: An actual AGI, that can think and has it's own motives and initiative is an entirely different danger to GPT-4, -5 etc which cannot. Those might post a danger in terms of fake news, or worker displacement, but they're just social issues. No rate improvement in chat GPT will make it "decide to kill all the humans" etc.
- Dalewyn 3y agoEvery time I hear how artificial intelligence with its "own motives" etc. are dangerous, I wonder how it's different from just dealing with people and animals. Or to put it another way, I can't see how artificial intelligence could be any more dangerous than intelligence as we already know and have them.
- jonplackett 3y agoI think that’s the problem isn’t it? Intelligence has proven to be exceptionally dangerous!
- LatteLazy 3y agoI don't want to sound dismissive but... Well yeah. We already know humans are dangerous. And that is humans who are conditioned to be social, and who have empathy and who rely on other humans for all sorts of things. Plus with fear of a legal system punishing them. Now imagine a human like intelligence without empathy, that is not conditioned from childhood to be social, and that is inherently threatened by us and not reliant on us. Would you want that guy living next door? Oh and it's 100x smarter than you. I think all this is 100 years away, but that's the theory. Skynet doesn't have a subconscious telling it not to murder it's parents...
- robnado 3y agoHow dangerous can a dictator with complete knowledge and super human speed actually be?
- Jagah 3y ago[dead]
- lesuorac 3y agoI mean if you wire the output of GPT-x to a nuclear reactor it can cause all kinds of danger. The thing that keeps us safe from computers is generally we don't let them have access to anything dangerous and when we do theres typically an evaluation of what can go wrong and how we'll prevent the computer from doing that (often in the other direction, we only program the computer to do X in Y situations). But sometimes that fails and you get the unexpected rapid acceleration of cars.
- pydry 3y ago>That stuff is very speculative. But we don't need AI to get to that level for it to be very dangerous. Just imagine that we have an open source GPT that is something like 33% smarter than GPT-4 and less brittle. Make the output 50 times faster than human thought and suppose that we can run it very inexpensively, in a "swarm" of agents cooperating. Then you have a type of superintelligence No. Then you have something that can spout plausible sounding nonsense mixed in with facts gleaned from crawling the internet. There are some dangers to this - don't get me wrong. It could be used to create bots which engage in much more sophisticated social media manipulation on a large scale, for instance.