5 ms·
> And one possibility might be that AIs could “improve themselves” to produce a single “apex intelligence” that would in a sense dominate everything else. That
by Animats 4y ago
> And one possibility might be that AIs could “improve themselves” to produce a single “apex intelligence” that would in a sense dominate everything else.
That's not what worries me. It's when AIs learn to distribute work to other AIs and humans. Once that starts to work, machines are going to be better at coordination than humans, simply because they have more I/O bandwidth.
- echelon 4y agoNo more middle managers! But seriously, taking middle management self-promotion out of the equation could reduce some of the rotten malinvestment and political in-fighting that happens in large organizations.
- still_grokking 4y agoTo the AI all humans are "middle managers" though…
- ta1243 4y agoOr it could result in the first distopia of "Manna", where humans are there for cleaning, but there's no middle management. https://marshallbrain.com/manna1 https://marshallbrain.com/manna1
- still_grokking 4y agoYes, being able to perform orders of magnitude more APM than the opponent is a sure win, even when the particular actions aren't optimal.
- Faark 4y agoImagine an AI spawning many parallel instances of itself, all feeding back into the same conscience. So many coordination problems just gone. All the resources humans spend fighting each other in some form or another... And entirely depending only on itself. No need for any external being. That could prove very deadly to us, since "we need each other" is the core pillar forming our society. A king needs his peasants. An AI? Could just boot up another instance...
- deleted 4y ago[deleted]
- meh8881 4y agoI think a lot of people get hung up on the tech Not having the will to do this kind of thing. And they’re right. But it would be very easy to prompt such a model to act as if it were a paranoid AI doing whatever it can to ensure its survival. We can do that now it’s just not capable of “doing things”. We don’t really know what “sentient” AI that organically finds its own identity would be like. But we know exactly what a non sentient LLM that is down to clown and prompted to be evil would be like. That’s… a hard problem. Sentience is potentially a barrier to evil AI, not a pre-req. (No, I don’t want to debate the definition of sentience again).
- ChatGTP 4y agoIt would be much, much easier to turn an AI that isn't sentient off though?
- meh8881 4y agoIf you have access to it, yes. The problem is the low barriers to entry for some guy to try and cause a bank run for the lols.
- falcor84 4y agoPutting my contrarian hat on, I think it's much better that we have tons of "guys" testing the limits of this new tech for the lolz, than leaving easy exploits on the table for state actors and/or terrorists.
- RandomLensman 4y agoThose AIs still need the ability to manipulate the physical world at scale, otherwise they can be as intelligent as they want.
- flemhans 4y agoCould be inderectly or subtly using e.g. social media to nudge people in the desired direction, like the Russians do
- Teever 4y agoHell, some people wouldn't even need subtle or indirect messaging, they'd volunteer to work for an AI just to fuck with the status quo.
- RandomLensman 4y agoThose things can be shut down or the AI can be shut down.
- TheOtherHobbes 4y agoOnly if humans notice them happening.
- flangola7 4y agoThen why isn't it shut down already? Social manipulation campaigns have been known for years, and AI will be far more effective than anything that has come before. "We'll just turn it off before it causes damage" is a scenario that's been wargamed many times, and it usually does not end in our favor if the AI is sufficiently capable.
- Teever 4y agoCan an AI describe the hardware that it runs on? Can it provide instructions for a human to follow to build that hardware? Or even better can it provide the instructions for a human to follow to build a humanoid robot that can build humanoid robots?