6 ms·

My whole life I have been terrified of AI. It all started with watching the Iron Giant when I was 10, and realising, holy crap, we could probably build a crazy

by kypro 4y ago

My whole life I have been terrified of AI. It all started with watching the Iron Giant when I was 10, and realising, holy crap, we could probably build a crazy monster like this in the near-future. Obviously as I got older and went on to learn about AI and neural nets in university my opinions on this subject matured. Instead of worrying about killer robots my thoughts became more nuanced, however what didn't change was my fear – if anything this increased from a silly childhood fear of a terminator-like robot, to unleashing an uncontrollable super AI which is indifferent to human life. Finding Eliezer's work sometime in the early 2010s was almost a therapeutic experience for me. Finally someone understood and was talking about the kind of concerns I had and was accused of being a lunatic for thinking. My primary concerns with AI basically boil down to two things: 1. Politics is largely a power struggle. Democracies give power to the collective, but this power is cemented by the reality that political leaders and institutions could not function without taxation and labourers. The reason the governments can't just do things that are deeply unpopular and suppress any public revolts with armed responses is because eventually you will either have to kill everyone or everyone will just stop working in protest. Either way the entire system will collapse. AI being able to create wealth without human labours and industrialised weaponry fundamentally removes the power dynamic needed to support democratic societies – you cannot withhold labour and you cannot overpower the state with force. At the same time it would also make humans a waste of resources to those in power, as the unemployed masses are basically just leeches on the resources and space of the state – resources which could otherwise be hoarded by those in power. If super AGI systems ever exist, the state will be all-powerful while having even less incentive to care about your opinions as it does a rat's. 2. Super-intelligence is fundamentally uncontrollable. I can't even be bothered to argue this point it's that self evident. If you disagree you either don't understand how modern AI systems work, or you're not thinking about the control problem deeply enough. But the argument here is simple – all other technologies with destructive capabilities rely on human decision making. No gun is ever shot, no war is ever started, no nuke is ever dropped without a human decision maker. Super-AI removes this control from us and requires a kind of hopeium that AI will just be nice to us and care about improving our standards of living. And for those who want to argue "you can just program it to be good" – no you can't, for the most part we have no clear understanding of how advanced AI systems of today internally operate. The alignment conversation is fundamentally pointless in regards to a super-intelligence because you can not reason with any level of certainty about the alignment of an intelligence far superior to your own. Instead we should assume unalignment by default because even if you assume we can somehow create an aligned super-intelligence the likelihood that eventually an AI system would be created which is unaligned is practically 100% – this is true even if you assume humans will never intend to create it. Why? Because in a world where we can all create a super-intelligent aligned AIs, you also have near-infinite opportunities for someone to create an unaligned one. And here's another statement you should assume by default – the only world in which nukes aren't going to be used destructively again is a world in which humans or nukes no longer exist. The same will be true for AI systems, but this time the barrier for creating destructive AI is likely going to be orders of magnitude lower than that of nukes, so unlike nuclear holocaust we don't even have probabilities on our side here. -------- Look, we're doomed. Progress isn't going to stop and AGI isn't going to be the fairytale people are crossing their fingers for. I don't quite know why no one really seems to have clocked onto this yet, but this will soon become obvious, by which time it will be too late. Those who think AI is going to improve their productivity are suffering from some kind of delusion in which they believe their ability to type text in a website and copy and paste its output is a marketable skill. It's not, and you're going to be unemployed by tools like GPT in the near-future even without the existential risk posed by super-AGI.

carapace 4y ago
There is no overlap between the ecological niches of humans and GAI, in other words, there is no cost to the GAI for allowing us to continue to exist. GAI will feed on information. Humans are the densest source of information in the known Universe. (The rest of the Universe is a poem in physics, in information space humans shine as the sole star in a dark universe.) Ergo, from first principles, hyper-intelligent GAI will hide from us and practice non-interference, following something like the "Prime Directive" of Star Trek. Cheers, hope this helps.
- hollerith 4y ago
  >There is no overlap between the ecological niches of humans and GAI, in other words, there is no cost to the GAI for allowing us to continue to exist. We are made of atoms that the GAI can use for something else. Ditto our farms and cities. Moreover, unlike most of the atoms in the universe, the GAI doesn't have to spend time traveling to get to us. If we could arrange for the GAI to appear in some distant galaxy, then yeah, by the time it gets to us, it'd already have control over so many atom that it might just leave us alone (because we are so different from most of the atoms). The GAI will know about the prime directive because it will have been trained on internet conversations about the prime directive, but there is no particular reason to hope that exposure to moral arguments will alter the GAI's objectives similar to what tends to happen with young human beings: instead it will have whatever objectives its creator gave it, which (given the deplorable state of most AI research) is unlikely to be the objectives that its creator thought it was giving it. (By "creator" I mean of course a team of human researchers.) Your poetical imagery might make you feel better, but won't save us. >Humans are the densest source of information in the known Universe. You feel that way about humans because evolution made you that way. It is unlikely that any of the leading AI research teams will make the first transformative AI that way: they do not know how. They certainly know how to train AIs on human cultural information, but that is different from inculcating in the AI a desire for the continued cultural output of humanity. It will create its own culture (knowledge and tools) that is much more powerful than human culture where "power" means basically the ability to get stuff done.
  - 4y ago

by kypro 4y ago

carapace 4y ago

There is no overlap between the ecological niches of humans and GAI, in other words, there is no cost to the GAI for allowing us to continue to exist. GAI will feed on information. Humans are the densest source of information in the known Universe. (The rest of the Universe is a poem in physics, in information space humans shine as the sole star in a dark universe.) Ergo, from first principles, hyper-intelligent GAI will hide from us and practice non-interference, following something like the "Prime Directive" of Star Trek. Cheers, hope this helps.

hollerith 4y ago
>There is no overlap between the ecological niches of humans and GAI, in other words, there is no cost to the GAI for allowing us to continue to exist. We are made of atoms that the GAI can use for something else. Ditto our farms and cities. Moreover, unlike most of the atoms in the universe, the GAI doesn't have to spend time traveling to get to us. If we could arrange for the GAI to appear in some distant galaxy, then yeah, by the time it gets to us, it'd already have control over so many atom that it might just leave us alone (because we are so different from most of the atoms). The GAI will know about the prime directive because it will have been trained on internet conversations about the prime directive, but there is no particular reason to hope that exposure to moral arguments will alter the GAI's objectives similar to what tends to happen with young human beings: instead it will have whatever objectives its creator gave it, which (given the deplorable state of most AI research) is unlikely to be the objectives that its creator thought it was giving it. (By "creator" I mean of course a team of human researchers.) Your poetical imagery might make you feel better, but won't save us. >Humans are the densest source of information in the known Universe. You feel that way about humans because evolution made you that way. It is unlikely that any of the leading AI research teams will make the first transformative AI that way: they do not know how. They certainly know how to train AIs on human cultural information, but that is different from inculcating in the AI a desire for the continued cultural output of humanity. It will create its own culture (knowledge and tools) that is much more powerful than human culture where "power" means basically the ability to get stuff done.
- 4y ago