I think the argument about AI overtaking humanity is wrongly framed. Sci-fi has taught us to be on the look out for an all-powerful machine, robot, algorithm, t

I think the argument about AI overtaking humanity is wrongly framed. Sci-fi has taught us to be on the look out for an all-powerful machine, robot, algorithm, turning against humanity. A single, united swarm of machines that only needs humans for energy. This is unlikely to happen soon. But if we borrow an idea from Richard Dawkin's The Selfish Gene, things look differently. He argues evolution is not about the success of individuals or even species, but about the proliferation of their DNAs. The organisms are just the vessels for DNA's replication. Let's look at the corpus of AI technology as an evil machine's DNA. It already claims a lot of human energy and capital for its evolution and proliferation. Our top researchers invest their time into making it better. We burn fossil fuels to provide the power for its computational cycles. What's the primary fuel for its evolution? Money. Which comes from stock markets and VCs, which use AI models. I'm sure a lot of fundamentals-based financial models, based on very different lines of reasoning, end up valuing AI-based companies higher. It is in no way a deliberate conspiracy, just a result of the AI doing it's job well. Of course, self driving cars, painting funny moustaches on your faces, or playing Goare all benign, non-threating abilities of AI. But it is easy to see how falling into the wrong hands, they might lead to not-so-great results. Our stereotypical AI researcher is surely very careful about AI's potential dangers. But do you think Uber, a notoriously profit-driven company, would invest this much care if it doesn't help the bottom line? The cat in a cage argument is surely convincing, but AI is already taking a lot of decisions in place of humans. Be it mortgages, crime detection and prevention, or what information reaches us through our news feeds. And it is easy to see how a single imperfectly-configured algorithm can lead to real-world effects. For example, imagine a threat assessment AI, as already used in some shape and form by police forces around the world, which is configured not to minimize deaths and injuries, but to minimize damage to the economy by an individual. A benign sounding setup. Imagine such a system has to evaluate the threat level of a known researcher, who specializes in AI safety, thus reducing the profit of a bunch of the world's biggest companies. Is he a threat to the economy? Should he go to jail?