3 ms·

The great risk of AI to survival is in military applications, I believe. Not because of super-intelligent Skynets - in fact their lack of general intelligence m

The great risk of AI to survival is in military applications, I believe. Not because of super-intelligent Skynets - in fact their lack of general intelligence may make them more dangerous - but because of superhuman response times. While banning military AI is as useful as banning nuclear weapons (impossible on a practical level), I feel that Google and Facebook and others should be leaders and address these issues, rather than conveniently bury their heads because the problem, though unavoidable, is still on the horizon, difficult, and disturbing and inconvenient to their happy visions. Imagine a human stock trader trying to compete with a 'flash' trading program, which operates on the scale of (micro?)-seconds. Now imagine they have guns and are trying to kill each other. Imagine that the survival of millions, of your nation, of democracy and liberty depend on winning. The military has no choice but to build robots that autonomously decide and act to kill people and to destroy the things that people build; it's that or surrender and learn to speak Russian (or Chinese or the language of whoever does build those robots). The U.S. military already has systems that do this, such as the guns that defend ships from incoming missiles - no human could identify, aim at and shoot down a supersonic missile quickly enough. They plan to implement it in other situations such as, interestingly for HN readers, defending and attacking IT systems: Imagine an AI that can launch and adjust attacks in micro- or even just milli-seconds, while the human sysadmin's brain is just having the first inklings of thought about the response procedure to the first attack. Again, either surrender or use autonomous AIs yourself. AIs and associated military robots raise other serious issues: Throughout human history military power was tied closely to the magnitude of wealth and population; this restricted the major threats to just a few great powers. But perhaps you will only need to build a robot army, not a human one; one good AI development team and an underground manufacturing facility (anywhere in the world) might be sufficient; your AI doesn't have to be precise or make good decisions, it only needs to kill so rabidly that others surrender. Nuclear weapons at least require very specialized materials, limiting potential manufacturers (though even NK can do it now). Could Singapore do it and become the Rome of the future? Saudi Arabia? Japan? Finland? In fact, who needs a nation - wealthy individuals, groups, or corporations might pull it off. What about GE, Boeing, Google - and those are just a few American companies. Putin recently said that "the one who becomes the leader in this sphere will be the ruler of the world."

otakucode 9y ago
Well first off any time someone wants to try making their technology the centerpiece of their war effort, you just ignore the technology. You attack the people controlling it, the people building it, the infrastructure, etc. And yeah, all the 'civilized' countries have invested very heavily into ballistic weaponry and agreed that's how they will fight their wars... but when it comes to survival, to wondering whether what is left after the war will even be worth preserving, those biological and chemical weapons aren't going to be left on the table. The US' view of drone technology really disturbs me. They see drone strikes as 'not warfare' and have permitted civilian agencies with no military involvement to conduct them (the CIA). I'm not sure if they realize that a drone could fly quite easily in the air above Manhattan or not. Most likely they will simply do the slimey thing of waiting until that happens and THEN deciding it's an act of war when somebody else does it. If they can even attribute it. I'd expect such uses of drones against a capable adversary to transform the control rooms in Kansas into the primary target. You can shuffle war around all you want, but no adversary you are trying to kill is going to just sit back and let themselves be picked off while you and yours stay safe at home, expecting the robots to take the fall. It's war. They're going to show up wherever your blood-filled bodies are and they're going to destroy them. If it's truly overwhelming, they might just decide to salt the earth and leave you nothing to rule, destroying whatever it was you were killing them for just to spite you. The only winning move is not to play. But aside from that, the trading example you mentioned is a good one. So if you build a system that is splendidly fast at reacting... you won't have gotten much of anywhere. The arbitrage markets, where that kind of thing is valuable, are already owned by people rich enough to make sure you never get a piece. In the typical market, those systems have a weakness. They can't account for their own impact upon the market. This is a fundamental problem, and not one which can be solved with more computing power or even better algorithms. In order to accurately predict a system that is impacted by the actions you take in it you need a system bigger than what you're simulating. So until the whole of the global human economy can be simulated at a speed greater than realtime, it probably won't be a problem there. The systems will also face problems where they encounter unexpected feedback loops when interacting in the same market as other automated systems. There are dangers, though. My primary worry is turning machine learning systems loose on fundamentally unpredictable data, and then reading the systems outputs like the meaningless tea-leaves they are but giving them total trust. We know you can't predict 'black swan' events, but that doesn't stop people from claiming they can. And nothing about machine learning will change the fundamentals like 'you can't predict a chaotic systems whose initial condition you can't even know.' A high school in Dubai bought a 'crime prediction' ML system and presumably is using it to target students before they do anything wrong. I'll be surprised if that's not debuted in US schools within 5 years. And probably on the street in the UK on the same timeframe. The sad thing is, if the systems actually work (in the sense of actually calling out bad things before they're done), no one will be satisfied. I graduated high school right before Columbine happened. I watched as schools adopted these lists of "warning signs" and used them to target students and persecute them under the guise of 'preventing school shootings'. There was one problem with those lists that nobody mentioned. A list of warning signs that was accurate would identify 1 or 2 kids in 1 school every 5-10 years. Instead, the lists identified a couple dozen kids. In every school. Every year. That's what people actually WANTED. They just wanted tacit approval to grind 'problem kids' under their thumb while claiming safety as their virtuous goal. Mean little micro-dictators pretending to be superheroes while indulging the worst flaws of their own character. And I expect the same will be the outcome of the 'prediction' systems that eventually come to dominate society. Just a way to hide going after 'troublesome' people who don't fit into the spaces on the forms the policy enforcers have.

otakucode 9y ago

Well first off any time someone wants to try making their technology the centerpiece of their war effort, you just ignore the technology. You attack the people controlling it, the people building it, the infrastructure, etc. And yeah, all the 'civilized' countries have invested very heavily into ballistic weaponry and agreed that's how they will fight their wars... but when it comes to survival, to wondering whether what is left after the war will even be worth preserving, those biological and chemical weapons aren't going to be left on the table. The US' view of drone technology really disturbs me. They see drone strikes as 'not warfare' and have permitted civilian agencies with no military involvement to conduct them (the CIA). I'm not sure if they realize that a drone could fly quite easily in the air above Manhattan or not. Most likely they will simply do the slimey thing of waiting until that happens and THEN deciding it's an act of war when somebody else does it. If they can even attribute it. I'd expect such uses of drones against a capable adversary to transform the control rooms in Kansas into the primary target. You can shuffle war around all you want, but no adversary you are trying to kill is going to just sit back and let themselves be picked off while you and yours stay safe at home, expecting the robots to take the fall. It's war. They're going to show up wherever your blood-filled bodies are and they're going to destroy them. If it's truly overwhelming, they might just decide to salt the earth and leave you nothing to rule, destroying whatever it was you were killing them for just to spite you. The only winning move is not to play. But aside from that, the trading example you mentioned is a good one. So if you build a system that is splendidly fast at reacting... you won't have gotten much of anywhere. The arbitrage markets, where that kind of thing is valuable, are already owned by people rich enough to make sure you never get a piece. In the typical market, those systems have a weakness. They can't account for their own impact upon the market. This is a fundamental problem, and not one which can be solved with more computing power or even better algorithms. In order to accurately predict a system that is impacted by the actions you take in it you need a system bigger than what you're simulating. So until the whole of the global human economy can be simulated at a speed greater than realtime, it probably won't be a problem there. The systems will also face problems where they encounter unexpected feedback loops when interacting in the same market as other automated systems. There are dangers, though. My primary worry is turning machine learning systems loose on fundamentally unpredictable data, and then reading the systems outputs like the meaningless tea-leaves they are but giving them total trust. We know you can't predict 'black swan' events, but that doesn't stop people from claiming they can. And nothing about machine learning will change the fundamentals like 'you can't predict a chaotic systems whose initial condition you can't even know.' A high school in Dubai bought a 'crime prediction' ML system and presumably is using it to target students before they do anything wrong. I'll be surprised if that's not debuted in US schools within 5 years. And probably on the street in the UK on the same timeframe. The sad thing is, if the systems actually work (in the sense of actually calling out bad things before they're done), no one will be satisfied. I graduated high school right before Columbine happened. I watched as schools adopted these lists of "warning signs" and used them to target students and persecute them under the guise of 'preventing school shootings'. There was one problem with those lists that nobody mentioned. A list of warning signs that was accurate would identify 1 or 2 kids in 1 school every 5-10 years. Instead, the lists identified a couple dozen kids. In every school. Every year. That's what people actually WANTED. They just wanted tacit approval to grind 'problem kids' under their thumb while claiming safety as their virtuous goal. Mean little micro-dictators pretending to be superheroes while indulging the worst flaws of their own character. And I expect the same will be the outcome of the 'prediction' systems that eventually come to dominate society. Just a way to hide going after 'troublesome' people who don't fit into the spaces on the forms the policy enforcers have.