9 ms·
Kalashnikov’s new autonomous weapons and the “Terminator conundrum”
- pulse7 9y agoSad truth: it WILL be secretly developed with or without ban... We just discovered yet another self-destruction method... In Smalltalk you could destroy everything with: "Smalltalk become: nil"
- enord 9y agoApplication of unrefined AI in the context of deadly machinery is a real and immediate AI-danger. Self-refining, wolrd-dominating AI is at present time just a distraction from this.
- Anderkent 9y agoNot really, the stakes are completely different. Robot armies might change warfare and balance of state power. They're not going to threaten humanity as a whole.
- dogma1138 9y agoIt's not about threatening humanity it's about threathening humans. I agree with the GP I always said that a dumb AI with sufficient agency in the real world is much scarier now me than some super intelligence. The former is much more likely to be realized and it already has to some extent and unlike the latter cannot be reasoned with.
- enord 9y agoPray tell, how should AI threaten humanity as a whole without a robot army (or some other deadly embodiment)?
- TeMPOraL 9y agoIn a hundred million ways. Systems could be hacked, or humans social-engineered to start a nuclear war or manufacture and release a deadly plague, as some most trivial and devastating examples. Also, threatening humanity is boring and irrelevant. What counts is threatening technological civilization. If we collapse to preindustrial times, it's game over for us anyway.
- greedo 9y agoIf we have a civilization collapse event, then the terminators are done as well. They will be dependent on our technology. The idea of an AI run world is just too far away.
- andrei_says_ 9y agoI can imagine solar powered ai-run portable snipers covering large territories and murdering a lot of civilians over an extended period of time. Think forgotten land mines.
- bigleagueposter 9y agoThe AI could use a human army. If it wanted to eliminate humanity in a not lethal way it could sterilize everyone with a virus or something.
- ClassyJacket 9y agoIt can be worrying to think that aowrson could right now mount a gun to a drone and have it point at things it thinks look like people and shoot. Then I remember that with any amount of technology they could just kill me themselves. Getting hit by a car is a much bigger risk, and the application of AI to prevent that is a much larger positive than the negatives AI introduces.
- enord 9y agoThis remains to be seen. It might be, I don't necessarily disbelieve it, but general handwaving about applying AI to solve a problem is just that.
- dahoramanodoceu 9y agoThat must mean you are not in a middle eastern country. It's self centered to only have concern for your own well-being and myopic to think that, in a world full of self-centered people, with this tech being mass produced, that you'll continue living in a northern-atlantic utopia, where the probable causes of untimely death remain unchanged.
- ClassyJacket 9y agoYou're absolutely right. And it did occur to me, but I suppose I'm talking about things that we expect to change our way of life. I dare say if I was living in a third world country I'd have those problems now. I hope and believe that the forces using AI for good will outweigh the forces using AI for evil.
- aseipp 9y agoAs an American I'm going to go out on a limb and say a big part of this is: Americans in particular are absolutely insulated from mass death. Why wouldn't we be? We've never fought major wars on our own soil in over a century. We haven't had major disease outbreaks killing mass people in like 100 years. We cannot comprehend it; the idea that mass amounts of people die (1,000,000 iraqis, 100,000s of people in warzones like Syria) in relatively short time spans is impossible for us to comprehend. This is the same reason why after 9/11, where a few thousand people died, people all over the country were literally having shared hysteria -- to the point of microwaving mail envelopes (Anthrax attacks!) or buying duct tape and plastic wrap to seal windows (Gas attacks!) They told kids in school about this shit and to prepare for it. It was hysteria, it was not isolated. I think most people willingly choose to believe it doesn't happen at all. A few bombings here and tactical strikes there, twice (or ten) times a day, in a war for 15 years. "Not that bad". Meanwhile we sit around and dream up AI scenarios where they're either A) Sci-Fi novel hellmachines who are bent on total domination of reality unless we "Fight the Machines" or B) useful, peaceful Roombas and car services that run us around and do all our chores and can never do any wrong. Look, my car can get my groceries for me! How fantastic is that! (provided your grocery store was not blown up by a Hellfire missile last week, of course) The third possibility, C, that hostile, unrefined AI will be used to continue to enforce colonial and imperial desires around the world, killing huge amounts of people? Not possible. It's not fanciful or fantastic enough to want to believe, so people don't. It's not a Steven King novel or an Elon Musk jerk-off fantasy. It's just boring old poor people dying. You only need to watch any discussion on Hacker News about "The threat of AI" to see how quickly people start talking about self-imposed God-machines, or talking about Ultra Roombas that do all your earthly chores -- while ignoring murder-robots in 3rd world countries, manned by massive armies. One of those is a fantasy dreamt up by 10 year olds, the other is a reality that already happens today with drone warfare.
- pjc50 9y agoYou've forgotten the logistics chain. A "hostile" AI without any human support is going to run out of fuel, ammo or spares very very quickly. The real threat from AI is automated colonialism. An empire where the people running it never have to get their hands dirty or face rebellion from troops suffering from post-atrocity PTSD. Send in the killbots followed by corpse disposal Roombas. No (human) witnesses.
- greendestiny_re 9y ago>A "hostile" AI without any human support is going to run out of fuel, ammo or spares very very quickly. How about an AI that convinces the humanity to build it an army of androids and then wages war against its creators? Dibs on the idea!
- eb0la 9y agoI guess there will be a market soon for adversarial camouflage clothing in order to hide from roboweapons. In fact I should be writing a business plan for that business now.
- chrisan 9y agoIt also may be a good time to invest in EMP manufacturers
- kbart 9y agoIt's hard to hide from IR using insulating clothes, because of the heat building up inside. But an insulating suit with active cooling might be an interesting idea to explore.
- tormeh 9y agoYou don't need to hide, you just need to look like something other than a human.
- dsr_ 9y agoIf you know where the sensors are, you only need to disguise on that side.
- eb0la 9y agoIn fact you need no disguise: Just the precise amount of interference for the neural network to see something different. https://blog.openai.com/adversarial-example-research/ https://blog.openai.com/adversarial-example-research/
- RobertoG 9y agoI think the grandparent plan is not to hide from the sensors, but from the "mind" behind it.
- qilo 9y agoit seems metallic "invisibility cloak" is quite effective: https://media.ccc.de/v/33c3-8238-retail_surveillance_retail_countersurveillance#video&t=783 https://media.ccc.de/v/33c3-8238-retail_surveillance_retail_...
- mathieubordere 9y agoThis cannot end well.
- ricardobeat 9y agoWe're moving a bit too fast... three years ago this was still science fiction: https://m.youtube.com/watch?v=e9xqdoDc3oI https://m.youtube.com/watch?v=e9xqdoDc3oI
- vanderZwan 9y agoI was expecting this classic scene from Robocop: https://www.youtube.com/watch?v=Hzlt7IbTp6M https://www.youtube.com/watch?v=Hzlt7IbTp6M
- PhantomGremlin 9y agoI'm sure it's only a glitch: https://www.youtube.com/watch?v=ZFvqDaFpXeM https://www.youtube.com/watch?v=ZFvqDaFpXeM
- adzicg 9y agoinstead of WWII films with people hiding in mud for a tank to pass by so they could stick a bomb on it, we'll get WWIII films with people hiding in mud with an ethernet cable
- rimliu 9y agoOh, you reminded me of that horrible horrible TV Series "Scorpion". Could not get past the second episode and watched the second only too see how much more stupid can it get.
- zimpenfish 9y agoWas that the one where they were driving under a plane to hack it?
- PhantomGremlin 9y agoI haven't seen it. But it couldn't be worse than the Independence Day movie from 1996. In that one, the aliens have huge ships; the mothership has one fourth the mass of the moon. Fortunately, Jeff Goldblum figures out how to upload a computer virus into the mothership. My memory of the plot is kind of sketchy, since I certainly wasn't tempted to see it again. https://en.wikipedia.org/wiki/Independence_Day_(1996_film)#Plot https://en.wikipedia.org/wiki/Independence_Day_(1996_film)#P...
- Cthulhu_ 9y agoFlashbacks of Arnold in Predator now. DO IT NOOWWWW!
- chroem- 9y agoI'm curious if the author would also refer to the M4 carbine as an "infamous killing machine," given the opportunity. The fact of the matter is that the US is also developing these systems, albeit with more focus on autonomous air combat vehicles. To say that this is a Russia-only problem vastly understates how serious of an issue this is.
- RobertoG 9y agoBut it's not the same, the article makes it clear here: "Known informally inside the corridors of the Pentagon as 'the Terminator conundrum,' the question being asked is whether stifling the development of these types of weapons would actually allow other less ethically minded countries to leap ahead?" As a not American, I feel totally safe when it's the ethically minded USA who does it. Yep. That was sarcasm, sorry.
- candiodari 9y agoThat was the question until the Kalashnikov announcement. Now the question is, "given that everybody from Putin, to terrorists, Maduro, and North Korea have these weapons, do you want the US government facing them without such weapons ?" In an arms race, you can perhaps refuse to pull ahead (even that is risky), but nothing good will come of falling behind.
- euyyn 9y agoIs there even real competition, though? Doesn't the US already have the military power of the 6 following nations combined, or something like that?
- candiodari 9y agoIf that's supposed to work like a threat to keep others in line, then of course, the US should work to (at least) maintain and (preferably) increase that power. Otherwise the threat won't matter. In other words, this depends on developing and demonstrating any and all weapons capabilities before others develop them, because if just one such technology provides an overwhelming advantage ... Furthermore, there can also be no doubt that the US would use such weapons to maintain it's own and it's allies' security. There can be no doubt that we'd fire such weapons at Russian armies attempting to invade the Ukranian parliament for example. More controversially, there can be no doubt that we'd fire them at not really civilian Palestinians if Israel's army loses control.
- arethuza 9y agoIn what tactical scenarios would an autonomous mobile heavy machine gun make sense? Urban warfare against lightly armed opponents?
- kbart 9y ago"Kill everything that moves" scenarios. Like during Chechnya wars.
- walshemj 9y agoFrom the pics it looks like some of these are HMG mounts I could see Tanks and EVFs having an Ai controlled secondary turrets to reduce crew size. The Soviets did like multi turreted tanks see the T35
- arethuza 9y agoSo like a progression of the protection systems that some MBTs already have? That doesn't sound quite as bad or threatening - although probably very bad news for anyone trying to take on such a vehicle at close range!
- walshemj 9y agoWhich would have to face the accompanying infantry as well Tanks don't do very well in FIBA or Fish And Chips (Fighting In Someone's House, and Causing Havoc In Public Spaces)
- konart 9y agoThings like BMPT-72 Terminator 2 (yeah, T2, really https://en.wikipedia.org/wiki/BMPT_Terminator https://en.wikipedia.org/wiki/BMPT_Terminator) are planned to be semi- or fully-automonous is the near future. So - tank support (instead of sending infantry), clearing missions when you don't really care - just shoot everything that moves, defence turrets like this one - https://www.techcult.ru/content/2016/3690/robot-chasovoj-FLfr.jpg https://www.techcult.ru/content/2016/3690/robot-chasovoj-FLf.... I'd say using autonomous machine gun on a light truck might be useful - no need for a human to risk his life, not to mention this thing will have better aim.
- atemerev 9y agoTo put some context: we Russians always preferred automated systems to having humans in the loop — we have a lot of history with particularly prominent examples of less than fully sane humans in the loop, without proper checks and balances to restrain them. So I am absolutely not surprised of the development, and we'll see much more instances of these systems soon. The only good part is that Russians strongly dislike complicated control systems, so the "AI" in question will be much simpler than what would Americans do.
- RobertoG 9y agoYou have history of good examples of humans in the loop, too: https://en.wikipedia.org/wiki/Stanislav_Petrov https://en.wikipedia.org/wiki/Stanislav_Petrov
- FRex 9y agoRussians' dislike for complicated systems sounds very good but I hope it will have humans watching over it too. There were already many close calls and accidents (some fatal) due to computer and programming errors in both civilian and military systems. Case in point: https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alarm_incident https://en.wikipedia.org/wiki/1983_Soviet_nuclear_false_alar... if it wasn't for Stanislav Petrov the world might have ended back then.
- hyperdunc 9y agoHumans design, program and service these automated systems. They're still in the loop.
- sharemywin 9y agoCan't wait for the new "hot updates from the cloud feature"
- atemerev 9y agoNo need. We have first-year privates with USB sticks instead. (And our USB sticks are, of course, plug-in incompatible with Western USB sticks, and weigh at least 15 kilograms, which greatly aids security).
- baybal2 9y agoFully automatic shoot-to-kill sentry guns are nothing new, and were used at least since eighties. Patent in Russian http://www.findpatent.ru/patent/237/2373483.html http://www.findpatent.ru/patent/237/2373483.html
- jlebrech 9y agobuilding something like that with rubber bullets would be cheaper than THE wall
- trevyn 9y agoWhy would Kalashnikov choose to do a dog and pony show? Is it like the North Korean missle parades and a show of power for Russia? Do they just plain up sell these to certain countries and it's just like any other product launch? Does Kalashnikov have competitors that they are trying to look better than?
- Cthulhu_ 9y agoBecause they're a business and appearing on a trade show is one of the ways they do marketing?
- konart 9y ago>we have NEVER caught the Russians doing ANYTHING beneficial for the Human Race. EVER. These comments though.
- vanderZwan 9y agoI'm not sure what I find more worrying, this, or the other end of the scale: the inevitability that it becomes technically easier and easier to turn cheap small autonomous drones into killing machines.
- agumonkey 9y agoMaybe man will inadvertently create its own predators ..
- jpttsn 9y agoToday, an opponent would try to fire back at the killing machine's "pilot," is that right? With these weapons, the only defense is shooting at the machine itself, breaking it. That reduces your chance of stealing the killing machine after taking out its pilot.
- throwanem 9y ago> Today, an opponent would try to fire back at the killing machine's "pilot," is that right? If you're dealing with an armored vehicle, you're shooting at the vehicle. You can't shoot at the people operating it, because the armor is in the way. If you don't have anything to hand that can deal with the armor, you find something else to do instead. If you do have something that fits the bill, it's not going to leave the vehicle in an operable state after you use it. Killing the crew of an IFV or tank during a combat engagement, and then stealing the vehicle, is something that happens in movies and video games - in real life, not so much.
- FRex 9y agoI read that there have been cases of poorly briefed Ukrainian tank crews escaping and leaving behind tanks that had huge holes in armor despite the armor working as intended and preventing damage to anything important and tank still being fully operational. Separatists took these over and patched/replaced the broken armor piece and used them. It might all have been bullshit though and I can't find that article from years ago(and it wasn't in English anyway).
- throwanem 9y agoClosest I can find is the claim that an IS-3 from late World War II has been reactivated from a war memorial and possibly sent into combat - where it would be unlikely to survive the modern environment for very long, I hasten to note, despite being apparently pretty good for its day. It's not implausible that a modern weapon could punch a hole in tank armor and nonetheless not only fail to achieve even a mission kill, but leave the crew alive. It is really, really improbable, though. I could buy that it'd happened once, although I'd like to see some kind of cite for that. I'd have a hard time believing it had happened twice in the same, as yet rather short, conflict.
- dalbasal 9y agoThere are two significant parts (IMO) to the trend towards autonomous weapons. They're very different. This article seems mostly concerned with autonomous/AI controlled decision making: picking targets and firing on them. The 2nd part taking combatants out of the battlefield. This is more significant, IMO. This applies equally to autonomous & remote operated weapons. For thousands of years, fighting wars meant endangering soldiers. As this changes, all sorts of dynamics change, and conflicts may become more Think of drones. Whatever the tactical implications, I think the strategic implications have been more important. The US is willing to deploy remote operated drones in scenarios where they would not have committed troops, even airforce only.
- kofejnik 9y agoLook, you have to understand something: in Russia, human life has no intrinsic value at all, so it's not a conundrum in any way. New weapon can kill more people, all by itself? Great, bring more of those! If you don't believe me, just read up on Russian history. Maybe start with Ukrainian genocide in 1932-33.
- Grangar 9y agoAnd other arms manufacturing countries are different from this how..?
- Sommersonn 9y agoFor starters, they're not trying to kill their own citizens by millions. Except North Korea maybe.
- throw2016 9y agoThere is a certain inevitability about this. It will and should make many uneasy but like everything else autonomous weapons will advance. Constraints have only worked so far for bio-chemical weapons and even there we don't know who is doing what. Nuclear weapons are easier to restrict because some sort of control can be effected on the underlying technology. The only thing with both the above is the cost of using them is too high, only one country has used nuclear weapons against a population and it remains an unconscionable blot for many of us. Autonomous weapons are much more accessible and will be integrated into modern conflict and law enforcement, shifting the balance in many scenarios.
- throwawaymanbot 9y agoI cant help but feel that the military and its money sucking contractors are in for HUGE disruption. Why even have a military. Just keep a few special forces and augment the rest with Autonomous/AI gadgets such as these. Sell your defense shares.
- smsm42 9y agoReading the article: > In the imminent future, the Group will unveil a range of products based on neural networks," said Sofiya Ivanova, the Group's Director for Communications. "A fully automated combat module featuring this technology is planned to be demonstrated at the Army-2017 forum, So there's no actual product yet, and there's nothing known about the future one except that there are "neural networks" involved, whatever it means. Clearly, there's enough information to make wide-ranging speculations!