15 ms·
I am more fearful of trolls, tricking the technology. I doubt, that the software / hardware will have a common sense, like we humans have. Its this common sense
by einrealist 7y ago
I am more fearful of trolls, tricking the technology. I doubt, that the software / hardware will have a common sense, like we humans have. Its this common sense that keeps us alive in unexpected scenarios, like when the paint on the road is missing.
- bpfrh 7y agoI would argue that signing beacon messages with a PKI infrastructure would migate that issue.
- jethro_tell 7y agoAlso, a stake in the ground on the edge of the road that says, I'm 15 feet to the right of the center lane will probably pay for itself in a couple years of stripping. You put two in the ground that say, I'm 15 feet to the right of the centerline and 35 feet from the next beacon and you never have to paint a centerline there again.
- stevehawk 7y agoBut all you've done is swapped one thing that will be poorly maintained for one thing that's already poorly maintained.
- jethro_tell 7y ago*with a much longer life. Stripping needs to be done every year to stay fresh. You can put a beacon in the ground and I should be able to be there and working 10 years later.
- 0xEFF 7y agoWho is the signing authority? China? The United States? Russia?
- dodobirdlord 7y agoProbably whatever authority is responsible for the sign installation/road maintenance.
- paulryanrogers 7y agoMakes sense assuming that authority doesn't leak their private keys like a sieve.
- dodobirdlord 7y agoProvided they have a way to easily rotate keys via access to the physical hardware it's probably fine. If a local government loses a signing key they would have to register a new one and send a work crew to swap a micro SD card in every traffic light or whatever, but at least that's all they would have to do. In the meantime all an adversary can do with their signing key is install fake beacons, but that requires actually fabricating beacons, and putting themselves at risk by physically installing them in a place they will quickly be noticed, so the potential for damage is pretty limited. Much like how people can already steal stop signs or put up their own speed limit signs, in practice I don't think this will be more than a nuisance.
- paulryanrogers 7y ago> If a local government loses a signing key they would have to... They'd have to know they were compromised first. I think that is a big 'if', and creation of a centralized repository of backdoor keys makes for tempting targets.
- Fnoord 7y agoThose beacons can be DoSed. Consider how Hong Kong protestors avoid the face recognition with laser pens.
- bpfrh 7y agoYeah, but couldn't you also simply disconnect the beacons with a wirecutter? I mean, every technical system can be shutdown and will shutdown sometimes, isn't that a inherent weakness of such a system? if that beacon does not respond disable automation for that part?
- Fnoord 7y agoSure, but I was thinking about temporary disabling (like a temporary, reversible DoS). If you can temporarily disable something, it is more difficult to find the culprit, and the impact is higher because of the surprise element. If something is permanently disabled, you fix it right away. Just like bullying: bullying itself is annoying, but what I found annoying is the inconsistency of it. At some point, my bullies were friendly, at another point once more bullying. If it were permanently on-going and not sneaky, they'd be found out by e.g. teachers long ago already.
- bpfrh 7y agoAhh, yes that would be a problem. On the other hand, wouldn't a ddos be visible in the logs? Logs should be checked centrally anyway, so a 30 second flood of messages should be visible. But in the end, you are right that it will be hard to do this the right way.
- mavhc 7y agoStill better than humans, humans kill 3000 people using vehicles every day, and most of those aren't even trying
- est31 7y agoYeah but if the system is designed in the wrong way, hackers could kill millions in a single moment. Admittedly, this danger already exists as in all modern cars between driver and car there is a layer of (hackable and wireless network connected) software.
- the-pigeon 7y agoHackers could hijack a missile and kill a lot of people too. This is why we have security. And honestly modern cars are so computerized already I think your argument applies to them as much as autonomous cars.
- ben_w 7y ago“We’re already doomed” isn’t a good reason to not be concerned about new ways to be doomed. Yes, we have security. Yes, cars can (probably) already be hacked, perhaps even en-mass rather than in targeted ways. But: one the big advantages of machine learning in cars is that every car can learn from the experiences of every other car. That makes them monocultures. Monocultures are fragile. You find the weakness of one, you find the weakness of all. I want the benefits of the former without the risks of the latter. I don’t know if that’s even possible.
- umvi 7y ago> I want the benefits of the former without the risks of the latter. I don’t know if that’s even possible. That's where competition comes in. Just like AMD vs. Intel or OSX vs. Windows. If you find a weakness in one, you don't necessarily find a weakness in the other. Hence, finding a weakness in Tesla doesn't mean it will work against Waymo or Uber.
- jstummbillig 7y ago> I doubt, that the software / hardware will have a common sense, like we humans have Aren't sanity checks basically common sense? I would suspect they already play a major role in making autonomous driving a reality.
- Groxx 7y agoI'd argue those are the opposite - due to the software/hardware failing to have common sense, we've added some extremely coarse checks to shut it down when it's clearly going nuts. Less "common sense", more "don't push this button, it will kill you [presses button anyway]"
- sjy 7y agoCommon sense is not just a large number of sanity checks, it also includes the ability to solve some of the problems identified by failed sanity checks. For example, common sense might lead you to break the road rules in an emergency, or drive a car with a safety problem for a short distance instead of getting it towed.
- rsj_hn 7y agoImagine something like a dangerous situation -- there is an oncoming fire or flood, and you need to get out of there, but your self-driving car decides to do a precautionary stop because one of the sensors stopped working. People are totally underestimating the intelligence required to make these types of decisions. Surrendering your mobility to a program is a much bigger hurdle than people think, and the bar is much higher than being able to successfully avoid collisions.
- beerandt 7y agoMore then solving problems, common sense involves identifying a problem as a problem. Unknown unknowns.
- MauranKilom 7y agoYou might want to consider https://xkcd.com/1958/ https://xkcd.com/1958/.
- mavhc 7y agoRight, the main question is can one murderer murder lots of people at once with new technology. Can they buy a machine gun, or reprogram every car to crash, or lobby the government to force doctors to prescribe the opiates they sell.
- cameronh90 7y agoWhat's stopping people from destroying yield signs and other safety critical road infrastructure now? Sometimes common sense helps, but it you're out to cause chaos, there's a ton of things you can do today that would cause crashes - but most of the time, people don't.
- aetherson 7y agoIf you destroy a yield sign, maybe it will eventually cause some problems, or maybe not. We could imagine a world in which destroying smart infrastructure could pretty instantaneously and reliably cause massive traffic jams, which seems like it would appeal more to certain trolls and perhaps protestors. We could also imagine a world in which more subtle attacks could reliably and fairly quickly cause injury/fatality accidents, which might appeal to terrorists.
- jameslevy 7y agoI think this type of "stress testing" of destroyed or modified signage would be an obvious direction for AV development once the basics are down. With HD mapping and other technology used to augment what is gathered by their sensors, AVs are eventually overall going to be better at handling unexpected scenarios at the road compared to humans
- TwoBit 7y agoIf you destroy a yield sign, the system will immediately know because all the cars previously recorded it.
- incompatible 7y agoThe permanent signs would already be in the database. The signs you'd worry about would be temporary, for road works or accidents, but perhaps these locations could be made electronically available.
- blub 7y agoAnd it will play a helpful message like "Recalculating route: 3 self-driving fatalities registered at this location."
- malandrew 7y agoIt would actually be easier to make smart infrastructure tamper proof relative to existing infrastructure.
- Zarel 7y agoThis is definitely one of the things xkcd has talked about: https://xkcd.com/1958/ https://xkcd.com/1958/
- peteretep 7y agoTrolls can also drop bricks on cars from overpasses, and I suspect fucking with road signs will rapidly carry the same taboo and punishment
- Kuinox 7y agoThere is an ia that we try to learn it common sense.
- scottlocklin 7y agoWhile "autonomous vehicles" are the biggest practical joke of the 2010's, this artists joke trapping an "autonomous" car in a circle of salt like a medieval demon was pretty good: https://imgur.com/gallery/24jLd https://imgur.com/gallery/24jLd
- threatofrain 7y agoYou can already screw with infrastructure to kill people. Sometimes we just have to hope that the combination of positives and negatives convince people not to go out and screw the world.
- BurningFrog 7y agoEven current tech can easily drive a car on a road completely without digital beacons. We won't lose that capacity.
- SirLJ 7y agoTry up north during the winter...
- mrfusion 7y agoYou realize trolls can wreak havoc with traffic today if they wanted to?