4 ms·
systemic failure involving large number of self-driving cars at the same time This happens to human drivers every year. Snow, freezing rain, fog; these adverse
by chongli 8y ago
systemic failure involving large number of self-driving cars at the same time
This happens to human drivers every year. Snow, freezing rain, fog; these adverse weather conditions can lead to large numbers of collisions at the same time. Expecting self driving cars to never have a problem during rare environmental corner cases is to expect them to be vastly superior to human drivers all the time. If you're waiting for that day before you consent to self-driving cars sharing the road with you, you'll be waiting a long time.
- deleted 8y ago[deleted]
- m0zg 8y ago>> Snow, freezing rain, fog; Well, self-driving cars can currently handle none of the aforementioned conditions. That's why they are being tested where those conditions don't exist, and that's why we're about 20 years from widespread adoption. It rains and snows in much of the rest of the world.
- chongli 8y agoIt rains and snows in much of the rest of the world. Yes, I live in the rest of the world and my family has been affected by fatal collisions brought on by adverse weather. I remain skeptical that self driving cars will be able to handle all of these things. I do think they'll be okay for mundane driving where the biggest risk is driver distractions, however. I would like to see cars and maybe even roads made smarter in order to make driving safer. If all the cars on the road were part of a wireless network and they could signal to one another when they're braking or turning, then other vehicles could respond more quickly. Heck, these systems could even be included in phones and pedestrian crossings so that cars could be signalled instantly when a pedestrian steps into the road in order to cross.
- Swenrekcah 8y agoI used to think that this kind of system must be the future, but now I can only think of the greatly increased attack surface if my self-controlled car is not only taking directions from visual input and other sensors, but also all sorts of other unknown devices.
- chongli 8y agoThe system needs to be sane. The network should not be able to force your car to drive into a wall. Like any safety critical system, it should fail-safe. That means if the data it receives over the network would be unsafe to follow, given information from the other sensors, then it should discount that information as erroneous. This is something humans do all the time when, for example, reading a stranger's intentions.
- Swenrekcah 8y agoBut people are just too creative when it comes to exploiting systems. What if I force someone’s car to halt just under a bridge and then accidentally drop another car on top of it. Oops, a terrible ‘accident’! I realise this is a bit ridiculous but some other people are way more creative than I in this way.
- Cpoll 8y agoWhat if I point a gun at someone and shoot them? As usual, the only thing that keeps you alive is that there are no motivated individuals that want to kill you. If there are, then they don't need you to drive a self-driving car.
- m0zg 8y agoYou're ignoring the possibility of a mass-exploit, meaning the kind of cheap exploit that affects multiple vehicles at the same time, possibly killing a bunch of people, destroying infrastructure, or stopping the flow of traffic. Your typical adversarial "single pixel" change that makes your network believe unreasonable things, but in the real world. The existence of such exploits has already been demonstrated.
- m0zg 8y agoI think there is a potential for them to exceed human performance in all driving conditions. Machines can perceive and react much faster than humans, and once the perceptual and control problems are solved, all of them will become uniformly "good" at driving in difficult conditions. BUT (and there's always a "but"), the problem is exponentially hard, and it can't be solved with today's technology. I agree with you that widespread adoption will require modification of roadways, and potentially also segregation of autonomous and human-controlled traffic streams.
- gambler 8y ago>these adverse weather conditions can lead to large numbers of collisions at the same time. Cars sliding out of control on a snowy road is not a systemic failure. If you have an entire fleet of self-driving cars using the same (or similar) technology, the exact same condition could cause all of them to misbehave in the exact the same way. This could happen simultaneously with a large number of cars. Or could be something that propagates in a cascading manner. (A car breaks, turns and spins out of control. The next car detects that, breaks, turns and spins out of control, etc.) If you have no clear idea how your driving software works, how are you going to make sure this doesn't happen? People have common sense, variability and adaptive learning to prevent things like that. Moreover, our traffic system, our cars and our driver education have been co-evolving prioritizing safety for the last 100 years. (Heck, some of it started with horses and carriages.) The complexity and size of the system have been gradually scaled up for just as long, and it was designed and refined to be used by humans. It's astounding how many of AI enthusiasts fail to see these issues and differences. Traffic system isn't a board game.
- johnwyles 8y ago> If you have no clear idea how your driving software works, how are you going to make sure this doesn't happen? A lot of people don't know how typing "www.google.com" into their browser works but DNS "just works" for them. There is software in nearly every part of our lives that could systemically go wrong in catastrophic ways - we do our best to plan and prevent them. > People have common sense, variability and adaptive learning to prevent things like that. Moreover, our traffic system, our cars and our driver education have been co-evolving for safety for the last 100 years. (Heck, some of it started with horses and carriages.) The complexity and size of the system have been gradually scaled up for just as long, and it's designed and refined to be used by humans. Isn't that incredible? The complexity goes up: we adapt. As self-driving cars are introduced we adapt and evolve and incorporate these changes in technology, the driving "environment", security, safety measures, and laws and regulations. > Traffic system isn't a board game. I didn't see any efforts at self-driving vehicles just say "ahhhh! this driving stuff is so easy!" Of course it comes with huge difficulties, an enormous amount of complexity, a baffling amount of data, a ton of edge cases, and now a wealth of tribal knowledge on the subject. When considering these things think of the systemic benefits as well. There was a story on the Google self-driving cars in Mountain View not taking a turn when the passengers of the car expected it would. Turns out that the IR sensors on the vehicle saw that a jogger behind a wall of hedges was about to enter the intersection to cross while jogging. Systemically implementing that change across a fleet of cars would then save untold numbers of injuries or lives. We adapt.
- nradov 8y agoYes outside of a few limited areas (mostly controlled access highways) and benign weather conditions we're going to be waiting a long time for level 4+ self-driving cars.