4 ms·
Although I'm personally optimistic about driverless cars, traffic is a distributed autonomous-agent system: a failure of one driver is compensated for by other
by derefr 11y ago
Although I'm personally optimistic about driverless cars, traffic is a distributed autonomous-agent system: a failure of one driver is compensated for by other drivers. Someone does a "rude" merge, or an illegal U-turn—and everybody just modifies their own driving to let it happen, even though it's "against the rules." On the road, I'm not really trusting other drivers individually; I'm trusting traffic as a fluid.
My big worry is that we'll have autonomous-car-AI monoculture. If that happens, we'd lose the advantage of the "compensation" other drivers in traffic can do. We could end up seeing car-behavioral "prions": glitches in car software that cause a strange behavior, which in turn triggers the same glitch in the cars attempting to compensate for that behavior. You could see, for example, every car getting into a state where it thinks it needs to swerve "more right than other cars" to avoid an emergency, and thus an entire "pack" of cars driving off the road and into a fast clockwise-torquing circle.
Or, less eyecatching but more fatal in the near-term, you could see a monoculture-wide decision that works alright for single driverless cars, but actively harms drivered cars (and other driverless cars with other-model AIs) when the majority of a "pack" of cars is from the monoculture. For example, I could see a "pack" of driverless cars all deciding to jam on the brakes at once (because, say, a bunch of spike-strips suddenly popped out of the road in between them or somesuch)—leaving the one drivered car boxed in by them to keep going and crash into the vanguard of the pack.
- ahussain 11y agoI don't think this works because you can have very safe failure modes - much like nuclear reactor control rods. Instead of all swerving to the right the cars would all slow down as quickly as it is safe to do so. In the second example the situation is no worse than a human driver slamming on the brakes. Sure there will be some accidents, but nowhere near the same frequency as today. Autonomous cars are much better/faster at "compensating" than humans are.