9 ms·
i hope they keep that bug locked down. if that was weaponized there would be an issue for sure.
by rolph 6y ago
i hope they keep that bug locked down. if that was weaponized there would be an issue for sure.
- Animats 6y agoIt's the usual Tesla bug - plowing into a stationary vehicle partly blocking a lane. Just like the fire truck, the street sweeper, the stalled car...
- Traster 6y agoAt what point does that become criminal negligence?
- trhway 6y agoit seems we're living at the golden age of autonomous driving development as even Uber killing a woman due to disabled emergency braking didn't make it even to the negligence (granted Uber showing poor dynamic range video recorded by what seems like a cheap cell phone camera to the sheriff in that village (Phoenix) also played the trick - that sheriff can be fooled by such a video is the major part of why it is the golden age right now). The same way with Tesla - notwithstanding me being a big fan of Musk - self-driving without 3d scene building is pure arrogance and negligence which is enabled only by general public and government not-understanding of the things - thus the golden age.
- toomuchtodo 6y agoFunny enough, it happens with human drivers too. Perhaps the fix is to block the entire lane instead of partially blocking it. At least the Tesla will stop for such a scenario, where a human still might not. Can't be that expensive to roll an additional rescue vehicle to sit in the lane with their lights on (I see it quite often in Florida, unsure about other states). > According to recent data taken from the AAA Foundation for Traffic Safety, approximately 12 percent of all interstate highway deaths result from shoulder accidents. This means that an estimated 600 people a year are killed—and thousands more are injured—while making emergency stops off the highway. https://aaafoundation.org/wp-content/uploads/2017/12/PedestrianFatalitiesonInterstatesReport.pdf https://aaafoundation.org/wp-content/uploads/2017/12/Pedestr... It's unfortunately fairly difficult to find stats on accidents scoped solely to vehicles rear ending emergency vehicles partially obstructing a travel lane. The stat I present is a proxy.
- throwaway0a5e 6y agoRunning into the back of a police cruiser that's partly in the lane with lights on is something humans generally only do before blowing a .1x on the breathalyzer. The link you posted shows a clear correlation between low light conditions and hitting people on the shoulder. A police cruiser has blue lights and presumable the federally mandated reflective surfaces that all vehicles sold in the US have (usually in the tail lights). This is a typical case of software making stupid mistakes a human would never make.
- toomuchtodo 6y agoIt's a fairly common problem even without being under the influence. > Distracted driving is dangerous, claiming 3,142 lives in 2019. https://www.nhtsa.gov/risky-driving/distracted-driving https://www.nhtsa.gov/risky-driving/distracted-driving > The National Safety Council reports that cell phone use while driving leads to 1.6 million crashes each year. Nearly 390,000 injuries occur each year from accidents caused by texting while driving. 1 out of every 4 car accidents in the United States is caused by texting and driving. https://www.nsc.org/road-safety/safety-topics/distracted-driving https://www.nsc.org/road-safety/safety-topics/distracted-dri...
- throwaway0a5e 6y agoYour statistics are for human drivers hitting cars and people. Not objects with blue lights (intentionally designed to be highly visible) that can be seen for miles at night being hit by vehicles that are augmented with computers that don't get distracted. Stop being intentionally misleading. An always-on computer hitting a police cruiser (the rear end of which conforms to known standards regarding number, position and size of reflective surfaces) is nowhere near equivalent to someone hitting something because they were looking at their phone. Is it too much to expect technology to at least avoid the kind of accidents only incapacitated humans have?
- Aeronwen 6y agoThose lights are blinding as hell at night. The only reason a person wouldn't have hit the car is because it's the only thing you can see when trying to pass one.
- gpm 6y agoIt's the usual AI bug, crashing on a rare edge case (i.e. not well represented in the training data) that seems straightforward logically to a human, leading to us being surprised/shocked/horrified that the AI missed the issue. The most interesting part of this, is that this is a case that humans screw up on a pretty regular basis too. However we understand the human reason for doing so (lack of concentration), and understand that AIs don't suffer from the same fault. So when an AI screws it up we are still surprised.
- shagie 6y agoThe issue is that the AI will do the exact same thing each time it is presented with the same data (crash) whereas a human could go to remedial driver's education. And taking away the license of one inattentive driver gets them (theoretically) off the road... whereas if you have an AI crash into a fire truck and you said "stop it" - you still have all the other identical deployments out there with the same flaw. And "add some partial blockage by a vehicle in the lane, retrain the AI, and then redeploy it" isn't a proper answer... wouldn't that be a different model AI that would need to be validated again?
- deleted 6y ago[deleted]
- gpm 6y agoYou can turn this around though. It's harder to teach the AI, sure. But once you've taught it, it stays taught. With humans you have to deal with the continually renewing stream of young people who are new to driving. > And "add some partial blockage by a vehicle in the lane, retrain the AI, and then redeploy it" isn't a proper answer... wouldn't that be a different model AI that would need to be validated again? You're the one adding unnecessary constraints here. If the validation process harms more than helps, you don't do it. In all likelyhood the correct validation process is something lightweight that can be done quickly for simple changes. This sort of training process is likely to be a continuous process, they're always ingesting new data, trying to learn how to drive better from it (and keep up with how patterns in the world change), and therefore continually deploying new models. Because that makes driving safer overall. Not a "this bad thing happened so let's figure out how to put out a bugfix for it".