3 ms·

> if the CV algorithm fails against these examples when humans don't, then the CV algorithm is too brittle and should not be used in the real world. This is th

by mabbo 5y ago

> if the CV algorithm fails against these examples when humans don't, then the CV algorithm is too brittle and should not be used in the real world. This is the tricky bit. Night-time driving, bad weather, icy roads, bumper-to-bumper traffic: these are all situations in which some algorithms can outdo humans in terms of safety. Faster reactions, better vision (beyond what human eyes can see), and unlimited 'mental stamina' can make a big difference in safe driving. But then there will be the occasional situation in which the CV screws up, and there's an accident. Some of those are ones where many/most humans could have handled the situation better and avoided the accident. So how do we decide when the automated car is 'good enough'? Do we have to reach a point where in no situation could any human have done better? Must it be absolutely better than all humans, all the time? Because we may never reach that point. And all the while, we could be avoiding a lot more accidents (and deaths) from situations the AI could have handled.

glitchc 5y ago
> Night-time driving, bad weather, icy roads, bumper-to-bumper traffic: these are all situations in which some algorithms can outdo humans in terms of safety. Faster reactions, better vision (beyond what human eyes can see), and unlimited 'mental stamina' can make a big difference in safe driving. To be clear we are talking about CV which relies on passive optical sensing in the visual spectrum through cameras, not radar or lidar or IR or multi-spectral sensors. Within this context, your statement is incorrect. A typical camera’s dynamic range is orders of magnitude lower than the human visual dynamic range. Ergo a camera sees a lot less at night compared to a human and what it does see is a lot more noisy. Note that this is the input to the detection, tracking and classification stages, the ouput of which feeds into the control loop(s). It doesn’t matter how good the control system is, it cannot avoid what the vision system cannot see.
- astrange 5y ago
  > To be clear we are talking about CV which relies on passive optical sensing in the visual spectrum through cameras, not radar or lidar or IR or multi-spectral sensors. Well, I think you mean working off RGB data? That's not necessarily the problem you have to solve even if your parts are regular cameras, as long as they're dedicated to your uses. You can modify them to see IR or polarization.
semi-extrinsic 5y ago
> So how do we decide when the automated car is 'good enough'? This is actually a really interesting point. I don't think people appreciate how far accident rates have actually dropped for modern cars without self driving. Even at million-cars-per-year sales rate you will need years of data to prove that a single self-driving software+hardware combo is better than humans with high statistical confidence. Your development cycles would be decades-long, like in aviation, if you want to be sure you're actually improving.
- mgraczyk 5y ago
  Fortunately waymo and others do have years of data. Also, you can make reasonable inferences about fatal accidents using non-fatal accidents. All fatal accidents are also normal accidents. If waymo has far fewer non fatal accidents, you can reasonably infer it would have fewer fatal accidents. Otherwise you'd have to believe waymo's accidents are more likely to be fatal, but the opposite is probably true because of the locations and speeds where they drive (at least for the passenger lol) You can also make inferences about accidents based on disengagements or undesirable events (human labeled). It's not as data limited as you might think.

by mabbo 5y ago

glitchc 5y ago

> Night-time driving, bad weather, icy roads, bumper-to-bumper traffic: these are all situations in which some algorithms can outdo humans in terms of safety. Faster reactions, better vision (beyond what human eyes can see), and unlimited 'mental stamina' can make a big difference in safe driving. To be clear we are talking about CV which relies on passive optical sensing in the visual spectrum through cameras, not radar or lidar or IR or multi-spectral sensors. Within this context, your statement is incorrect. A typical camera’s dynamic range is orders of magnitude lower than the human visual dynamic range. Ergo a camera sees a lot less at night compared to a human and what it does see is a lot more noisy. Note that this is the input to the detection, tracking and classification stages, the ouput of which feeds into the control loop(s). It doesn’t matter how good the control system is, it cannot avoid what the vision system cannot see.

astrange 5y ago
> To be clear we are talking about CV which relies on passive optical sensing in the visual spectrum through cameras, not radar or lidar or IR or multi-spectral sensors. Well, I think you mean working off RGB data? That's not necessarily the problem you have to solve even if your parts are regular cameras, as long as they're dedicated to your uses. You can modify them to see IR or polarization.

semi-extrinsic 5y ago

> So how do we decide when the automated car is 'good enough'? This is actually a really interesting point. I don't think people appreciate how far accident rates have actually dropped for modern cars without self driving. Even at million-cars-per-year sales rate you will need years of data to prove that a single self-driving software+hardware combo is better than humans with high statistical confidence. Your development cycles would be decades-long, like in aviation, if you want to be sure you're actually improving.

mgraczyk 5y ago
Fortunately waymo and others do have years of data. Also, you can make reasonable inferences about fatal accidents using non-fatal accidents. All fatal accidents are also normal accidents. If waymo has far fewer non fatal accidents, you can reasonably infer it would have fewer fatal accidents. Otherwise you'd have to believe waymo's accidents are more likely to be fatal, but the opposite is probably true because of the locations and speeds where they drive (at least for the passenger lol) You can also make inferences about accidents based on disengagements or undesirable events (human labeled). It's not as data limited as you might think.