7 ms·

Great points regarding aligning precision/recall of AI systems with actual human supervision capabilities. One quibble: I hate seeing articles uncritically rep

Great points regarding aligning precision/recall of AI systems with actual human supervision capabilities. One quibble: I hate seeing articles uncritically repeat Uber's line that a human driver could not have reacted in the same scenario. The dashboard cam footage they released does not accurately represent either human or car perceptual capabilities [1], and folks recreating the scene have shown that it doesn't even look like a good (or unaltered) example of what a dashcam should have perceived [2]. [1] https://ideas.4brad.com/it-certainly-looks-bad-uber https://ideas.4brad.com/it-certainly-looks-bad-uber [2] https://dgit.com/residents-recreate-crash-uber-car-case-56161/ https://dgit.com/residents-recreate-crash-uber-car-case-5616...

mark-r 8y ago
There's no way of knowing if a human driver could have reacted properly in this specific scenario. However if you live in an area where deer frequently cross the road, you'll know that collisions with deer occur regularly; that would indicate that it's easy to hit things that cross your path, even when you should be able to see them.
- rpedela 8y ago
  True, but a human would have applied the brakes at least which may not avoid the collision but would reduce the injuries. We can never know but maybe a human would have been able to stop. Or maybe a human could have braked enough to prevent death. Or maybe the exact outcome would have happened even if the brakes were applied.
  - jimbofisher1 8y ago
    In fact, as anyone who has read the NTSB report would know, a Human was responsible for braking in this situation and FAILED to do so in time. The above discussion is irrelevant and results from ignorance of the facts. From the report: ". At 1.3 seconds before impact, the self-driving system determined that an emergency braking maneuver was needed to mitigate a collision (see figure 2).[2] According to Uber, emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior. The vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator." For anyone interested in actually being informed about this incident. https://www.ntsb.gov/investigations/AccidentReports/Pages/HWY18MH010-prelim.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HW...
    - rpedela 8y ago
      There is going to be a reaction time difference between someone actively driving the vehicle and someone who is monitoring an automated system. It is human nature to get bored and not pay close enough attention. This is exactly why some of the Tesla autopilot crashes have happened in addition to the system screwing up.
      - Animats 8y ago
        Yes. I've pointed that out previously. NASA has a simple tool for studying this, the Multi-Attribute Test Battery.[1] It uses a Windows PC and a joystick. The person being tested uses the joystick to center a marker in a square. The marker tends to drift away, and has to be corrected. This simulates the most basic task of flying - flying straight and level. There's an "autopilot", which can be turned on to keep the marker centered. The user can then let go of the joystick. But sometimes the "autopilot" fails, and the marker drifts out of the square. How long it takes for the subject to notice is measured. That's a good approximation for Tesla's semi-automatic driving system. There are lots of papers on this. It's well understood in aviation that takeover time after a fault is seconds to tens of seconds. [1] https://matb.larc.nasa.gov/ https://matb.larc.nasa.gov/
        deleted 8y ago
        [deleted]
  - mark-r 8y ago
    A human could also have attempted swerving to avoid the accident. The question that I feel can't be answered is how visible the pedestrian would have been under the conditions and if there would have been time to take action. Even braking to half the recorded velocity would likely have been fatal.
    - davidgould 8y ago
      > Even braking to half the recorded velocity would likely have been fatal. This is not so. Half would have been under 25 mph which is much less than 50% fatal. See http://www.humantransport.org/sidewalks/SpeedKills.htm http://www.humantransport.org/sidewalks/SpeedKills.htm
- evrydayhustling 8y ago
  > There's no way of knowing if a human driver could have reacted properly in this specific scenario. What? Why? The specific scenario is recorded, the surrounding conditions are known and relatively easy to reproduce, and there are both norms and requirements about human capabilities when driving a car. If we can't attempt to determine whether a human could have responded in this case, I'm not sure where we could determine anything about any kind of accountability. People obviously make mistakes too -- I've hit a deer before, unfortunately. But when the consequences are high, we look into culpability about whether someone should have done more to prevent the mistake. Driving under the influence, without a license, etc. are all things that can make someone responsible for the big outcome of a relatively easy-to-make mistake. Accepting Uber's "could you react to this video I'm showing you?" defense is skipping all the steps that would let us assign responsibility or even learn / improve from the incident.
  - mark-r 8y ago
    The comment I was responding to specifically mentioned the dash-cam video, which I hope we can all agree would not be representative of what the operator could see. Nor would any of the on-board sensors be relevant. I suppose they might provide enough information to do a re-enactment. At no point did I suggest that this would be an excuse to absolve Uber of any responsibility.
    - smilekzs 8y ago
      I disagree to a minor degree, because humans can have widely different ability of night vision.
  - bronco21016 8y ago
    The miracle on the Hudson comes to mind in this case. After the event there were some wondering if the crew could have safely glided to an airport. Investigators painstakingly recreated the event in a simulator and ran several trials. The issue with the experiment? The crew in the simulator knew what was coming. They were able to make the field because they were already primed to turn for the field immediately without any time to think. I’m sure you could do a blind study of this scenario but I’m sure people participating in a driving study in a simulator are not going to be driving as they do in real life. We see it every single day in the airline industry. It’s why we’re checked and observed at random intervals while out flying on the line. The surprise aspect gives you no time to prepare or study up to be in perfect shape like you do when you’re headed into a training event.
- ummonk 8y ago
  Not in locations with substantial street lighting. It's usually in an unlit road that you get the danger (especially if you aren't using high beams due to an oncoming car).
- jimbofisher1 8y ago
  The human operator was responsible for braking in this situation and failed to do so. https://www.ntsb.gov/investigations/AccidentReports/Pages/HWY18MH010-prelim.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HW...
  - petre 8y ago
    What's the point of a self/assisted driving vehicle then? If it doesn't compensate for human error, then it's useless, since humans can already operate vehicles.
  - mark-r 8y ago
    That was my point, human operators are responsible for not hitting deer yet it happens every day. The only reason it doesn't happen to people more often is that people are better at avoiding running in front of cars.
    - romwell 8y ago
      >That was my point, human operators are responsible for not hitting deer How is this even comparable? A deer jumping in front of a car is not the same as a person walking the bicycle slowly across several lanes (and almost making it to the other side) on a rather well-lit road.
      - zaarn 8y ago
        Because the human is currently monitoring the car to not hit a deer, as other comments mentions, it can take dozens of seconds to regain full awareness.
        jimbofisher1 8y ago
        Dozens of seconds? Have you ever even driven a car? lol
        zaarn 8y ago
- JackCh 8y ago
  A few years ago I hit a deer. Had the deer been waiting to jump into the road from on a sidewalk instead of behind a bush, I would not have hit it. The reason I hit the deer is because I didn't know the deer was there. When people hit deer, it's not because the deer was casually walking across the road like a human jaywalker. It's because the deer jumps into your path in an instant. As somebody who's hit a deer and done plenty of urban driving with no shortage of jaywalkers, I can assure you there is no fair comparison. The only sort of situation that might be comparable is if somebody is trying to commit suicide by car and deliberately leaps in front of you, or if a small child concealed by a car parked by the side of the road runs out into the road suddenly. But those scenarios are unrelated to this incident.
YeGoblynQueenne 8y ago
>> Great points regarding aligning precision/recall of AI systems with actual human supervision capabilities. May I nitpick? The article discusses False Positive Rate and False Negative Rate. These are, respectively, the complements of True Negative Rate and True Positive Rate a.k.a. Sensitivity, a.k.a. Recall. Precision is not the same as False Negative Rate. Specificity is, as in Sensitivity/Specificity. These metrics tend to be reported in pairs: TPR/TNR, FPR/FNR, Precision/Recall, Sensitivity/Specificity, which confuses the issue- but not all pairs are the same.
- tcmb 8y ago
  To add to that, since I see it nowhere mentioned explicitly, neither in this thread nor in the article: the theoretical framework here is Signal Detection Theory (SDT) [1]. The problem at hand can be nicely visualized with two overlapping bell curves for the signal present/absent situations [2]. [1] https://en.wikipedia.org/wiki/Detection_theory https://en.wikipedia.org/wiki/Detection_theory [2] https://jdlee888.shinyapps.io/SDT_Demo/ https://jdlee888.shinyapps.io/SDT_Demo/
  - sgt101 8y ago
    Two points - first, these frameworks don't really account for driving into concrete bollards vs. a shaky ride. Second none of these frameworks estimate production performance well; time after time after time I see classifiers that did "95%" in test and "88%" in prod and "that's pretty good!". My point is that we are really bad at estimating performance of classifiers. Really bad, and mostly we are pretending it's fine.
SiempreViernes 8y ago
A better source for the example videos showing the light levels at that point is the arstechnica page being cited: https://arstechnica.com/cars/2018/03/police-chief-said-uber-victim-came-from-the-shadows-dont-believe-it/ https://arstechnica.com/cars/2018/03/police-chief-said-uber-...

mark-r 8y ago

There's no way of knowing if a human driver could have reacted properly in this specific scenario. However if you live in an area where deer frequently cross the road, you'll know that collisions with deer occur regularly; that would indicate that it's easy to hit things that cross your path, even when you should be able to see them.

rpedela 8y ago
True, but a human would have applied the brakes at least which may not avoid the collision but would reduce the injuries. We can never know but maybe a human would have been able to stop. Or maybe a human could have braked enough to prevent death. Or maybe the exact outcome would have happened even if the brakes were applied.
- jimbofisher1 8y ago
  In fact, as anyone who has read the NTSB report would know, a Human was responsible for braking in this situation and FAILED to do so in time. The above discussion is irrelevant and results from ignorance of the facts. From the report: ". At 1.3 seconds before impact, the self-driving system determined that an emergency braking maneuver was needed to mitigate a collision (see figure 2).[2] According to Uber, emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior. The vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator." For anyone interested in actually being informed about this incident. https://www.ntsb.gov/investigations/AccidentReports/Pages/HWY18MH010-prelim.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HW...
  - rpedela 8y ago
    There is going to be a reaction time difference between someone actively driving the vehicle and someone who is monitoring an automated system. It is human nature to get bored and not pay close enough attention. This is exactly why some of the Tesla autopilot crashes have happened in addition to the system screwing up.
    - Animats 8y ago
      Yes. I've pointed that out previously. NASA has a simple tool for studying this, the Multi-Attribute Test Battery.[1] It uses a Windows PC and a joystick. The person being tested uses the joystick to center a marker in a square. The marker tends to drift away, and has to be corrected. This simulates the most basic task of flying - flying straight and level. There's an "autopilot", which can be turned on to keep the marker centered. The user can then let go of the joystick. But sometimes the "autopilot" fails, and the marker drifts out of the square. How long it takes for the subject to notice is measured. That's a good approximation for Tesla's semi-automatic driving system. There are lots of papers on this. It's well understood in aviation that takeover time after a fault is seconds to tens of seconds. [1] https://matb.larc.nasa.gov/ https://matb.larc.nasa.gov/
      - deleted 8y ago
        [deleted]
- mark-r 8y ago
  A human could also have attempted swerving to avoid the accident. The question that I feel can't be answered is how visible the pedestrian would have been under the conditions and if there would have been time to take action. Even braking to half the recorded velocity would likely have been fatal.
  - davidgould 8y ago
    > Even braking to half the recorded velocity would likely have been fatal. This is not so. Half would have been under 25 mph which is much less than 50% fatal. See http://www.humantransport.org/sidewalks/SpeedKills.htm http://www.humantransport.org/sidewalks/SpeedKills.htm
evrydayhustling 8y ago
> There's no way of knowing if a human driver could have reacted properly in this specific scenario. What? Why? The specific scenario is recorded, the surrounding conditions are known and relatively easy to reproduce, and there are both norms and requirements about human capabilities when driving a car. If we can't attempt to determine whether a human could have responded in this case, I'm not sure where we could determine anything about any kind of accountability. People obviously make mistakes too -- I've hit a deer before, unfortunately. But when the consequences are high, we look into culpability about whether someone should have done more to prevent the mistake. Driving under the influence, without a license, etc. are all things that can make someone responsible for the big outcome of a relatively easy-to-make mistake. Accepting Uber's "could you react to this video I'm showing you?" defense is skipping all the steps that would let us assign responsibility or even learn / improve from the incident.
- mark-r 8y ago
  The comment I was responding to specifically mentioned the dash-cam video, which I hope we can all agree would not be representative of what the operator could see. Nor would any of the on-board sensors be relevant. I suppose they might provide enough information to do a re-enactment. At no point did I suggest that this would be an excuse to absolve Uber of any responsibility.
  - smilekzs 8y ago
    I disagree to a minor degree, because humans can have widely different ability of night vision.
- bronco21016 8y ago
  The miracle on the Hudson comes to mind in this case. After the event there were some wondering if the crew could have safely glided to an airport. Investigators painstakingly recreated the event in a simulator and ran several trials. The issue with the experiment? The crew in the simulator knew what was coming. They were able to make the field because they were already primed to turn for the field immediately without any time to think. I’m sure you could do a blind study of this scenario but I’m sure people participating in a driving study in a simulator are not going to be driving as they do in real life. We see it every single day in the airline industry. It’s why we’re checked and observed at random intervals while out flying on the line. The surprise aspect gives you no time to prepare or study up to be in perfect shape like you do when you’re headed into a training event.
ummonk 8y ago
Not in locations with substantial street lighting. It's usually in an unlit road that you get the danger (especially if you aren't using high beams due to an oncoming car).
jimbofisher1 8y ago
The human operator was responsible for braking in this situation and failed to do so. https://www.ntsb.gov/investigations/AccidentReports/Pages/HWY18MH010-prelim.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HW...
- petre 8y ago
  What's the point of a self/assisted driving vehicle then? If it doesn't compensate for human error, then it's useless, since humans can already operate vehicles.
- mark-r 8y ago
  That was my point, human operators are responsible for not hitting deer yet it happens every day. The only reason it doesn't happen to people more often is that people are better at avoiding running in front of cars.
  - romwell 8y ago
    >That was my point, human operators are responsible for not hitting deer How is this even comparable? A deer jumping in front of a car is not the same as a person walking the bicycle slowly across several lanes (and almost making it to the other side) on a rather well-lit road.
    - zaarn 8y ago
      Because the human is currently monitoring the car to not hit a deer, as other comments mentions, it can take dozens of seconds to regain full awareness.
      - jimbofisher1 8y ago
        Dozens of seconds? Have you ever even driven a car? lol
        zaarn 8y ago
JackCh 8y ago
A few years ago I hit a deer. Had the deer been waiting to jump into the road from on a sidewalk instead of behind a bush, I would not have hit it. The reason I hit the deer is because I didn't know the deer was there. When people hit deer, it's not because the deer was casually walking across the road like a human jaywalker. It's because the deer jumps into your path in an instant. As somebody who's hit a deer and done plenty of urban driving with no shortage of jaywalkers, I can assure you there is no fair comparison. The only sort of situation that might be comparable is if somebody is trying to commit suicide by car and deliberately leaps in front of you, or if a small child concealed by a car parked by the side of the road runs out into the road suddenly. But those scenarios are unrelated to this incident.

YeGoblynQueenne 8y ago

>> Great points regarding aligning precision/recall of AI systems with actual human supervision capabilities. May I nitpick? The article discusses False Positive Rate and False Negative Rate. These are, respectively, the complements of True Negative Rate and True Positive Rate a.k.a. Sensitivity, a.k.a. Recall. Precision is not the same as False Negative Rate. Specificity is, as in Sensitivity/Specificity. These metrics tend to be reported in pairs: TPR/TNR, FPR/FNR, Precision/Recall, Sensitivity/Specificity, which confuses the issue- but not all pairs are the same.

tcmb 8y ago
To add to that, since I see it nowhere mentioned explicitly, neither in this thread nor in the article: the theoretical framework here is Signal Detection Theory (SDT) [1]. The problem at hand can be nicely visualized with two overlapping bell curves for the signal present/absent situations [2]. [1] https://en.wikipedia.org/wiki/Detection_theory https://en.wikipedia.org/wiki/Detection_theory [2] https://jdlee888.shinyapps.io/SDT_Demo/ https://jdlee888.shinyapps.io/SDT_Demo/
- sgt101 8y ago
  Two points - first, these frameworks don't really account for driving into concrete bollards vs. a shaky ride. Second none of these frameworks estimate production performance well; time after time after time I see classifiers that did "95%" in test and "88%" in prod and "that's pretty good!". My point is that we are really bad at estimating performance of classifiers. Really bad, and mostly we are pretending it's fine.

SiempreViernes 8y ago

A better source for the example videos showing the light levels at that point is the arstechnica page being cited: https://arstechnica.com/cars/2018/03/police-chief-said-uber-victim-came-from-the-shadows-dont-believe-it/ https://arstechnica.com/cars/2018/03/police-chief-said-uber-...