12 ms·

Fatalities vs. False Positives: The Lessons from the Tesla and Uber Crashes

116 points by szczys 8y ago

evrydayhustling 8y ago
Great points regarding aligning precision/recall of AI systems with actual human supervision capabilities. One quibble: I hate seeing articles uncritically repeat Uber's line that a human driver could not have reacted in the same scenario. The dashboard cam footage they released does not accurately represent either human or car perceptual capabilities [1], and folks recreating the scene have shown that it doesn't even look like a good (or unaltered) example of what a dashcam should have perceived [2]. [1] https://ideas.4brad.com/it-certainly-looks-bad-uber https://ideas.4brad.com/it-certainly-looks-bad-uber [2] https://dgit.com/residents-recreate-crash-uber-car-case-56161/ https://dgit.com/residents-recreate-crash-uber-car-case-5616...
- mark-r 8y ago
  There's no way of knowing if a human driver could have reacted properly in this specific scenario. However if you live in an area where deer frequently cross the road, you'll know that collisions with deer occur regularly; that would indicate that it's easy to hit things that cross your path, even when you should be able to see them.
  - rpedela 8y ago
    True, but a human would have applied the brakes at least which may not avoid the collision but would reduce the injuries. We can never know but maybe a human would have been able to stop. Or maybe a human could have braked enough to prevent death. Or maybe the exact outcome would have happened even if the brakes were applied.
    - jimbofisher1 8y ago
      In fact, as anyone who has read the NTSB report would know, a Human was responsible for braking in this situation and FAILED to do so in time. The above discussion is irrelevant and results from ignorance of the facts. From the report: ". At 1.3 seconds before impact, the self-driving system determined that an emergency braking maneuver was needed to mitigate a collision (see figure 2).[2] According to Uber, emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior. The vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator." For anyone interested in actually being informed about this incident. https://www.ntsb.gov/investigations/AccidentReports/Pages/HWY18MH010-prelim.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HW...
      - rpedela 8y ago
        There is going to be a reaction time difference between someone actively driving the vehicle and someone who is monitoring an automated system. It is human nature to get bored and not pay close enough attention. This is exactly why some of the Tesla autopilot crashes have happened in addition to the system screwing up.
        Animats 8y ago
        Yes. I've pointed that out previously. NASA has a simple tool for studying this, the Multi-Attribute Test Battery.[1] It uses a Windows PC and a joystick. The person being tested uses the joystick to center a marker in a square. The marker tends to drift away, and has to be corrected. This simulates the most basic task of flying - flying straight and level. There's an "autopilot", which can be turned on to keep the marker centered. The user can then let go of the joystick. But sometimes the "autopilot" fails, and the marker drifts out of the square. How long it takes for the subject to notice is measured. That's a good approximation for Tesla's semi-automatic driving system. There are lots of papers on this. It's well understood in aviation that takeover time after a fault is seconds to tens of seconds. [1] https://matb.larc.nasa.gov/ https://matb.larc.nasa.gov/
        deleted 8y ago
        [deleted]
    - mark-r 8y ago
      A human could also have attempted swerving to avoid the accident. The question that I feel can't be answered is how visible the pedestrian would have been under the conditions and if there would have been time to take action. Even braking to half the recorded velocity would likely have been fatal.
      - davidgould 8y ago
        > Even braking to half the recorded velocity would likely have been fatal. This is not so. Half would have been under 25 mph which is much less than 50% fatal. See http://www.humantransport.org/sidewalks/SpeedKills.htm http://www.humantransport.org/sidewalks/SpeedKills.htm
  - evrydayhustling 8y ago
    > There's no way of knowing if a human driver could have reacted properly in this specific scenario. What? Why? The specific scenario is recorded, the surrounding conditions are known and relatively easy to reproduce, and there are both norms and requirements about human capabilities when driving a car. If we can't attempt to determine whether a human could have responded in this case, I'm not sure where we could determine anything about any kind of accountability. People obviously make mistakes too -- I've hit a deer before, unfortunately. But when the consequences are high, we look into culpability about whether someone should have done more to prevent the mistake. Driving under the influence, without a license, etc. are all things that can make someone responsible for the big outcome of a relatively easy-to-make mistake. Accepting Uber's "could you react to this video I'm showing you?" defense is skipping all the steps that would let us assign responsibility or even learn / improve from the incident.
    - mark-r 8y ago
      The comment I was responding to specifically mentioned the dash-cam video, which I hope we can all agree would not be representative of what the operator could see. Nor would any of the on-board sensors be relevant. I suppose they might provide enough information to do a re-enactment. At no point did I suggest that this would be an excuse to absolve Uber of any responsibility.
      - smilekzs 8y ago
        I disagree to a minor degree, because humans can have widely different ability of night vision.
    - bronco21016 8y ago
      The miracle on the Hudson comes to mind in this case. After the event there were some wondering if the crew could have safely glided to an airport. Investigators painstakingly recreated the event in a simulator and ran several trials. The issue with the experiment? The crew in the simulator knew what was coming. They were able to make the field because they were already primed to turn for the field immediately without any time to think. I’m sure you could do a blind study of this scenario but I’m sure people participating in a driving study in a simulator are not going to be driving as they do in real life. We see it every single day in the airline industry. It’s why we’re checked and observed at random intervals while out flying on the line. The surprise aspect gives you no time to prepare or study up to be in perfect shape like you do when you’re headed into a training event.
  - ummonk 8y ago
    Not in locations with substantial street lighting. It's usually in an unlit road that you get the danger (especially if you aren't using high beams due to an oncoming car).
  - jimbofisher1 8y ago
    The human operator was responsible for braking in this situation and failed to do so. https://www.ntsb.gov/investigations/AccidentReports/Pages/HWY18MH010-prelim.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HW...
    - petre 8y ago
      What's the point of a self/assisted driving vehicle then? If it doesn't compensate for human error, then it's useless, since humans can already operate vehicles.
    - mark-r 8y ago
      That was my point, human operators are responsible for not hitting deer yet it happens every day. The only reason it doesn't happen to people more often is that people are better at avoiding running in front of cars.
      - romwell 8y ago
        >That was my point, human operators are responsible for not hitting deer How is this even comparable? A deer jumping in front of a car is not the same as a person walking the bicycle slowly across several lanes (and almost making it to the other side) on a rather well-lit road.
        zaarn 8y ago
        Because the human is currently monitoring the car to not hit a deer, as other comments mentions, it can take dozens of seconds to regain full awareness.
        jimbofisher1 8y ago
        Dozens of seconds? Have you ever even driven a car? lol
        zaarn 8y ago
  - JackCh 8y ago
    A few years ago I hit a deer. Had the deer been waiting to jump into the road from on a sidewalk instead of behind a bush, I would not have hit it. The reason I hit the deer is because I didn't know the deer was there. When people hit deer, it's not because the deer was casually walking across the road like a human jaywalker. It's because the deer jumps into your path in an instant. As somebody who's hit a deer and done plenty of urban driving with no shortage of jaywalkers, I can assure you there is no fair comparison. The only sort of situation that might be comparable is if somebody is trying to commit suicide by car and deliberately leaps in front of you, or if a small child concealed by a car parked by the side of the road runs out into the road suddenly. But those scenarios are unrelated to this incident.
- YeGoblynQueenne 8y ago
  >> Great points regarding aligning precision/recall of AI systems with actual human supervision capabilities. May I nitpick? The article discusses False Positive Rate and False Negative Rate. These are, respectively, the complements of True Negative Rate and True Positive Rate a.k.a. Sensitivity, a.k.a. Recall. Precision is not the same as False Negative Rate. Specificity is, as in Sensitivity/Specificity. These metrics tend to be reported in pairs: TPR/TNR, FPR/FNR, Precision/Recall, Sensitivity/Specificity, which confuses the issue- but not all pairs are the same.
  - tcmb 8y ago
    To add to that, since I see it nowhere mentioned explicitly, neither in this thread nor in the article: the theoretical framework here is Signal Detection Theory (SDT) [1]. The problem at hand can be nicely visualized with two overlapping bell curves for the signal present/absent situations [2]. [1] https://en.wikipedia.org/wiki/Detection_theory https://en.wikipedia.org/wiki/Detection_theory [2] https://jdlee888.shinyapps.io/SDT_Demo/ https://jdlee888.shinyapps.io/SDT_Demo/
    - sgt101 8y ago
      Two points - first, these frameworks don't really account for driving into concrete bollards vs. a shaky ride. Second none of these frameworks estimate production performance well; time after time after time I see classifiers that did "95%" in test and "88%" in prod and "that's pretty good!". My point is that we are really bad at estimating performance of classifiers. Really bad, and mostly we are pretending it's fine.
- SiempreViernes 8y ago
  A better source for the example videos showing the light levels at that point is the arstechnica page being cited: https://arstechnica.com/cars/2018/03/police-chief-said-uber-victim-came-from-the-shadows-dont-believe-it/ https://arstechnica.com/cars/2018/03/police-chief-said-uber-...
deleted 8y ago
[deleted]
Semirhage 8y ago
If a company is playing fast and loose with the false negatives rate, drivers and pedestrians will die needlessly, but if they are too strict the car will be undriveable and erratic. Both Tesla and Uber, when faced with this difficult tradeoff, punted: they require a person to watch out for the false negatives, taking the burden off of the machine. Companies should not be allowed to punt this way, when lives are on the line. This really is a problem in need of a regulatory or legislative solution, as multiple companies prove they don’t have anyone’s interests at heart, but their own. Worse, every pedestrian, cyclist, and other driver on the road who didn’t sign up for this in-the-wild alpha is being drafted to enrich the likes of Tesla and Uber.
- simion314 8y ago
  Especially the companies that need to solve autopilot sooner or will fail, so they fake it till they make it.
- ckinnan 8y ago
  Lives on the line? On average, existing cars kill more than 100 people each day on American roads. The current system of human-driving cars has killed one million Americans over the past couple of generations, mostly from human error. To save lives we need to push forward on self-driving tech.
  - Semirhage 8y ago
    Ok, you go volunteer on a test track then. While you’re at it, you should volunteer for medical testing, do you know how many people die each year from various diseases?! Maybe the needed progress will come soon, or maybe not, but since your only justification is the magnitude of the present problem I’m sure that you won’t object. Gooood luck!
    - Game_Ender 8y ago
      Because sick people are desperate we actually make it quite hard for them to volunteer for things like risky medical tests. Requiring more animal or smaller pilot studies first. I don't know the numbers well enough to know whether this is a net win for humanity, but just like any technology development project, the longer your dev cycles the slower things go.
ithilglin909 8y ago
It seems like self-driving cars may not (and maybe should not) become commonplace until the roads are built for self-driving cars – that might be a better thing to focus on, instead of systems that mimic human judgment.
- sgslo 8y ago
  Seems unlikely that road infrastructure will be upgraded/changed prior to construction of self driving cars. When developing the self driving tech, a company retains all IP associated with research/development costs. Would they ever instead pay for development of public infra? Would a municipality build roads that conform to Company A's standard of self driving roads, omitting the requests of Company B?
  - ithilglin909 8y ago
    I absolutely agree that it's a tall order, but it seems at least possible that development could happen in tandem, and some kind of standards could be agreed upon for what kind of technologies would be used across the board. I'm picturing that if some new infrastructure and cars implement that, those could be roads on which self-driving cars would be permitted to drive, and a shift would happen gradually.
    - danpat 8y ago
      Volvo proposed embedding small permanent magnets in the road surface that would allow vehicles to track more reliably. Their press-release for it is here, and has a couple of minor technical details: https://www.media.volvocars.com/global/en-gb/media/pressreleases/140760/volvo-car-group-tests-road-magnets-for-accurate-positioning-of-self-driving-cars https://www.media.volvocars.com/global/en-gb/media/pressrele... I kind of liked the idea - it's basically a higher-tech line-following robot concept, and doesn't look too expensive to roll out compare to other proposed infrastructure changes. However, I haven't heard anything about it since the 2014 press frenzy over the release linked above. Maybe getting road maintainers to make changes like this really is too much to ask.
Fricken 8y ago
A video was released today showing what the classifier in Tesla's Autopilot sees, provided by an individual who hacked it. False positives galore: https://youtu.be/Yyak-U2vPxM https://youtu.be/Yyak-U2vPxM
- andor 8y ago
  You should read the description: "These are my old snapshots with radar data overlaid." It's just the radar data, not the camera based system. Further: "Sometimes the circle is in place, where there's nothing present. It should be well known, that radar have problems with determining the elevation at which the object is located, so such readout can be caused by an object that is higher or lower on the image." and "Radar reports coordinates in 3D space and so the conversion to image space might not be super accurate at times"
- mephitix 8y ago
  Dunno about ‘galore’. The color of the circles seems to correlate with the confidence.
  - Isinlor 8y ago
    It's stationary (orange) vs. moving (green). Tesla ignores stationary objects detected by radar and the video makes it painfully obvious why.
- chillingeffect 8y ago
  this is great. It would be fun to turn off the source video and imagine what it would be like to drive with only what the algorithm detects!
simion314 8y ago
I can still see the people commenting that the Tesla driver did not had his hands on the wheel and he was warned when this warring happened 15 minutes before the accident, so the Tesla PR blog blaming the driver had results proving again how hard it is to undo a lie/almost lie.
- Animats 8y ago
  Also, the people still commenting that the Uber video is meaningful. That's not from the vehicle sensors. That's from a completely separate dashcam and recorder.
sgslo 8y ago
The upside of self driving vehicles is so immense that I can't help but find in favor of giving leeway to companies developing this tech. I would vote legislation limiting the accident liability of 'qualified' companies developing self driving tech. No, I don't know what separates 'qualified' from 'not qualified'. Try looking at this a different way: there were 37,461 vehicle related fatalities in 2016 in the US alone. In the current risk-averse climate, would we - over the early 1900's - ever allowed the development of public infrastructure to support an invention leading to so many fatalities? Likely not, but the benefit we enjoy from motorized transportation far outweighs the cost of those 37k yearly lives. The point is that continuing with a risk-averse, liability-obsessed development culture will stifle invention that can otherwise lead to great quality of life improvements.
- s2g 8y ago
  Or we could not let companies murder people with shitty half baked hardware and software combinations.
- dbt00 8y ago
  The theorized upside is immense but that doesn't mean the practical difference now is even positive. If Tesla (Uber's already out of the game, probably) wants to keep operating autopilot in public, they owe us actual statistics on safety so that the public can make an informed decision. The private data laundered through the NTSB two years ago is incredibly suspect, and the fatality rate for Tesla since then appears to exceed what the average human piloted car would do.
- Semirhage 8y ago
  Then you and other people like you can volunteer to work at a test track as obstacles. Leave the rest of us out of it. You’re entitled to your utopian fantasy of what might be, and the best way to express that is to put your life on the line instead of volunteering others to do it. Statements like this seem to be trying to make the point that any amount of increased public risk is unacceptable unless everyone opts in to accept it. No, statements like mine reject the notion that sacrificing lives today for the possibility of an unproven improvement sat some unspecified later date is morally reprehensible. My answer is to invite the person endorsing this to put their own life on the line instead. If we were talking about medical testing on humans instead of cars, maybe it would be easier for some to understand. “It’s so bad today, the only course of action is to act recklessly to improve things as fast as possible!” is a thoroughly rejected argument.
tzakrajs 8y ago
Good thing L5 self-driving can be done without low-latency LIDAR and that Tesla didn't defraud many Model S owners by selling them cars fundamentally incapable of providing L5 self-driving. /s
- deleted 8y ago
  [deleted]
niftich 8y ago
We're asking a lot from this kind of software (for good reasons), but humans commit hundreds of leaps of faith of various severity on the roads daily -- failure to yield, failure to maintain a safe following distance, assuming other drivers immediately adjacent to you, like in lanes that are significantly slower than yours, will keep driving safely and carefully. Urban, peak-hour traffic on most US freeways is an exercise in collective insanity, riding people's bumpers at 55+ mph (and often significantly higher), leaving little room to stop for incidents [1][2][3][4] or debris. But only a small subset of these situations result in significant accidents, because unimpaired humans, largely, have some intuition for self-preservation. On the other hand, we're expecting an algorithm coded by humans to perform better than a complicated bioelectric system we barely understand. Waymo's self-driving program has opted to thoroughly understanding its environment, which is why their cars drive in a manner that bears no resemblance to how humans actually drive. We as a society have to eventually reconcile the implications of the disconnect. [1] Unsafe lane change in traffic with different lane speeds: https://gfycat.com/CleanGleefulArawana https://gfycat.com/CleanGleefulArawana [2] Tailgating causes crash, swerve, multi-vehicle accident: https://www.youtube.com/watch?v=j0rj2sZ1KA4 https://www.youtube.com/watch?v=j0rj2sZ1KA4 [3] Inattention to incident causes further accidents: https://www.youtube.com/watch?v=hZL6OKwQGew https://www.youtube.com/watch?v=hZL6OKwQGew [4] Inattention in slowing traffic causes accident: https://www.youtube.com/watch?v=Ff7wbSwTuEk https://www.youtube.com/watch?v=Ff7wbSwTuEk
- ghostbrainalpha 8y ago
  Are you saying that even if AI can't react as well as humans, the roads would potentially be safer with more AI drivers because of the cumulative effect of all the little safety rules they follow that human drivers do not?
  - bsder 8y ago
    Yes. Computers won't get distracted. Computers won't accidentally mix up medications and get foggy. Computers won't get tired. Computers won't drive drunk. I can create a self-driving car better than 50% of Southern Californians right now: "<beep> High humidity detected. I'm sorry, I can't let you drive today, Dave. <beep>" People suck at driving in new places--take a plane flight to a new city, rent a car, and have a passenger record how many illegal things you do--you probably won't make it out of the airport before the first infraction. People are good at driving in the familiar because they memorize it. A new stop sign at a previously uncontrolled intersection causes chaos for a while. Computers will also memorize the local environment at some point. Remember how bad Google Maps was when they started? This will be the same.
ska 8y ago
This set of trade-offs in self driving cars is exactly the same one that CAD (computer aided detection/diagnosis) systems have been making in medicine for decades. In many cases both type I and type II errors will (with statistical certainty) kill people over enough iterations. It's not an easy problem, but the best you can do is demonstrate that the system improves on the standard of care. In other words, overall the trade-off has better outcomes than not using the system. The same will be true for self driving cars if/when they reach mainstream use. The important thing is the average performance is improved significantly. It's worth noting that, like the CAD systems, focusing too much on fixing an individual error can cause a degradation to the overall system. As these things get better and better, you'll have to be much more careful about applying non-obvious fixes, or risk a much worse outcome that doing nothing at all.
- rayiner 8y ago
  Imagine a CAD system that was so fundamentally unreliable that it ignored positive hits because the high rate of false positives otherwise would make the system useless. Instead, the CAD system came with a warning printed on the instruction manual that a doctor had to review every scan anyway.
  - bsder 8y ago
    Welcome to breast cancer screening. The current systems are so good at detecting tiny anomalies that probably don't matter that their usage has had to be tuned down.
    - deleted 8y ago
      [deleted]
    - ska 8y ago
      You have to be very careful with false positive rates with something like breast screening, as the work up rate has definite negative consequences. Radiologists have too high a false negative rate on such screens, so you're trying to achieve a balance. To be really blunt about it, you are trading off complications/deaths from missed cancers against complications/deaths from biopsies, but these categories occur at very different rates.
hartator 8y ago
> In contrast to the Tesla accident, where the human driver could have saved himself from the car’s blindness, the Uber car probably could have braked in time to prevent the accident entirely where a human driver couldn’t have. That's a strong statement. Watching the actual video [1] of the Uber accident, it seems both an attentive driver (she seems to be not looking at the road before the crash) or a better decision algorithm, could have prevented the crash. Keep also in mind, videos are very bad at viewing low light shots, and the scene must have been significantly clearer to both the driver and the computer when you're actually there. [1] https://www.youtube.com/watch?v=pO9iRUx5wmM https://www.youtube.com/watch?v=pO9iRUx5wmM
- tomp 8y ago
  Why would it be clearer to a compuer? Unless thhe car has some extra sensors (radar, laser), surely it cannot see better than the camera?
  - agency 8y ago
    Uber's self driving cars have both LIDAR and Radar in addition to cameras. https://www.gearbrain.com/autonomous-uber-car-technology-2551793303.html https://www.gearbrain.com/autonomous-uber-car-technology-255...
  - sp332 8y ago
    The camera is just a dashcam and is not part of the self-driving system.
- jimbofisher1 8y ago
  Read the actual report. The Uber system registered that emergency braking was necessary to avoid a collision but the brakes were not under the control of that system. https://www.ntsb.gov/investigations/AccidentReports/Pages/HWY18MH010-prelim.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HW... Relevant section: "At 1.3 seconds before impact, the self-driving system determined that an emergency braking maneuver was needed to mitigate a collision (see figure 2).[2] According to Uber, emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior. The vehicle operator is relied on to intervene and take action. "
rhacker 8y ago
Turning one system off in favor of another is basically admitting that they have no faith in their software. Also isn't driving a car with sensors (without self-driving tech) basically a giant data collector? The mere act of driving a car (with a human brain) should help a SDV system understand a false-positive and a false negative simply by learning what a human does in contrast to what the machine thinks it is supposed to do. I imagine this is what separates Waymo and the others. I feel like Waymo is using math/neural nets, etc.. whereas Tesla and Uber were probably giant hand-written if/else machines. I have nothing to back that up, except that that's kinda what this article is about.
- hacker11235 8y ago
  Speaking for Tesla and Uber, they use neural nets. Tesla has the most data which is what you're referring to above and they do exactly what you were stating. The software is/has always been shadowing human drivers, that is learning and comparing to the decisions it would make on its own. Uber is also most definitely using a similar set up. I think neural nets are a basic necessity in this field since there are an almost infinite number of scenarios.
rasz 8y ago
This certainly explains reports of almost every Google car accident being the case of human rear-ending it. Google did it right from the start, didnt gamble with false positives.
Shivetya 8y ago
I will hold to my opinion stated before, these systems are not safe enough to be on public roads as of yet let alone in the hands of consumers. that out I still see no reason why we don't adapt limited access roadways to support self driving in that realm. a large number of cities have dedicate HOV and (express) toll lanes that can all be adopted and marked much easier to support self driving capability in a controlled environment. plus its probably a good chance than both manufacturers of cars as well as AD systems along with drivers themselves would be willing to pay more for such access. I just recall the imagery from the old William Shatner series Tekwar that showed a similar approach, get on highway and car takes you into the city. It could even then be extended to service event centers so that it continue driving for you till parked; think off expressway to airports and stadiums
fcolas 8y ago
Nice article; thanks. * * * . Tesla claims: A 40% crash rate reduction with the autopilot as compared to no autopilot, over an 18 month period [1]. . If this is true—and we could imagine it is (at least partially?)—then Elon's remark to journalists would make sense: "It's [..] irresponsible [..] to write an article that [..] lead people to believe that autonomy is less safe,” [..] “Because people might actually turn it off, and then die" [1] * * * But to have an opinion about the autopilot's risk statistics I would also need to know: a) What populations (data) they compare; b) How each population is defined (inclusion and exclusion criteria); c) What's the sample size (18 months, and?); d) Who makes these calculations (to clearly identify possible conflicts of interests). - Not sure if this type of data is publicly available? * * * Actually the National Highway Traffic Safety Administration (NHTSA) seems to indicate[1]: 1) that the data comes from Tesla—cf. point d) => conflict of interest; 2) Autopilot on/off was NOT used for the risk statistics—although it's central (point a); 3) instead the "40%" would measure the "number of airbag deployments per million miles" which is a proxy-metric that's not directly related to car accidents. - Hey, this is odd (it's definitively not a Science or Nature method protocol). * * * . "The Insurance Institute for Highway Safety suggests: A "13%" reduction in collision claim frequency, indicating sedans with Autopilot enabled got into fewer crashes that resulted in collision claims to insurers." However it's a small difference and there're possible confounders like social status (a "Tesla driver"), gender, geographical area, and usually the confounders have a large influence on experiments, so it's unlikely that this (small) 13% difference would remain if we adjust for confounders... * * * Here's the article AARIAN MARSHALL on Wired. [1] https://www.wired.com/story/tesla-autopilot-safety-statistics/ https://www.wired.com/story/tesla-autopilot-safety-statistic...
dsfyu404ed 8y ago
>It may seem cold to couch such life-and-death decisions in terms of pure statistics, but the fact is that there is an unavoidable design tradeoff between... The fact that we as a society can't have an adult discussion about that topic is a large part of why it's so hard to strike the right balance.
madrox 8y ago
I wrote about this recently as well. This is actually going to be more of a concern as we incorporate models into everyday products. Stakeholders and QA need a solid understanding of Type I and Type II error so they can assess how much risk they're willing to take on and make that part of the quality process. Shameless plug: https://blog.d8a.me/the-qa-of-stochastic-processes-a15a9406519c https://blog.d8a.me/the-qa-of-stochastic-processes-a15a94065...
bsaul 8y ago
I think this is one of the first time in my life i see marketing (calling something « autopilot » which it is not) and overhype (we have to be the first to have self driving cars, now !) actually kill people and not just rip people or investor off. I think history will judge those companies very severely, and i’m wondering if justice isn’t going to be just as severe right now.
- taneq 8y ago
  Cigarettes were advertised as good for your health, for decades.
LinuxBender 8y ago
Could any of the fatalities be related to the malicious code commits? [1] [1] - https://www.fastcompany.com/40586864/read-elon-musks-email-alleging-there-is-a-saboteur-at-tesla https://www.fastcompany.com/40586864/read-elon-musks-email-a...

116 points by szczys 8y ago

Great points regarding aligning precision/recall of AI systems with actual human supervision capabilities. One quibble: I hate seeing articles uncritically repeat Uber's line that a human driver could not have reacted in the same scenario. The dashboard cam footage they released does not accurately represent either human or car perceptual capabilities [1], and folks recreating the scene have shown that it doesn't even look like a good (or unaltered) example of what a dashcam should have perceived [2]. [1] https://ideas.4brad.com/it-certainly-looks-bad-uber https://ideas.4brad.com/it-certainly-looks-bad-uber [2] https://dgit.com/residents-recreate-crash-uber-car-case-56161/ https://dgit.com/residents-recreate-crash-uber-car-case-5616...

mark-r 8y ago
There's no way of knowing if a human driver could have reacted properly in this specific scenario. However if you live in an area where deer frequently cross the road, you'll know that collisions with deer occur regularly; that would indicate that it's easy to hit things that cross your path, even when you should be able to see them.
- rpedela 8y ago
  True, but a human would have applied the brakes at least which may not avoid the collision but would reduce the injuries. We can never know but maybe a human would have been able to stop. Or maybe a human could have braked enough to prevent death. Or maybe the exact outcome would have happened even if the brakes were applied.
  - jimbofisher1 8y ago
    In fact, as anyone who has read the NTSB report would know, a Human was responsible for braking in this situation and FAILED to do so in time. The above discussion is irrelevant and results from ignorance of the facts. From the report: ". At 1.3 seconds before impact, the self-driving system determined that an emergency braking maneuver was needed to mitigate a collision (see figure 2).[2] According to Uber, emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior. The vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator." For anyone interested in actually being informed about this incident. https://www.ntsb.gov/investigations/AccidentReports/Pages/HWY18MH010-prelim.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HW...
    - rpedela 8y ago
      There is going to be a reaction time difference between someone actively driving the vehicle and someone who is monitoring an automated system. It is human nature to get bored and not pay close enough attention. This is exactly why some of the Tesla autopilot crashes have happened in addition to the system screwing up.
      - Animats 8y ago
        Yes. I've pointed that out previously. NASA has a simple tool for studying this, the Multi-Attribute Test Battery.[1] It uses a Windows PC and a joystick. The person being tested uses the joystick to center a marker in a square. The marker tends to drift away, and has to be corrected. This simulates the most basic task of flying - flying straight and level. There's an "autopilot", which can be turned on to keep the marker centered. The user can then let go of the joystick. But sometimes the "autopilot" fails, and the marker drifts out of the square. How long it takes for the subject to notice is measured. That's a good approximation for Tesla's semi-automatic driving system. There are lots of papers on this. It's well understood in aviation that takeover time after a fault is seconds to tens of seconds. [1] https://matb.larc.nasa.gov/ https://matb.larc.nasa.gov/
        deleted 8y ago
        [deleted]
  - mark-r 8y ago
    A human could also have attempted swerving to avoid the accident. The question that I feel can't be answered is how visible the pedestrian would have been under the conditions and if there would have been time to take action. Even braking to half the recorded velocity would likely have been fatal.
    - davidgould 8y ago
      > Even braking to half the recorded velocity would likely have been fatal. This is not so. Half would have been under 25 mph which is much less than 50% fatal. See http://www.humantransport.org/sidewalks/SpeedKills.htm http://www.humantransport.org/sidewalks/SpeedKills.htm
- evrydayhustling 8y ago
  > There's no way of knowing if a human driver could have reacted properly in this specific scenario. What? Why? The specific scenario is recorded, the surrounding conditions are known and relatively easy to reproduce, and there are both norms and requirements about human capabilities when driving a car. If we can't attempt to determine whether a human could have responded in this case, I'm not sure where we could determine anything about any kind of accountability. People obviously make mistakes too -- I've hit a deer before, unfortunately. But when the consequences are high, we look into culpability about whether someone should have done more to prevent the mistake. Driving under the influence, without a license, etc. are all things that can make someone responsible for the big outcome of a relatively easy-to-make mistake. Accepting Uber's "could you react to this video I'm showing you?" defense is skipping all the steps that would let us assign responsibility or even learn / improve from the incident.
  - mark-r 8y ago
    The comment I was responding to specifically mentioned the dash-cam video, which I hope we can all agree would not be representative of what the operator could see. Nor would any of the on-board sensors be relevant. I suppose they might provide enough information to do a re-enactment. At no point did I suggest that this would be an excuse to absolve Uber of any responsibility.
    - smilekzs 8y ago
      I disagree to a minor degree, because humans can have widely different ability of night vision.
  - bronco21016 8y ago
    The miracle on the Hudson comes to mind in this case. After the event there were some wondering if the crew could have safely glided to an airport. Investigators painstakingly recreated the event in a simulator and ran several trials. The issue with the experiment? The crew in the simulator knew what was coming. They were able to make the field because they were already primed to turn for the field immediately without any time to think. I’m sure you could do a blind study of this scenario but I’m sure people participating in a driving study in a simulator are not going to be driving as they do in real life. We see it every single day in the airline industry. It’s why we’re checked and observed at random intervals while out flying on the line. The surprise aspect gives you no time to prepare or study up to be in perfect shape like you do when you’re headed into a training event.
- ummonk 8y ago
  Not in locations with substantial street lighting. It's usually in an unlit road that you get the danger (especially if you aren't using high beams due to an oncoming car).
- jimbofisher1 8y ago
  The human operator was responsible for braking in this situation and failed to do so. https://www.ntsb.gov/investigations/AccidentReports/Pages/HWY18MH010-prelim.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HW...
  - petre 8y ago
    What's the point of a self/assisted driving vehicle then? If it doesn't compensate for human error, then it's useless, since humans can already operate vehicles.
  - mark-r 8y ago
    That was my point, human operators are responsible for not hitting deer yet it happens every day. The only reason it doesn't happen to people more often is that people are better at avoiding running in front of cars.
    - romwell 8y ago
      >That was my point, human operators are responsible for not hitting deer How is this even comparable? A deer jumping in front of a car is not the same as a person walking the bicycle slowly across several lanes (and almost making it to the other side) on a rather well-lit road.
      - zaarn 8y ago
        Because the human is currently monitoring the car to not hit a deer, as other comments mentions, it can take dozens of seconds to regain full awareness.
        jimbofisher1 8y ago
        Dozens of seconds? Have you ever even driven a car? lol
        zaarn 8y ago
- JackCh 8y ago
  A few years ago I hit a deer. Had the deer been waiting to jump into the road from on a sidewalk instead of behind a bush, I would not have hit it. The reason I hit the deer is because I didn't know the deer was there. When people hit deer, it's not because the deer was casually walking across the road like a human jaywalker. It's because the deer jumps into your path in an instant. As somebody who's hit a deer and done plenty of urban driving with no shortage of jaywalkers, I can assure you there is no fair comparison. The only sort of situation that might be comparable is if somebody is trying to commit suicide by car and deliberately leaps in front of you, or if a small child concealed by a car parked by the side of the road runs out into the road suddenly. But those scenarios are unrelated to this incident.
YeGoblynQueenne 8y ago
>> Great points regarding aligning precision/recall of AI systems with actual human supervision capabilities. May I nitpick? The article discusses False Positive Rate and False Negative Rate. These are, respectively, the complements of True Negative Rate and True Positive Rate a.k.a. Sensitivity, a.k.a. Recall. Precision is not the same as False Negative Rate. Specificity is, as in Sensitivity/Specificity. These metrics tend to be reported in pairs: TPR/TNR, FPR/FNR, Precision/Recall, Sensitivity/Specificity, which confuses the issue- but not all pairs are the same.
- tcmb 8y ago
  To add to that, since I see it nowhere mentioned explicitly, neither in this thread nor in the article: the theoretical framework here is Signal Detection Theory (SDT) [1]. The problem at hand can be nicely visualized with two overlapping bell curves for the signal present/absent situations [2]. [1] https://en.wikipedia.org/wiki/Detection_theory https://en.wikipedia.org/wiki/Detection_theory [2] https://jdlee888.shinyapps.io/SDT_Demo/ https://jdlee888.shinyapps.io/SDT_Demo/
  - sgt101 8y ago
    Two points - first, these frameworks don't really account for driving into concrete bollards vs. a shaky ride. Second none of these frameworks estimate production performance well; time after time after time I see classifiers that did "95%" in test and "88%" in prod and "that's pretty good!". My point is that we are really bad at estimating performance of classifiers. Really bad, and mostly we are pretending it's fine.
SiempreViernes 8y ago
A better source for the example videos showing the light levels at that point is the arstechnica page being cited: https://arstechnica.com/cars/2018/03/police-chief-said-uber-victim-came-from-the-shadows-dont-believe-it/ https://arstechnica.com/cars/2018/03/police-chief-said-uber-...

deleted 8y ago

[deleted]

Semirhage 8y ago

If a company is playing fast and loose with the false negatives rate, drivers and pedestrians will die needlessly, but if they are too strict the car will be undriveable and erratic. Both Tesla and Uber, when faced with this difficult tradeoff, punted: they require a person to watch out for the false negatives, taking the burden off of the machine. Companies should not be allowed to punt this way, when lives are on the line. This really is a problem in need of a regulatory or legislative solution, as multiple companies prove they don’t have anyone’s interests at heart, but their own. Worse, every pedestrian, cyclist, and other driver on the road who didn’t sign up for this in-the-wild alpha is being drafted to enrich the likes of Tesla and Uber.

simion314 8y ago
Especially the companies that need to solve autopilot sooner or will fail, so they fake it till they make it.
ckinnan 8y ago
Lives on the line? On average, existing cars kill more than 100 people each day on American roads. The current system of human-driving cars has killed one million Americans over the past couple of generations, mostly from human error. To save lives we need to push forward on self-driving tech.
- Semirhage 8y ago
  Ok, you go volunteer on a test track then. While you’re at it, you should volunteer for medical testing, do you know how many people die each year from various diseases?! Maybe the needed progress will come soon, or maybe not, but since your only justification is the magnitude of the present problem I’m sure that you won’t object. Gooood luck!
  - Game_Ender 8y ago
    Because sick people are desperate we actually make it quite hard for them to volunteer for things like risky medical tests. Requiring more animal or smaller pilot studies first. I don't know the numbers well enough to know whether this is a net win for humanity, but just like any technology development project, the longer your dev cycles the slower things go.

ithilglin909 8y ago

It seems like self-driving cars may not (and maybe should not) become commonplace until the roads are built for self-driving cars – that might be a better thing to focus on, instead of systems that mimic human judgment.

sgslo 8y ago
Seems unlikely that road infrastructure will be upgraded/changed prior to construction of self driving cars. When developing the self driving tech, a company retains all IP associated with research/development costs. Would they ever instead pay for development of public infra? Would a municipality build roads that conform to Company A's standard of self driving roads, omitting the requests of Company B?
- ithilglin909 8y ago
  I absolutely agree that it's a tall order, but it seems at least possible that development could happen in tandem, and some kind of standards could be agreed upon for what kind of technologies would be used across the board. I'm picturing that if some new infrastructure and cars implement that, those could be roads on which self-driving cars would be permitted to drive, and a shift would happen gradually.
  - danpat 8y ago
    Volvo proposed embedding small permanent magnets in the road surface that would allow vehicles to track more reliably. Their press-release for it is here, and has a couple of minor technical details: https://www.media.volvocars.com/global/en-gb/media/pressreleases/140760/volvo-car-group-tests-road-magnets-for-accurate-positioning-of-self-driving-cars https://www.media.volvocars.com/global/en-gb/media/pressrele... I kind of liked the idea - it's basically a higher-tech line-following robot concept, and doesn't look too expensive to roll out compare to other proposed infrastructure changes. However, I haven't heard anything about it since the 2014 press frenzy over the release linked above. Maybe getting road maintainers to make changes like this really is too much to ask.

Fricken 8y ago

A video was released today showing what the classifier in Tesla's Autopilot sees, provided by an individual who hacked it. False positives galore: https://youtu.be/Yyak-U2vPxM https://youtu.be/Yyak-U2vPxM

andor 8y ago
You should read the description: "These are my old snapshots with radar data overlaid." It's just the radar data, not the camera based system. Further: "Sometimes the circle is in place, where there's nothing present. It should be well known, that radar have problems with determining the elevation at which the object is located, so such readout can be caused by an object that is higher or lower on the image." and "Radar reports coordinates in 3D space and so the conversion to image space might not be super accurate at times"
mephitix 8y ago
Dunno about ‘galore’. The color of the circles seems to correlate with the confidence.
- Isinlor 8y ago
  It's stationary (orange) vs. moving (green). Tesla ignores stationary objects detected by radar and the video makes it painfully obvious why.
chillingeffect 8y ago
this is great. It would be fun to turn off the source video and imagine what it would be like to drive with only what the algorithm detects!

simion314 8y ago

I can still see the people commenting that the Tesla driver did not had his hands on the wheel and he was warned when this warring happened 15 minutes before the accident, so the Tesla PR blog blaming the driver had results proving again how hard it is to undo a lie/almost lie.

Animats 8y ago
Also, the people still commenting that the Uber video is meaningful. That's not from the vehicle sensors. That's from a completely separate dashcam and recorder.

sgslo 8y ago

The upside of self driving vehicles is so immense that I can't help but find in favor of giving leeway to companies developing this tech. I would vote legislation limiting the accident liability of 'qualified' companies developing self driving tech. No, I don't know what separates 'qualified' from 'not qualified'. Try looking at this a different way: there were 37,461 vehicle related fatalities in 2016 in the US alone. In the current risk-averse climate, would we - over the early 1900's - ever allowed the development of public infrastructure to support an invention leading to so many fatalities? Likely not, but the benefit we enjoy from motorized transportation far outweighs the cost of those 37k yearly lives. The point is that continuing with a risk-averse, liability-obsessed development culture will stifle invention that can otherwise lead to great quality of life improvements.

s2g 8y ago
Or we could not let companies murder people with shitty half baked hardware and software combinations.
dbt00 8y ago
The theorized upside is immense but that doesn't mean the practical difference now is even positive. If Tesla (Uber's already out of the game, probably) wants to keep operating autopilot in public, they owe us actual statistics on safety so that the public can make an informed decision. The private data laundered through the NTSB two years ago is incredibly suspect, and the fatality rate for Tesla since then appears to exceed what the average human piloted car would do.
Semirhage 8y ago
Then you and other people like you can volunteer to work at a test track as obstacles. Leave the rest of us out of it. You’re entitled to your utopian fantasy of what might be, and the best way to express that is to put your life on the line instead of volunteering others to do it. Statements like this seem to be trying to make the point that any amount of increased public risk is unacceptable unless everyone opts in to accept it. No, statements like mine reject the notion that sacrificing lives today for the possibility of an unproven improvement sat some unspecified later date is morally reprehensible. My answer is to invite the person endorsing this to put their own life on the line instead. If we were talking about medical testing on humans instead of cars, maybe it would be easier for some to understand. “It’s so bad today, the only course of action is to act recklessly to improve things as fast as possible!” is a thoroughly rejected argument.

tzakrajs 8y ago

Good thing L5 self-driving can be done without low-latency LIDAR and that Tesla didn't defraud many Model S owners by selling them cars fundamentally incapable of providing L5 self-driving. /s

deleted 8y ago
[deleted]

niftich 8y ago

We're asking a lot from this kind of software (for good reasons), but humans commit hundreds of leaps of faith of various severity on the roads daily -- failure to yield, failure to maintain a safe following distance, assuming other drivers immediately adjacent to you, like in lanes that are significantly slower than yours, will keep driving safely and carefully. Urban, peak-hour traffic on most US freeways is an exercise in collective insanity, riding people's bumpers at 55+ mph (and often significantly higher), leaving little room to stop for incidents [1][2][3][4] or debris. But only a small subset of these situations result in significant accidents, because unimpaired humans, largely, have some intuition for self-preservation. On the other hand, we're expecting an algorithm coded by humans to perform better than a complicated bioelectric system we barely understand. Waymo's self-driving program has opted to thoroughly understanding its environment, which is why their cars drive in a manner that bears no resemblance to how humans actually drive. We as a society have to eventually reconcile the implications of the disconnect. [1] Unsafe lane change in traffic with different lane speeds: https://gfycat.com/CleanGleefulArawana https://gfycat.com/CleanGleefulArawana [2] Tailgating causes crash, swerve, multi-vehicle accident: https://www.youtube.com/watch?v=j0rj2sZ1KA4 https://www.youtube.com/watch?v=j0rj2sZ1KA4 [3] Inattention to incident causes further accidents: https://www.youtube.com/watch?v=hZL6OKwQGew https://www.youtube.com/watch?v=hZL6OKwQGew [4] Inattention in slowing traffic causes accident: https://www.youtube.com/watch?v=Ff7wbSwTuEk https://www.youtube.com/watch?v=Ff7wbSwTuEk

ghostbrainalpha 8y ago
Are you saying that even if AI can't react as well as humans, the roads would potentially be safer with more AI drivers because of the cumulative effect of all the little safety rules they follow that human drivers do not?
- bsder 8y ago
  Yes. Computers won't get distracted. Computers won't accidentally mix up medications and get foggy. Computers won't get tired. Computers won't drive drunk. I can create a self-driving car better than 50% of Southern Californians right now: "<beep> High humidity detected. I'm sorry, I can't let you drive today, Dave. <beep>" People suck at driving in new places--take a plane flight to a new city, rent a car, and have a passenger record how many illegal things you do--you probably won't make it out of the airport before the first infraction. People are good at driving in the familiar because they memorize it. A new stop sign at a previously uncontrolled intersection causes chaos for a while. Computers will also memorize the local environment at some point. Remember how bad Google Maps was when they started? This will be the same.

ska 8y ago

This set of trade-offs in self driving cars is exactly the same one that CAD (computer aided detection/diagnosis) systems have been making in medicine for decades. In many cases both type I and type II errors will (with statistical certainty) kill people over enough iterations. It's not an easy problem, but the best you can do is demonstrate that the system improves on the standard of care. In other words, overall the trade-off has better outcomes than not using the system. The same will be true for self driving cars if/when they reach mainstream use. The important thing is the average performance is improved significantly. It's worth noting that, like the CAD systems, focusing too much on fixing an individual error can cause a degradation to the overall system. As these things get better and better, you'll have to be much more careful about applying non-obvious fixes, or risk a much worse outcome that doing nothing at all.

rayiner 8y ago
Imagine a CAD system that was so fundamentally unreliable that it ignored positive hits because the high rate of false positives otherwise would make the system useless. Instead, the CAD system came with a warning printed on the instruction manual that a doctor had to review every scan anyway.
- bsder 8y ago
  Welcome to breast cancer screening. The current systems are so good at detecting tiny anomalies that probably don't matter that their usage has had to be tuned down.
  - deleted 8y ago
    [deleted]
  - ska 8y ago
    You have to be very careful with false positive rates with something like breast screening, as the work up rate has definite negative consequences. Radiologists have too high a false negative rate on such screens, so you're trying to achieve a balance. To be really blunt about it, you are trading off complications/deaths from missed cancers against complications/deaths from biopsies, but these categories occur at very different rates.

hartator 8y ago

> In contrast to the Tesla accident, where the human driver could have saved himself from the car’s blindness, the Uber car probably could have braked in time to prevent the accident entirely where a human driver couldn’t have. That's a strong statement. Watching the actual video [1] of the Uber accident, it seems both an attentive driver (she seems to be not looking at the road before the crash) or a better decision algorithm, could have prevented the crash. Keep also in mind, videos are very bad at viewing low light shots, and the scene must have been significantly clearer to both the driver and the computer when you're actually there. [1] https://www.youtube.com/watch?v=pO9iRUx5wmM https://www.youtube.com/watch?v=pO9iRUx5wmM

tomp 8y ago
Why would it be clearer to a compuer? Unless thhe car has some extra sensors (radar, laser), surely it cannot see better than the camera?
- agency 8y ago
  Uber's self driving cars have both LIDAR and Radar in addition to cameras. https://www.gearbrain.com/autonomous-uber-car-technology-2551793303.html https://www.gearbrain.com/autonomous-uber-car-technology-255...
- sp332 8y ago
  The camera is just a dashcam and is not part of the self-driving system.
jimbofisher1 8y ago
Read the actual report. The Uber system registered that emergency braking was necessary to avoid a collision but the brakes were not under the control of that system. https://www.ntsb.gov/investigations/AccidentReports/Pages/HWY18MH010-prelim.aspx https://www.ntsb.gov/investigations/AccidentReports/Pages/HW... Relevant section: "At 1.3 seconds before impact, the self-driving system determined that an emergency braking maneuver was needed to mitigate a collision (see figure 2).[2] According to Uber, emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior. The vehicle operator is relied on to intervene and take action. "

rhacker 8y ago

Turning one system off in favor of another is basically admitting that they have no faith in their software. Also isn't driving a car with sensors (without self-driving tech) basically a giant data collector? The mere act of driving a car (with a human brain) should help a SDV system understand a false-positive and a false negative simply by learning what a human does in contrast to what the machine thinks it is supposed to do. I imagine this is what separates Waymo and the others. I feel like Waymo is using math/neural nets, etc.. whereas Tesla and Uber were probably giant hand-written if/else machines. I have nothing to back that up, except that that's kinda what this article is about.

hacker11235 8y ago
Speaking for Tesla and Uber, they use neural nets. Tesla has the most data which is what you're referring to above and they do exactly what you were stating. The software is/has always been shadowing human drivers, that is learning and comparing to the decisions it would make on its own. Uber is also most definitely using a similar set up. I think neural nets are a basic necessity in this field since there are an almost infinite number of scenarios.

rasz 8y ago

This certainly explains reports of almost every Google car accident being the case of human rear-ending it. Google did it right from the start, didnt gamble with false positives.

Shivetya 8y ago

I will hold to my opinion stated before, these systems are not safe enough to be on public roads as of yet let alone in the hands of consumers. that out I still see no reason why we don't adapt limited access roadways to support self driving in that realm. a large number of cities have dedicate HOV and (express) toll lanes that can all be adopted and marked much easier to support self driving capability in a controlled environment. plus its probably a good chance than both manufacturers of cars as well as AD systems along with drivers themselves would be willing to pay more for such access. I just recall the imagery from the old William Shatner series Tekwar that showed a similar approach, get on highway and car takes you into the city. It could even then be extended to service event centers so that it continue driving for you till parked; think off expressway to airports and stadiums

fcolas 8y ago

Nice article; thanks. * * * . Tesla claims: A 40% crash rate reduction with the autopilot as compared to no autopilot, over an 18 month period [1]. . If this is true—and we could imagine it is (at least partially?)—then Elon's remark to journalists would make sense: "It's [..] irresponsible [..] to write an article that [..] lead people to believe that autonomy is less safe,” [..] “Because people might actually turn it off, and then die" [1] * * * But to have an opinion about the autopilot's risk statistics I would also need to know: a) What populations (data) they compare; b) How each population is defined (inclusion and exclusion criteria); c) What's the sample size (18 months, and?); d) Who makes these calculations (to clearly identify possible conflicts of interests). - Not sure if this type of data is publicly available? * * * Actually the National Highway Traffic Safety Administration (NHTSA) seems to indicate[1]: 1) that the data comes from Tesla—cf. point d) => conflict of interest; 2) Autopilot on/off was NOT used for the risk statistics—although it's central (point a); 3) instead the "40%" would measure the "number of airbag deployments per million miles" which is a proxy-metric that's not directly related to car accidents. - Hey, this is odd (it's definitively not a Science or Nature method protocol). * * * . "The Insurance Institute for Highway Safety suggests: A "13%" reduction in collision claim frequency, indicating sedans with Autopilot enabled got into fewer crashes that resulted in collision claims to insurers." However it's a small difference and there're possible confounders like social status (a "Tesla driver"), gender, geographical area, and usually the confounders have a large influence on experiments, so it's unlikely that this (small) 13% difference would remain if we adjust for confounders... * * * Here's the article AARIAN MARSHALL on Wired. [1] https://www.wired.com/story/tesla-autopilot-safety-statistics/ https://www.wired.com/story/tesla-autopilot-safety-statistic...

dsfyu404ed 8y ago

>It may seem cold to couch such life-and-death decisions in terms of pure statistics, but the fact is that there is an unavoidable design tradeoff between... The fact that we as a society can't have an adult discussion about that topic is a large part of why it's so hard to strike the right balance.

madrox 8y ago

I wrote about this recently as well. This is actually going to be more of a concern as we incorporate models into everyday products. Stakeholders and QA need a solid understanding of Type I and Type II error so they can assess how much risk they're willing to take on and make that part of the quality process. Shameless plug: https://blog.d8a.me/the-qa-of-stochastic-processes-a15a9406519c https://blog.d8a.me/the-qa-of-stochastic-processes-a15a94065...

bsaul 8y ago

I think this is one of the first time in my life i see marketing (calling something « autopilot » which it is not) and overhype (we have to be the first to have self driving cars, now !) actually kill people and not just rip people or investor off. I think history will judge those companies very severely, and i’m wondering if justice isn’t going to be just as severe right now.

taneq 8y ago
Cigarettes were advertised as good for your health, for decades.

LinuxBender 8y ago

Could any of the fatalities be related to the malicious code commits? [1] [1] - https://www.fastcompany.com/40586864/read-elon-musks-email-alleging-there-is-a-saboteur-at-tesla https://www.fastcompany.com/40586864/read-elon-musks-email-a...