9 ms·

Andrew Ng calls Tesla irresponsible for shipping an imperfect autopilot

51 points by impish19 10y ago

WalterSear 10y ago
An imperfect autopilot, running on extra cpu cycles from the automatic door lock processor.
archildress 10y ago
This is just the cost of progress. If every imperfect piece of technology didn't ship, we'd have no technology at all.
- riffraff 10y ago
  not all technologies are created equal. If my macbook's battery dies when it rains, it's not the same as if my car crashes into another one injuring multiple people.
  - moron4hire 10y ago
    One might make the argument that the automobile--as originally released without autopilot--is a deeply flawed device and that even this level of "flawed" autopilot is superior.
    - riffraff 10y ago
      the problem here is not that the thing is flawed, is that it's flawed in a radical way that users are not aware of, i.e. > Traffic-aware cruise control may not brake/decelerate for stationary vehicles When you drive a car, you somehow expect to crash into something if you don't pay attention. If you let your autopilot do it, crashing into stationary objects is not something you are prepared for.
      - moron4hire 10y ago
        The point I'm making is that we should not be concerned with making a perfect autopilot, but more importantly, we should not be concerned with making an autopilot that is as good as a human driver in all aspects in which human drivers are good. We should be concerned with making one that reduces injury. If that means that we have more low-speed fender benders--at the cost of zero drunk-driving accidents--I'm all for it.
    - erobbins 10y ago
      That would be a terrible argument, but ok.
- dman 10y ago
  One mishap which gets a lot of public scrutiny could set the field back by decades. When it comes to products with public safety implications its better to emulate the airline industry and their approach to safety rather than the "move fast and break things" mantra. For an example of how a few catastrophic events can destroy an entire industry / approach forever we just have to look at the nuclear industry with (chernobyl / three mile) and blimps (Hindenberg).
- nas 10y ago
  That's not much of an argument. There has to be a standard where you consider the new and imperfect technology "good enough". Is Telsa's auto-pilot good enough? I don't know. My perception is that since Telsa is an underdog of the auto industry, they are more willing to take a risk and put the technology out there. If it turns out to be too unsafe, their company will take a huge hit. If not, they gain market share.
- fucking_tragedy 10y ago
  If your wife or child was injured because of this, would that still be just the cost of progress? Technology can be refined and rigorously tested before it's pushed to consumers and that is still progress.
shawndumas 10y ago
he should try driving a model T...
83457 10y ago
Anyone have public link to video?
- tyrust 10y ago
  https://www.youtube.com/watch?v=qQkx-4pFjus https://www.youtube.com/watch?v=qQkx-4pFjus
- deleted 10y ago
  [deleted]
moron4hire 10y ago
Because the regular pilots are so good at the job?
- coralreef 10y ago
  Tesla isn't responsible for how a driver drives their car. But they are responsible for the software they ship.
  - moron4hire 10y ago
    If Tesla can ship a not-perfect autopilot that could significantly reduce injury over not using it at all, shouldn't we abandon this pursuit of "perfection" and release early?
    - dilemma 10y ago
      Tesla's non-perfect "autopilot" seems to be causing accidents a human wouldn't.
      - moron4hire 10y ago
        Have you been on the roads much?
        dilemma 10y ago
        Im referring to Teslas hitting stationary objects. One such story was posted here a week ago.
        moron4hire 10y ago
        People also hit stationary objects.
drivingmenuts 10y ago
Well, seeing how Tesla just handed every auto insurance company in the world a free get-out-of-paying card for accidents dealing with that car, I'd say Tesla screwed up. Seeing as how Tesla has potentially exposed everyone involved in the supply chain to lawsuits, then yes, they screwed up. If I were a well-backed lawyer, right about now, I'd be girding up to file requests for colonoscopies on the people who assembled the circuit boards in those cars. Seriously, right about now, some lawyer is figuring out how to suck money out of Tesla until it's a dry, twitching husk in the noonday say. Tesla handed that guy a gift.
xpda 10y ago
There will be better autopilots, but never perfect ones. Aircraft have been using imperfect autopilots since their inception. Most planes have a prominent red button you can press to disable the autopilot whenever it misbehaves. I'm not sure why this post routes through facebook, but here is a link to the article: http://www.cnet.com/roadshow/news/model-s-on-autopilot-crashes-into-other-car-proving-yet-again-tesla-owners-arent-paying-close-enough/ http://www.cnet.com/roadshow/news/model-s-on-autopilot-crash...
- alistairSH 10y ago
  Part of the problem is Tesla's "autopilot" isn't designed to be fully automatic. It's a diver's assistance package. As a comment in the Facebook thread mentioned, I too wonder if the problem is partial-autopilot in general. People will tend to put their trust in the system if it appears to be automatic. We probably shouldn't be shipping partial autopilot at all. Fully automatic or bust.
  - geoelectric 10y ago
    I have an assisted driving system in my Jeep. I'm pretty clear on its capabilities and, for example, stay alert to the possibility I have to hit the brakes when coming up on stopped traffic at freeway speeds because I'm outrunning its detection range for it to do so safely automatically. The problem is Tesla selling a system with similar limitations as "autopilot." It's not. It's an assisted driving system, albeit a very smart one, and they need to make it more clear that's what it is. They're setting incorrect expectations, and the various "I climbed into my passenger seat while it drives!" bits aren't helping. Basically we're living the old urban legend of the dude putting on cruise control and then crawling into his camper for a nap. The cruise control is a little bit smarter nowadays, but it comes down to a gross misunderstanding that Tesla is (inadvertently) encouraging.
    - alistairSH 10y ago
      Does the Jeep system function if you don't provide any inputs? The Subaru system will brake if it has too, but won't actually drive the car for you. Same for lane holding - it'll give some steering input and beep if you start to swerve, but it won't actively steer around a curve.
hayd 10y ago
The collision itself looks surprisingly minor.
raverbashing 10y ago
From the Article " However, as YouTube commenter Shaimach points out, the Model S manual calls out this exact situation as something drivers need to be aware of: Warning: Traffic-aware cruise control may not brake/decelerate for stationary vehicles, especially in situations when you are driving over 50 mph (80 km/h) and a vehicle you are following moves out of your driving path and a stationary vehicle or object is in front of you instead. Always pay attention to the road ahead and stay prepared to take immediate corrective action. Depending on TrafficAware Cruise Control to avoid a collision can result in serious injury or death."
gbin 10y ago
I guess from his experience their team rigging AI competitions at Baidu he might not be totally confident in real software :p
kbenson 10y ago
Crash? That was an accident, and it was definitely the auto-pilot's fault, but to me that looks like a fender-bender. Let's resist the media's attempt to use more extreme wording to sell stories. Yes, I know the driver prevented it from being worse, but that's one of the conditions Tesla's auto-pilot is supposed to be working under, a driver paying attention. There are problems with this model, but that doesn't mean we should ignore what actually happened in lieu of what could have happened when making a title.
- 27182818284 10y ago
  There has actually been a movement since at least 2000 to stop calling accidents "accidents" because it was found to make it seem like "lol accident nothing could have been done" I can recall the word "accident" being dropped by driver instructors and such back then at least. It might even go back further. Collision is better, but has a lot of letters so you condense it to crash.
  - FireBeyond 10y ago
    When driving emergency vehicles, the course you take that serves amongst other things as an exemption for having a CDL used to be called EVAP - Emergency Vehicle Accident Prevention, and is now called EVIP, Emergency Vehicle Incident Prevention.
pj_mukh 10y ago
I dont understand, the ML systems he designs are not deterministic by design! He's been letting that stuff fly (literally) for over a decade now[1]! P.S: I realize a "shipping" scenario is different. [1]https://www.youtube.com/watch?v=0JL04JJjocc https://www.youtube.com/watch?v=0JL04JJjocc
sksixk 10y ago
i can't get my head around a car and a human driver sharing the driving duties at the same time. if you can't trust the car to do everything correctly, then what am i supposed to do as a driver? sit there tensely with my right foot and hands hovering over the brake pedal and steering wheel?
- chadgeidel 10y ago
  How is this different than the automatic transmission? Antilock brakes? Traction control? Lane departure/encroachment warnings? Adaptive cruise control? Or even older tech like "drive by wire" (your foot isn't directly connected to the throttle bodies) or manually adjusting the choke in carbureted vehicles? These are innovations that have happened in my lifetime. I'm sure there are even older "car and driver sharing duties" examples others can come up with.
  - robbrown451 10y ago
    It's very different. But some of those things, like adaptive cruise control, get close. But even with adaptive cruise control, if you completely stop paying attention, you will be reminded very quickly that you are doing it wrong. With this, you could go for hours without having to do a single thing, and then suddenly you are expected to jump in and take control with barely a second's notice. That is just bad human factors.
    - chadgeidel 10y ago
      I would refer to @mikeash's answer as I feel it addresses this question.
      - robbrown451 10y ago
        Again, I don't see how that is supposed to work. How long are people going to pay attention to "the big picture" when they still are only very rarely required to do something? If they don't have to do some action to actually keep the car on the road or to avoid getting honked at every 30 seconds or so, their minds will drift off. This is just regular human nature. Sure you can do it for a while, but after a while you stop doing it. I am quite sure that Tesla et al will discover soon enough you can't rely on people to step in like that. They just need to get their cars to work better than humans, all the time. (or as Elon Musk said, they should be at least an order of magnitude safer)
suprgeek 10y ago
Tesla is innovating rapidly in the Autonomous driving space AND releasing those inventions to the customer, which is great. Typically these things used to take 5-7years to "trickle down" where a reasonably-priced car (not that $80k is reasonable) might have these available for end users. However Tesla absolutely NEEDS to do more to prevent this technology push model that they have from tainting the whole Autonomous Driving trend. Expecting that a Driver will read the f'ing manual about "....Traffic-aware cruise control may not brake/decelerate for stationary vehicles, especially in situations when you are driving over 50 mph (80 km/h) ....Depending on TrafficAware Cruise Control to avoid a collision can result in serious injury or death." is insane. My suggestion: Enforce a must watch video tutorial with questions (such as is common for say a driving test) before enabling the Partial AutoDrive. It really is a matter of Life or Death - you have a very short time to react on the road. With their distribution model Tesla definately has the resources to pull this off, and might serve as a model for the rest of the industry.
- vhold 10y ago
  I kind of wonder if Tesla figures "Hey, once a few crashes happen people will figure it out."
  - madelinecameron 10y ago
    It doesn't matter what Tesla does or says. Stupid always finds a way.
- vonklaus 10y ago
  my guess is, without any insider information, they were (or at least would be) willing to deal with negative pr, for the amount of data this program is likely to get.' While google (and many others) have been working on self-driving cars/testing systems, Tesla is a car company with many active vehicles. They benefit from huge fleets of vehicles (comparative to other autonomous car companies not car manufacturers) of which to test. So releasing this product and having potentially 50-100K people using it daily as well as over millions of miles logged could be net worth it. They probably assumed (and likely calculated) a falure mode would be non-lethal/minimal dents and very small risk of human life. So they would allow the tech out there early and hope that a lot of this data helped to improve the vehicles. It isn't PC to say, but this is how humans have learned most things. Being 90% something works, and then getting last 10 percent by learning what not to do which often comes at with a cost as high as human life. For example, the Challenger mission was a tradgedy and we, and very unfortunately some brave men and woman, paid a price for us to learn how to protect those future astronauts.
shogun21 10y ago
The autopilot is still in beta and warns drivers to still pay attention to the road. As long as drivers have control, they are responsible for the vehicle. Only after we remove steering wheels, the accelerator, and brakes; and have a fully autonomous system should the car manufacturers (or software manufacturers) be blamed.
- qbrass 10y ago
  They should be blamed simply for putting beta software in cars on public roads.
- mdorazio 10y ago
  While that might be true purely from a legal standpoint, it just doesn't make much sense logically. Let's compare it to something like a gas stove. If you turn the stove on with a bunch of combustible stuff around it and then walk away for ten minutes and your house burns down as a result, that's pretty obviously your own fault. However, if you turn the stove on with just a normal pot of water on top, walk away for ten minutes, and it self destructs and burns your house down, what would you think? Is it your fault for not standing there with a giant fire extinguisher, or is it the stove company's fault for making a poorly QA'd product? I think Tesla's Autopilot feature is a lot more like the latter example than the former.
calcsam 10y ago
Tesla should take a leaf out of Airbnb's book and pay for all damages caused by a customer using Autopilot in the manner it was intended to be used. http://mashable.com/2011/08/01/airbnb-ransackgate/#K9lVMU5w_iqF http://mashable.com/2011/08/01/airbnb-ransackgate/#K9lVMU5w_...
- rasz_pl 10y ago
  you mean like this: https://boingboing.net/2016/05/20/airbnb-stealth-updates-terms-o.html https://boingboing.net/2016/05/20/airbnb-stealth-updates-ter... ?
27182818284 10y ago
Cruise control can also lull you into not paying attention. Autopilot is simply marketing term for "advanced cruise control" which is what this really is right now.
bargl 10y ago
First, Andrew Ng is a co-founder of Coursera and helped teach a few Machine Learning classes just so we all know he has the chops to say something like this. That's how I recognized his name a little more digging shows he really does have a good ML/AI background. Second, He isn't saying an imperfect autopilot. He's saying that 1000 to 1 is NOT acceptable error. So the title is misleading, unless he edited his post. Third, he is correct. Lay people who see this will immediately think that Autopilot is WAY to hard and that software devs will never get it right. I'm not saying they are right, but I am saying it's terrible PR for AI to release a system that is as buggy as this and assume zero responsibility.
robbrown451 10y ago
I agree it is irresponsible. I think Google is correct in saying that self driving cars probably shouldn't even have steering wheels...it is absolutely unrealistic to think that people will hang out for an hour while the car does things correctly, and then be alert when it happens to not do things correctly. Humans don't work like that.
lacker 10y ago
If an imperfect autopilot is better than a human pilot, we should use it!
rasz_pl 10y ago
Tesla is NOT shipping an autopilot in the first place. They are selling LANE FOLLOWING driver assisted system and CALLING it autonomous AI "Autopilot". This is the main problem.
freddealmeida 10y ago
I'm just wondering if Tesla has responded to this? Do we know it was the Autopilot? Do we have evidence? I know Tesla captures a great deal of information from the car.
sunstone 10y ago
This 'imperfection' probably needs to be weighed against the fact that Tesla drivers apparently drive 2.5 million miles a DAY on autopilot. Cars have other imperfections that lead to accidents as well, like obstructed vision and blind spots for example. The real question is, is the autopilot, imperfections included, statistically a better driver than a human pilot also with imperfections. One or two fender benders does not make the case.

51 points by impish19 10y ago

WalterSear 10y ago

An imperfect autopilot, running on extra cpu cycles from the automatic door lock processor.

archildress 10y ago

This is just the cost of progress. If every imperfect piece of technology didn't ship, we'd have no technology at all.

riffraff 10y ago
not all technologies are created equal. If my macbook's battery dies when it rains, it's not the same as if my car crashes into another one injuring multiple people.
- moron4hire 10y ago
  One might make the argument that the automobile--as originally released without autopilot--is a deeply flawed device and that even this level of "flawed" autopilot is superior.
  - riffraff 10y ago
    the problem here is not that the thing is flawed, is that it's flawed in a radical way that users are not aware of, i.e. > Traffic-aware cruise control may not brake/decelerate for stationary vehicles When you drive a car, you somehow expect to crash into something if you don't pay attention. If you let your autopilot do it, crashing into stationary objects is not something you are prepared for.
    - moron4hire 10y ago
      The point I'm making is that we should not be concerned with making a perfect autopilot, but more importantly, we should not be concerned with making an autopilot that is as good as a human driver in all aspects in which human drivers are good. We should be concerned with making one that reduces injury. If that means that we have more low-speed fender benders--at the cost of zero drunk-driving accidents--I'm all for it.
  - erobbins 10y ago
    That would be a terrible argument, but ok.
dman 10y ago
One mishap which gets a lot of public scrutiny could set the field back by decades. When it comes to products with public safety implications its better to emulate the airline industry and their approach to safety rather than the "move fast and break things" mantra. For an example of how a few catastrophic events can destroy an entire industry / approach forever we just have to look at the nuclear industry with (chernobyl / three mile) and blimps (Hindenberg).
nas 10y ago
That's not much of an argument. There has to be a standard where you consider the new and imperfect technology "good enough". Is Telsa's auto-pilot good enough? I don't know. My perception is that since Telsa is an underdog of the auto industry, they are more willing to take a risk and put the technology out there. If it turns out to be too unsafe, their company will take a huge hit. If not, they gain market share.
fucking_tragedy 10y ago
If your wife or child was injured because of this, would that still be just the cost of progress? Technology can be refined and rigorously tested before it's pushed to consumers and that is still progress.

shawndumas 10y ago

he should try driving a model T...

83457 10y ago

Anyone have public link to video?

tyrust 10y ago
https://www.youtube.com/watch?v=qQkx-4pFjus https://www.youtube.com/watch?v=qQkx-4pFjus
deleted 10y ago
[deleted]

moron4hire 10y ago

Because the regular pilots are so good at the job?

coralreef 10y ago
Tesla isn't responsible for how a driver drives their car. But they are responsible for the software they ship.
- moron4hire 10y ago
  If Tesla can ship a not-perfect autopilot that could significantly reduce injury over not using it at all, shouldn't we abandon this pursuit of "perfection" and release early?
  - dilemma 10y ago
    Tesla's non-perfect "autopilot" seems to be causing accidents a human wouldn't.
    - moron4hire 10y ago
      Have you been on the roads much?
      - dilemma 10y ago
        Im referring to Teslas hitting stationary objects. One such story was posted here a week ago.
        moron4hire 10y ago
        People also hit stationary objects.

drivingmenuts 10y ago

Well, seeing how Tesla just handed every auto insurance company in the world a free get-out-of-paying card for accidents dealing with that car, I'd say Tesla screwed up. Seeing as how Tesla has potentially exposed everyone involved in the supply chain to lawsuits, then yes, they screwed up. If I were a well-backed lawyer, right about now, I'd be girding up to file requests for colonoscopies on the people who assembled the circuit boards in those cars. Seriously, right about now, some lawyer is figuring out how to suck money out of Tesla until it's a dry, twitching husk in the noonday say. Tesla handed that guy a gift.

xpda 10y ago

There will be better autopilots, but never perfect ones. Aircraft have been using imperfect autopilots since their inception. Most planes have a prominent red button you can press to disable the autopilot whenever it misbehaves. I'm not sure why this post routes through facebook, but here is a link to the article: http://www.cnet.com/roadshow/news/model-s-on-autopilot-crashes-into-other-car-proving-yet-again-tesla-owners-arent-paying-close-enough/ http://www.cnet.com/roadshow/news/model-s-on-autopilot-crash...

alistairSH 10y ago
Part of the problem is Tesla's "autopilot" isn't designed to be fully automatic. It's a diver's assistance package. As a comment in the Facebook thread mentioned, I too wonder if the problem is partial-autopilot in general. People will tend to put their trust in the system if it appears to be automatic. We probably shouldn't be shipping partial autopilot at all. Fully automatic or bust.
- geoelectric 10y ago
  I have an assisted driving system in my Jeep. I'm pretty clear on its capabilities and, for example, stay alert to the possibility I have to hit the brakes when coming up on stopped traffic at freeway speeds because I'm outrunning its detection range for it to do so safely automatically. The problem is Tesla selling a system with similar limitations as "autopilot." It's not. It's an assisted driving system, albeit a very smart one, and they need to make it more clear that's what it is. They're setting incorrect expectations, and the various "I climbed into my passenger seat while it drives!" bits aren't helping. Basically we're living the old urban legend of the dude putting on cruise control and then crawling into his camper for a nap. The cruise control is a little bit smarter nowadays, but it comes down to a gross misunderstanding that Tesla is (inadvertently) encouraging.
  - alistairSH 10y ago
    Does the Jeep system function if you don't provide any inputs? The Subaru system will brake if it has too, but won't actually drive the car for you. Same for lane holding - it'll give some steering input and beep if you start to swerve, but it won't actively steer around a curve.

hayd 10y ago

The collision itself looks surprisingly minor.

raverbashing 10y ago

From the Article " However, as YouTube commenter Shaimach points out, the Model S manual calls out this exact situation as something drivers need to be aware of: Warning: Traffic-aware cruise control may not brake/decelerate for stationary vehicles, especially in situations when you are driving over 50 mph (80 km/h) and a vehicle you are following moves out of your driving path and a stationary vehicle or object is in front of you instead. Always pay attention to the road ahead and stay prepared to take immediate corrective action. Depending on TrafficAware Cruise Control to avoid a collision can result in serious injury or death."

gbin 10y ago

I guess from his experience their team rigging AI competitions at Baidu he might not be totally confident in real software :p

kbenson 10y ago

Crash? That was an accident, and it was definitely the auto-pilot's fault, but to me that looks like a fender-bender. Let's resist the media's attempt to use more extreme wording to sell stories. Yes, I know the driver prevented it from being worse, but that's one of the conditions Tesla's auto-pilot is supposed to be working under, a driver paying attention. There are problems with this model, but that doesn't mean we should ignore what actually happened in lieu of what could have happened when making a title.

27182818284 10y ago
There has actually been a movement since at least 2000 to stop calling accidents "accidents" because it was found to make it seem like "lol accident nothing could have been done" I can recall the word "accident" being dropped by driver instructors and such back then at least. It might even go back further. Collision is better, but has a lot of letters so you condense it to crash.
- FireBeyond 10y ago
  When driving emergency vehicles, the course you take that serves amongst other things as an exemption for having a CDL used to be called EVAP - Emergency Vehicle Accident Prevention, and is now called EVIP, Emergency Vehicle Incident Prevention.

pj_mukh 10y ago

I dont understand, the ML systems he designs are not deterministic by design! He's been letting that stuff fly (literally) for over a decade now[1]! P.S: I realize a "shipping" scenario is different. [1]https://www.youtube.com/watch?v=0JL04JJjocc https://www.youtube.com/watch?v=0JL04JJjocc

sksixk 10y ago

i can't get my head around a car and a human driver sharing the driving duties at the same time. if you can't trust the car to do everything correctly, then what am i supposed to do as a driver? sit there tensely with my right foot and hands hovering over the brake pedal and steering wheel?

chadgeidel 10y ago
How is this different than the automatic transmission? Antilock brakes? Traction control? Lane departure/encroachment warnings? Adaptive cruise control? Or even older tech like "drive by wire" (your foot isn't directly connected to the throttle bodies) or manually adjusting the choke in carbureted vehicles? These are innovations that have happened in my lifetime. I'm sure there are even older "car and driver sharing duties" examples others can come up with.
- robbrown451 10y ago
  It's very different. But some of those things, like adaptive cruise control, get close. But even with adaptive cruise control, if you completely stop paying attention, you will be reminded very quickly that you are doing it wrong. With this, you could go for hours without having to do a single thing, and then suddenly you are expected to jump in and take control with barely a second's notice. That is just bad human factors.
  - chadgeidel 10y ago
    I would refer to @mikeash's answer as I feel it addresses this question.
    - robbrown451 10y ago
      Again, I don't see how that is supposed to work. How long are people going to pay attention to "the big picture" when they still are only very rarely required to do something? If they don't have to do some action to actually keep the car on the road or to avoid getting honked at every 30 seconds or so, their minds will drift off. This is just regular human nature. Sure you can do it for a while, but after a while you stop doing it. I am quite sure that Tesla et al will discover soon enough you can't rely on people to step in like that. They just need to get their cars to work better than humans, all the time. (or as Elon Musk said, they should be at least an order of magnitude safer)

suprgeek 10y ago

Tesla is innovating rapidly in the Autonomous driving space AND releasing those inventions to the customer, which is great. Typically these things used to take 5-7years to "trickle down" where a reasonably-priced car (not that $80k is reasonable) might have these available for end users. However Tesla absolutely NEEDS to do more to prevent this technology push model that they have from tainting the whole Autonomous Driving trend. Expecting that a Driver will read the f'ing manual about "....Traffic-aware cruise control may not brake/decelerate for stationary vehicles, especially in situations when you are driving over 50 mph (80 km/h) ....Depending on TrafficAware Cruise Control to avoid a collision can result in serious injury or death." is insane. My suggestion: Enforce a must watch video tutorial with questions (such as is common for say a driving test) before enabling the Partial AutoDrive. It really is a matter of Life or Death - you have a very short time to react on the road. With their distribution model Tesla definately has the resources to pull this off, and might serve as a model for the rest of the industry.

vhold 10y ago
I kind of wonder if Tesla figures "Hey, once a few crashes happen people will figure it out."
- madelinecameron 10y ago
  It doesn't matter what Tesla does or says. Stupid always finds a way.
vonklaus 10y ago
my guess is, without any insider information, they were (or at least would be) willing to deal with negative pr, for the amount of data this program is likely to get.' While google (and many others) have been working on self-driving cars/testing systems, Tesla is a car company with many active vehicles. They benefit from huge fleets of vehicles (comparative to other autonomous car companies not car manufacturers) of which to test. So releasing this product and having potentially 50-100K people using it daily as well as over millions of miles logged could be net worth it. They probably assumed (and likely calculated) a falure mode would be non-lethal/minimal dents and very small risk of human life. So they would allow the tech out there early and hope that a lot of this data helped to improve the vehicles. It isn't PC to say, but this is how humans have learned most things. Being 90% something works, and then getting last 10 percent by learning what not to do which often comes at with a cost as high as human life. For example, the Challenger mission was a tradgedy and we, and very unfortunately some brave men and woman, paid a price for us to learn how to protect those future astronauts.

shogun21 10y ago

The autopilot is still in beta and warns drivers to still pay attention to the road. As long as drivers have control, they are responsible for the vehicle. Only after we remove steering wheels, the accelerator, and brakes; and have a fully autonomous system should the car manufacturers (or software manufacturers) be blamed.

qbrass 10y ago
They should be blamed simply for putting beta software in cars on public roads.
mdorazio 10y ago
While that might be true purely from a legal standpoint, it just doesn't make much sense logically. Let's compare it to something like a gas stove. If you turn the stove on with a bunch of combustible stuff around it and then walk away for ten minutes and your house burns down as a result, that's pretty obviously your own fault. However, if you turn the stove on with just a normal pot of water on top, walk away for ten minutes, and it self destructs and burns your house down, what would you think? Is it your fault for not standing there with a giant fire extinguisher, or is it the stove company's fault for making a poorly QA'd product? I think Tesla's Autopilot feature is a lot more like the latter example than the former.

calcsam 10y ago

Tesla should take a leaf out of Airbnb's book and pay for all damages caused by a customer using Autopilot in the manner it was intended to be used. http://mashable.com/2011/08/01/airbnb-ransackgate/#K9lVMU5w_iqF http://mashable.com/2011/08/01/airbnb-ransackgate/#K9lVMU5w_...

rasz_pl 10y ago
you mean like this: https://boingboing.net/2016/05/20/airbnb-stealth-updates-terms-o.html https://boingboing.net/2016/05/20/airbnb-stealth-updates-ter... ?

27182818284 10y ago

Cruise control can also lull you into not paying attention. Autopilot is simply marketing term for "advanced cruise control" which is what this really is right now.

bargl 10y ago

First, Andrew Ng is a co-founder of Coursera and helped teach a few Machine Learning classes just so we all know he has the chops to say something like this. That's how I recognized his name a little more digging shows he really does have a good ML/AI background. Second, He isn't saying an imperfect autopilot. He's saying that 1000 to 1 is NOT acceptable error. So the title is misleading, unless he edited his post. Third, he is correct. Lay people who see this will immediately think that Autopilot is WAY to hard and that software devs will never get it right. I'm not saying they are right, but I am saying it's terrible PR for AI to release a system that is as buggy as this and assume zero responsibility.

robbrown451 10y ago

I agree it is irresponsible. I think Google is correct in saying that self driving cars probably shouldn't even have steering wheels...it is absolutely unrealistic to think that people will hang out for an hour while the car does things correctly, and then be alert when it happens to not do things correctly. Humans don't work like that.

lacker 10y ago

If an imperfect autopilot is better than a human pilot, we should use it!

rasz_pl 10y ago

Tesla is NOT shipping an autopilot in the first place. They are selling LANE FOLLOWING driver assisted system and CALLING it autonomous AI "Autopilot". This is the main problem.

freddealmeida 10y ago

I'm just wondering if Tesla has responded to this? Do we know it was the Autopilot? Do we have evidence? I know Tesla captures a great deal of information from the car.

sunstone 10y ago

This 'imperfection' probably needs to be weighed against the fact that Tesla drivers apparently drive 2.5 million miles a DAY on autopilot. Cars have other imperfections that lead to accidents as well, like obstructed vision and blind spots for example. The real question is, is the autopilot, imperfections included, statistically a better driver than a human pilot also with imperfections. One or two fender benders does not make the case.