10 ms·

"Our data shows that Tesla owners have driven this same stretch of highway with Autopilot engaged roughly 85,000 times since Autopilot was first rolled out in 2

"Our data shows that Tesla owners have driven this same stretch of highway with Autopilot engaged roughly 85,000 times since Autopilot was first rolled out in 2015. ... There are over 200 successful Autopilot trips per day on this exact stretch of road." Who wrote this statement does not understand software. If there is some kind of Heisenbug, those numbers are too small to prove that the software is ok, and actually the guy that died reporting that there was an issue there, and later crashing there, looks like a very interesting hint about a potential software bug that should be investigated. The Tesla statement is the equivalent of "works in my laptop" at a bigger scale. Also consider that 85k times since 2015 means that potentially only a fraction of those trips were executed with the latest version of the software. Moreover the street layout may now be kinda different triggering some new condition.

shawn 9y ago
Heisenbugs in AI are also interesting. As computers approach sentience, it will become progressively harder to explain their behavior. Why did you eat a bagel instead of eggs this morning? Usually there isn't a scientific answer. In the present, AI models are already so complicated that it seems hard to get reproducible diagnostic results, short of just saving every frame of data that the car's sensors pick up. And for video that seems rather prohibitive. Imagine collecting all sensor data for each of those 200 trips per day along the entire stretch of road.
- buvanshak 9y ago
  https://en.wikipedia.org/wiki/Catastrophic_interference https://en.wikipedia.org/wiki/Catastrophic_interference >Catastrophic interference, also known as catastrophic forgetting, is the tendency of an artificial neural network to completely and abruptly forget previously learned information upon learning new information.
  - tensor 9y ago
    These self driving systems are not likely learning in realtime. When the algorithms are updated, best practice is to validate their performance on a large regression set. So something like this should be caught during validation, assuming your test data is good.
    - throwaway2048 9y ago
      No set of test data can exist that covers every real world situation.
- smartician 9y ago
  Aren't many AI methods also inherently probabilistic? That would make it by definition impossible to definitively explain any particular behavior.
  - tytytytytytytyt 9y ago
    I don't think that means it's impossible. You can still log all inputs and sources of entropy.
  - baddox 9y ago
    What do you mean by probabilistic? If you mean that the models takes input and outputs one or more labeled probabilities (e.g. “90% confidence the input photo is a dog, 10% confidence the input photo is a cat), then yes, I believe that many AI systems work this way. If you mean random, in the sense that the system may return different results given the exact same inputs, then in not sure if there are AI systems that work that way.
    - joshuamorton 9y ago
      There are. Monte Carlo methods is the keyword you're looking for. AlphaGo (Monte Carlo Tree Search) is an example of one such AI. Obviously you can set the RNG seed to be the same every time too, but even that only works if your system is wholly synchronous, which a car probably isn't. Note that I doubt Monte Carlo methods are common in the autonomous vehicle space.
      - TaylorAlexander 9y ago
        Well I don’t work in autonomous cars but I did work on several autonomous robots, and “ Adaptive Monte Carlo Localization” is a common way to keep a ROS + LIDAR based robot localized. I wouldn’t be surprised if Monte Carlo techniques are useful for all manner of things related to ingesting sensor data on autonomous vehicles.
      - chaboud 9y ago
        I'd expect Monte Carlo methods to be used in a number of cases that have deterministic time envelopes for evaluation. Randomized selection and evaluation can be incredibly effective. They also resist degenerate structured input vulnerabilities. I'm not in the automotive space, but I'd be surprised if there were a viable self-driving car team not using Monte Carlo methods somewhere in the vehicle stack.
- azinman2 9y ago
  Id be willing to take a 1k bet that computers won’t approach sentience in our lifetime. Don’t believe the AI hype train, there hasn’t been a real breakthrough. Just more accurate versions of the same (dumb) classifiers.
  - baddox 9y ago
    Why do you think that humans are not just even more accurate versions of the same types of classifiers?
    - neffy 9y ago
      because we invented cars. And a few other things.
      - dorgo 9y ago
        Among these other things which we invented are algorithms which invented other algorithms to solve real world problems. There is nothing unuique or irreplicable about humans (save of culture, emotions, etc. which nobody wants to replicate).
        neffy 9y ago
        While it may come to pass that we do those things, so far all we've managed to replicate is the first 6 or so levels of the visual processing field. Methinks there is far further to go than we realise - especially with the current air of hubris around the entire endeavour.
    - azinman2 9y ago
      Because we integrate, learn, plan, abstract, create, have way more senses, complex memory, societal and cultural embeddings, and a whole host of things that make us human. A text box on hacker news is not the place for such a reply. Go take some cognitive science, neuroscience, anthropology, sociology, psychology, and philosophy classes and you’ll see why. “AI” as we know it is a joke, particularly as media has hyped it. The original guys (McCarthy, minskey, etc) at least understand the gigantic complexity of the undertaking in so far as we understand how the mind works at all, and they never reduced things purely to classifiers. That’s like 0.1% of the job. The real researchers take philosophy of mind seriously — technologists want a quick and cheap solution that cannot exist.
- joesmo 9y ago
  If they can't figure out how to store that data they shouldn't be on the road. You're telling me it's a data storage / transmission problem that their best engineers can't solve, yet we're supposed to trust their machine learning system? Tons of companies are working on this and plenty seem to have figured out even bigger big data issues already.
- jobigoud 9y ago
  We also see this in unexplainable flash crashes from high speed trading softwares gone bersek. The speed at which they handle data is so immense than going through the logs of a few seconds around the issue could take several years of human time.
debt 9y ago
I agree. I find it hypocritical of Tesla to talk about numbers in thousands in terms of Model 3 production, but low 100's is a totally sufficient metric for testing autopilot.
hcnews 9y ago
As someone who writes software (haven't ever worked for Tesla), I think this statement does provide some credibility. It probably means that the visualization system does understand such scenes and is able to navigate it. Ofcourse, this statement by itself doesn't absolve Tesla's software of any error. It's important to know the actual root cause before saying who/what is wrong or not, something which the HN community (and in general Internet/people) isn't good at.
- ironjunkie 9y ago
  Serious question, but how does working for "FAANG" companies provides any credibility in this specific context? That must be like half the bay area. I get that it's written as a token of pride for you, but maybe worth to take a step back?
  - deleted 9y ago
    [deleted]
  - hcnews 9y ago
    Fair enough, I added it in the flow of saying I don't work for Tesla. Removed it.
- jacquesm 9y ago
  Appeal to authority ignored. > It probably means that the visualization system does understand such scenes and is able to navigate it. I think there is a pretty good counter example proving that it doesn't.
- bonestamp2 9y ago
  > doesn't absolve Tesla's software of any error As someone who works on software for one of the automakers, I also want to note that there seems to be a lot of focus on the software... but it's also very possible the problem was with the hardware: something in the vision or sensor system was incorrectly installed, defective, was calibrated incorrectly, lost calibration during use, etc. I'm sure the software can always be improved, but if most cars are driving through this stretch without a problem and his always veers, that sounds like something unique to his vehicle which points to hardware in my opinion. In cars, the software can only ever be as good as the data it receives.
dirtae 9y ago
I believe the accident happened on US 101, which is a multi-lane highway with 5 or 6 lanes at the spot where the accident occurred. How many of the trips Tesla is citing occurred with the vehicle in the same lane as this guy? Comparing apples to apples would probably greatly reduce these 85,000 / 200 per day figures.
- ineedasername 9y ago
  Exactly. Same lane, same time of day, same time of year (sunset/sunrise changes in lighting conditions) same weather patterns, same level of traffic. The combinatorics involved expand the possibilities to the point that some scenariors may have had little or no coverage from a dataset with 85k observations.
  - ethbro 9y ago
    There was a big note in the GTC keynote this year about Nvidia using virtual environments to debug their autopilot algorithms. Think "car" being driven in a high fidelity video game. Advantage being that if you can provide representative, simulated input then you can increase the training miles by orders of magnitude in the same amount of time, limited by computation rather than physical mileage.
    - firebones 9y ago
      This is true, to the extent that the selection of input parameters to simulate provides coverage over the domain of all possible input parameters. At some point you're back to testing the failure of imagination of the simulation/test creators.
    - throwaway2048 9y ago
      The problem is the simulation by definition can only simulate things that are accounted for. Any number of completely arbitrary out of the blue things can happen in real life.
      - 9y ago
ineedasername 9y ago
It also means they don't have significant representation of all driving conditions on the stretch, placement of sun, overcast, lighting conditions at night, traffic, weather, etc. I can imagine at least a dozen very distinct conditions, and many more subtle variations. 85k isn't very large to get good coverage of all conditions and generalize from them.
ggg9990 9y ago
That person is likely Elon Musk who is well-known for his obstinate and even idiotic defense of Tesla against legitimate and thoughtful criticism. He has the defensiveness of Donald Trump, while even Zuckerberg admitted fault.
- zouhair 9y ago
  Didn't he sue Top Gear and lost over a bit Jeremy Clarkson did? EDIT: Yes he did and lost[0]. Jeremy Clarkson did another bit in The Grand Tour about the new Tesla and it was quite British[1]. [0]http://www.thedrive.com/sheetmetal/12536/remember-when-top-gear-and-tesla-clashed-over-the-roadster http://www.thedrive.com/sheetmetal/12536/remember-when-top-g... [1]https://www.youtube.com/watch?v=_DiGQRaaHvg https://www.youtube.com/watch?v=_DiGQRaaHvg
  - albertop 9y ago
    This is great. Thanks for posting.
    - tim333 9y ago
      The video is funny. Re the first argument Top Gear wrote a script with the Telsa breaking down and faked the thing so I can see why Musk was annoyed. They lost the court case because Top Gear argued Tesla couldn't show they'd hurt sales.
      - zouhair 9y ago
        It is obvious he didn't watch Top Gear, Jeremy Clarkson shat on many many cars, just ask Renault or Peugeot.
oldgradstudent 9y ago
They also provide numbers without any context. Is a crash every 85,000 times in that intersection means the Autopilot is safe or is it a death trap? There are over 200,000 cars a day driving at that stretch of highway, if a car crashes there once a week, then the Tesla Autopilot is an order of magnitude more dangerous than a human driver.
- greglindahl 9y ago
  How is Tesla supposed to know, yet, if Autopilot actually made the maneuver? They're saying what they know. Edit: in a posting today, Tesla has said more: https://www.tesla.com/blog/update-last-week’s-accident https://www.tesla.com/blog/update-last-week’s-accident but at the time of the earlier blog posting, Tesla says they did not know yet.
  - dingo_bat 9y ago
    They definitely know. They collect your driving data even when auto pilot is off, so I'd imagine they collect everything when the autopilot is actually engaged.
    - greglindahl 9y ago
      Tesla specifically said that the information was in the car and wasn't transmitted to HQ, so they didn't know. If what you say is true, they're lying to the USG. Let me share an example. I had an accident in a Model S, front-end, which didn't cause a loss of power. Tesla called me a couple of minutes later, saying that they knew I'd had an airbag-deploying accident, and asking if I needed an ambulance. From the description of this accident, power to the Tesla was lost before there was time to phone home.
  - brfox 9y ago
    From their blog: "In the US, there is one automotive fatality every 86 million miles across all vehicles from all manufacturers. For Tesla, there is one fatality, including known pedestrian fatalities, every 320 million miles in vehicles equipped with Autopilot hardware. If you are driving a Tesla equipped with Autopilot hardware, you are 3.7 times less likely to be involved in a fatal accident." Notice it says one fatality per "320 million miles in vehicles equipped with Autopilot hardware." But, how many of those miles (or what percent of the time) does a Tesla driver use Autopilot? Also, maybe those good stats are due to the structural safety features and not the autopilot. It may be more fair to compare to other cars with excellent crash protection but no autopilot. I think it would be a great service to the world to improve driving safety, but maybe we need to really start looking at the stats and get some more hubris as we transition to full autopilot. For example, require that drivers keep their hands on the wheel and eyes on the road. When more cars have autopilot, then mesh behavior between vehicles, and e-road features, then maybe we'll be more ready for driver-less cars.
skybrian 9y ago
I don't see what's wrong with what they wrote. This statement of course doesn't show that the software is safe, but it is evidence that if there is a bug, it probably isn't easily triggered. (Keep in mind that until they recover the logs, we don't even know if the autopilot was on.)
- rrdharan 9y ago
  They have confirmed the autopilot was on: https://www.tesla.com/blog/update-last-week’s-accident https://www.tesla.com/blog/update-last-week’s-accident
  - DrScump 9y ago
    But does "autopilot on" mean that "autopilot has exclusive control and driver inputs are ignored"?
    - pacificmint 9y ago
      No, of course driver inputs will not be ignored. They will override the autopilot. But none were given. From the blog: >In the moments before the collision, which occurred at 9:27 a.m. on Friday, March 23rd, Autopilot was engaged with the adaptive cruise control follow-distance set to minimum. The driver had received several visual and one audible hands-on warning earlier in the drive and the driver’s hands were not detected on the wheel for six seconds prior to the collision. The driver had about five seconds and 150 meters of unobstructed view of the concrete divider with the crushed crash attenuator, but the vehicle logs show that no action was taken.
      - lazyjones 9y ago
        > The driver had about five seconds and 150 meters of unobstructed view of the concrete divider with the crushed crash attenuator So, clearly, the AP is not capable of avoiding such obstacles yet, even when unobstructed. It also seems like a mistake to not slow down to a survivable speed when hands are off...
Animats 9y ago
Misaligned sensor?
- sschueller 9y ago
  In such a case there needs to be a fail safe that can determine that and disable auto pilot until it is fixed.
mirimir 9y ago
It's possible that some of those "200 successful Autopilot trips per day on this exact stretch of road" also involve pulling toward that barrier. However, it's not likely a huge percentage, or we'd have seen that posted. So the implication is that his Tesla was relatively unique. That is, broken.
lafar6502 9y ago
software guys should know better that programs will have bugs...
dengnan 9y ago
Let's play with statistics: Scenario 1: 100 human drivers drive some distance. 2 bad drivers caused 150 accidents and both died in their last fatal accidents. Scenario 2: 100 self-driving/autopilot-enabled cars drive the same road for the same distance. Everyone has one accident and one of them is fatal. Statistical numbers: Self-driving/autopilot-enabled cars cut both the accident rate and death rate by half. Death rate per accident is reduced from 2/150 to 1/100. Conclusion: self-driving/autopilot is better.
- deleted 9y ago
  [deleted]
- nikanj 9y ago
  Human accidents, however, are not news. "Tesla also posted these photos that raise another important question: they show what's called a "crash attenuator" or safety barrier in the proper condition ... and the way it was the day before Walter Huang's crash ... collapsed after a different accident." Did you see any news coverage of the previous accident? I know I didn't.
- CPLX 9y ago
  If we're making up statistics out of thin air can we do a couple with people riding ponies too?
z3t4 9y ago
When users discover a bug or issue they tend to work around it. Even as a developer for example you are debugging something and stumbling on another unrelated issue you are likely to ignore it. Users very rarely report issues. Self driving cars testers should be paid per issue found, not per hour.
merb 9y ago
Well I found the comment of tesla, odd as well. but it is also odd that the driver still kept driving the same road WITH autopilot activated, besides he KNEW it was broken. It's a strange accident, especially because of the strangeness of tesla and the whole background story (broken safety barrier). I'm looking forward to the outcome of the whole story. At the end I guess more than one party could've been a bad actor. Tesla, The guy itself and even that the road had no kind of warning (safety barrier not replaced or some kind of other warning sinces that it will be replaced soonish)
Qwertious 9y ago
> The Tesla statement is the equivalent of "works in my laptop" at a bigger scale. Erhem: https://i.imgur.com/sY3Eeln.png https://i.imgur.com/sY3Eeln.png
bryanrasmussen 9y ago
perhaps whoever wrote this statement does understand software but they also understand lying with numbers and knows the general public understands neither.

shawn 9y ago

Heisenbugs in AI are also interesting. As computers approach sentience, it will become progressively harder to explain their behavior. Why did you eat a bagel instead of eggs this morning? Usually there isn't a scientific answer. In the present, AI models are already so complicated that it seems hard to get reproducible diagnostic results, short of just saving every frame of data that the car's sensors pick up. And for video that seems rather prohibitive. Imagine collecting all sensor data for each of those 200 trips per day along the entire stretch of road.

buvanshak 9y ago
https://en.wikipedia.org/wiki/Catastrophic_interference https://en.wikipedia.org/wiki/Catastrophic_interference >Catastrophic interference, also known as catastrophic forgetting, is the tendency of an artificial neural network to completely and abruptly forget previously learned information upon learning new information.
- tensor 9y ago
  These self driving systems are not likely learning in realtime. When the algorithms are updated, best practice is to validate their performance on a large regression set. So something like this should be caught during validation, assuming your test data is good.
  - throwaway2048 9y ago
    No set of test data can exist that covers every real world situation.
smartician 9y ago
Aren't many AI methods also inherently probabilistic? That would make it by definition impossible to definitively explain any particular behavior.
- tytytytytytytyt 9y ago
  I don't think that means it's impossible. You can still log all inputs and sources of entropy.
- baddox 9y ago
  What do you mean by probabilistic? If you mean that the models takes input and outputs one or more labeled probabilities (e.g. “90% confidence the input photo is a dog, 10% confidence the input photo is a cat), then yes, I believe that many AI systems work this way. If you mean random, in the sense that the system may return different results given the exact same inputs, then in not sure if there are AI systems that work that way.
  - joshuamorton 9y ago
    There are. Monte Carlo methods is the keyword you're looking for. AlphaGo (Monte Carlo Tree Search) is an example of one such AI. Obviously you can set the RNG seed to be the same every time too, but even that only works if your system is wholly synchronous, which a car probably isn't. Note that I doubt Monte Carlo methods are common in the autonomous vehicle space.
    - TaylorAlexander 9y ago
      Well I don’t work in autonomous cars but I did work on several autonomous robots, and “ Adaptive Monte Carlo Localization” is a common way to keep a ROS + LIDAR based robot localized. I wouldn’t be surprised if Monte Carlo techniques are useful for all manner of things related to ingesting sensor data on autonomous vehicles.
    - chaboud 9y ago
      I'd expect Monte Carlo methods to be used in a number of cases that have deterministic time envelopes for evaluation. Randomized selection and evaluation can be incredibly effective. They also resist degenerate structured input vulnerabilities. I'm not in the automotive space, but I'd be surprised if there were a viable self-driving car team not using Monte Carlo methods somewhere in the vehicle stack.
azinman2 9y ago
Id be willing to take a 1k bet that computers won’t approach sentience in our lifetime. Don’t believe the AI hype train, there hasn’t been a real breakthrough. Just more accurate versions of the same (dumb) classifiers.
- baddox 9y ago
  Why do you think that humans are not just even more accurate versions of the same types of classifiers?
  - neffy 9y ago
    because we invented cars. And a few other things.
    - dorgo 9y ago
      Among these other things which we invented are algorithms which invented other algorithms to solve real world problems. There is nothing unuique or irreplicable about humans (save of culture, emotions, etc. which nobody wants to replicate).
      - neffy 9y ago
        While it may come to pass that we do those things, so far all we've managed to replicate is the first 6 or so levels of the visual processing field. Methinks there is far further to go than we realise - especially with the current air of hubris around the entire endeavour.
  - azinman2 9y ago
    Because we integrate, learn, plan, abstract, create, have way more senses, complex memory, societal and cultural embeddings, and a whole host of things that make us human. A text box on hacker news is not the place for such a reply. Go take some cognitive science, neuroscience, anthropology, sociology, psychology, and philosophy classes and you’ll see why. “AI” as we know it is a joke, particularly as media has hyped it. The original guys (McCarthy, minskey, etc) at least understand the gigantic complexity of the undertaking in so far as we understand how the mind works at all, and they never reduced things purely to classifiers. That’s like 0.1% of the job. The real researchers take philosophy of mind seriously — technologists want a quick and cheap solution that cannot exist.
joesmo 9y ago
If they can't figure out how to store that data they shouldn't be on the road. You're telling me it's a data storage / transmission problem that their best engineers can't solve, yet we're supposed to trust their machine learning system? Tons of companies are working on this and plenty seem to have figured out even bigger big data issues already.
jobigoud 9y ago
We also see this in unexplainable flash crashes from high speed trading softwares gone bersek. The speed at which they handle data is so immense than going through the logs of a few seconds around the issue could take several years of human time.

debt 9y ago

I agree. I find it hypocritical of Tesla to talk about numbers in thousands in terms of Model 3 production, but low 100's is a totally sufficient metric for testing autopilot.

hcnews 9y ago

As someone who writes software (haven't ever worked for Tesla), I think this statement does provide some credibility. It probably means that the visualization system does understand such scenes and is able to navigate it. Ofcourse, this statement by itself doesn't absolve Tesla's software of any error. It's important to know the actual root cause before saying who/what is wrong or not, something which the HN community (and in general Internet/people) isn't good at.

ironjunkie 9y ago
Serious question, but how does working for "FAANG" companies provides any credibility in this specific context? That must be like half the bay area. I get that it's written as a token of pride for you, but maybe worth to take a step back?
- deleted 9y ago
  [deleted]
- hcnews 9y ago
  Fair enough, I added it in the flow of saying I don't work for Tesla. Removed it.
jacquesm 9y ago
Appeal to authority ignored. > It probably means that the visualization system does understand such scenes and is able to navigate it. I think there is a pretty good counter example proving that it doesn't.
bonestamp2 9y ago
> doesn't absolve Tesla's software of any error As someone who works on software for one of the automakers, I also want to note that there seems to be a lot of focus on the software... but it's also very possible the problem was with the hardware: something in the vision or sensor system was incorrectly installed, defective, was calibrated incorrectly, lost calibration during use, etc. I'm sure the software can always be improved, but if most cars are driving through this stretch without a problem and his always veers, that sounds like something unique to his vehicle which points to hardware in my opinion. In cars, the software can only ever be as good as the data it receives.

dirtae 9y ago

I believe the accident happened on US 101, which is a multi-lane highway with 5 or 6 lanes at the spot where the accident occurred. How many of the trips Tesla is citing occurred with the vehicle in the same lane as this guy? Comparing apples to apples would probably greatly reduce these 85,000 / 200 per day figures.

ineedasername 9y ago
Exactly. Same lane, same time of day, same time of year (sunset/sunrise changes in lighting conditions) same weather patterns, same level of traffic. The combinatorics involved expand the possibilities to the point that some scenariors may have had little or no coverage from a dataset with 85k observations.
- ethbro 9y ago
  There was a big note in the GTC keynote this year about Nvidia using virtual environments to debug their autopilot algorithms. Think "car" being driven in a high fidelity video game. Advantage being that if you can provide representative, simulated input then you can increase the training miles by orders of magnitude in the same amount of time, limited by computation rather than physical mileage.
  - firebones 9y ago
    This is true, to the extent that the selection of input parameters to simulate provides coverage over the domain of all possible input parameters. At some point you're back to testing the failure of imagination of the simulation/test creators.
  - throwaway2048 9y ago
    The problem is the simulation by definition can only simulate things that are accounted for. Any number of completely arbitrary out of the blue things can happen in real life.
    - 9y ago

ineedasername 9y ago

It also means they don't have significant representation of all driving conditions on the stretch, placement of sun, overcast, lighting conditions at night, traffic, weather, etc. I can imagine at least a dozen very distinct conditions, and many more subtle variations. 85k isn't very large to get good coverage of all conditions and generalize from them.

ggg9990 9y ago

That person is likely Elon Musk who is well-known for his obstinate and even idiotic defense of Tesla against legitimate and thoughtful criticism. He has the defensiveness of Donald Trump, while even Zuckerberg admitted fault.

zouhair 9y ago
Didn't he sue Top Gear and lost over a bit Jeremy Clarkson did? EDIT: Yes he did and lost[0]. Jeremy Clarkson did another bit in The Grand Tour about the new Tesla and it was quite British[1]. [0]http://www.thedrive.com/sheetmetal/12536/remember-when-top-gear-and-tesla-clashed-over-the-roadster http://www.thedrive.com/sheetmetal/12536/remember-when-top-g... [1]https://www.youtube.com/watch?v=_DiGQRaaHvg https://www.youtube.com/watch?v=_DiGQRaaHvg
- albertop 9y ago
  This is great. Thanks for posting.
  - tim333 9y ago
    The video is funny. Re the first argument Top Gear wrote a script with the Telsa breaking down and faked the thing so I can see why Musk was annoyed. They lost the court case because Top Gear argued Tesla couldn't show they'd hurt sales.
    - zouhair 9y ago
      It is obvious he didn't watch Top Gear, Jeremy Clarkson shat on many many cars, just ask Renault or Peugeot.

oldgradstudent 9y ago

They also provide numbers without any context. Is a crash every 85,000 times in that intersection means the Autopilot is safe or is it a death trap? There are over 200,000 cars a day driving at that stretch of highway, if a car crashes there once a week, then the Tesla Autopilot is an order of magnitude more dangerous than a human driver.

greglindahl 9y ago
How is Tesla supposed to know, yet, if Autopilot actually made the maneuver? They're saying what they know. Edit: in a posting today, Tesla has said more: https://www.tesla.com/blog/update-last-week’s-accident https://www.tesla.com/blog/update-last-week’s-accident but at the time of the earlier blog posting, Tesla says they did not know yet.
- dingo_bat 9y ago
  They definitely know. They collect your driving data even when auto pilot is off, so I'd imagine they collect everything when the autopilot is actually engaged.
  - greglindahl 9y ago
    Tesla specifically said that the information was in the car and wasn't transmitted to HQ, so they didn't know. If what you say is true, they're lying to the USG. Let me share an example. I had an accident in a Model S, front-end, which didn't cause a loss of power. Tesla called me a couple of minutes later, saying that they knew I'd had an airbag-deploying accident, and asking if I needed an ambulance. From the description of this accident, power to the Tesla was lost before there was time to phone home.
- brfox 9y ago
  From their blog: "In the US, there is one automotive fatality every 86 million miles across all vehicles from all manufacturers. For Tesla, there is one fatality, including known pedestrian fatalities, every 320 million miles in vehicles equipped with Autopilot hardware. If you are driving a Tesla equipped with Autopilot hardware, you are 3.7 times less likely to be involved in a fatal accident." Notice it says one fatality per "320 million miles in vehicles equipped with Autopilot hardware." But, how many of those miles (or what percent of the time) does a Tesla driver use Autopilot? Also, maybe those good stats are due to the structural safety features and not the autopilot. It may be more fair to compare to other cars with excellent crash protection but no autopilot. I think it would be a great service to the world to improve driving safety, but maybe we need to really start looking at the stats and get some more hubris as we transition to full autopilot. For example, require that drivers keep their hands on the wheel and eyes on the road. When more cars have autopilot, then mesh behavior between vehicles, and e-road features, then maybe we'll be more ready for driver-less cars.

skybrian 9y ago

I don't see what's wrong with what they wrote. This statement of course doesn't show that the software is safe, but it is evidence that if there is a bug, it probably isn't easily triggered. (Keep in mind that until they recover the logs, we don't even know if the autopilot was on.)

rrdharan 9y ago
They have confirmed the autopilot was on: https://www.tesla.com/blog/update-last-week’s-accident https://www.tesla.com/blog/update-last-week’s-accident
- DrScump 9y ago
  But does "autopilot on" mean that "autopilot has exclusive control and driver inputs are ignored"?
  - pacificmint 9y ago
    No, of course driver inputs will not be ignored. They will override the autopilot. But none were given. From the blog: >In the moments before the collision, which occurred at 9:27 a.m. on Friday, March 23rd, Autopilot was engaged with the adaptive cruise control follow-distance set to minimum. The driver had received several visual and one audible hands-on warning earlier in the drive and the driver’s hands were not detected on the wheel for six seconds prior to the collision. The driver had about five seconds and 150 meters of unobstructed view of the concrete divider with the crushed crash attenuator, but the vehicle logs show that no action was taken.
    - lazyjones 9y ago
      > The driver had about five seconds and 150 meters of unobstructed view of the concrete divider with the crushed crash attenuator So, clearly, the AP is not capable of avoiding such obstacles yet, even when unobstructed. It also seems like a mistake to not slow down to a survivable speed when hands are off...

Animats 9y ago

Misaligned sensor?

sschueller 9y ago
In such a case there needs to be a fail safe that can determine that and disable auto pilot until it is fixed.

mirimir 9y ago

It's possible that some of those "200 successful Autopilot trips per day on this exact stretch of road" also involve pulling toward that barrier. However, it's not likely a huge percentage, or we'd have seen that posted. So the implication is that his Tesla was relatively unique. That is, broken.

lafar6502 9y ago

software guys should know better that programs will have bugs...

dengnan 9y ago

Let's play with statistics: Scenario 1: 100 human drivers drive some distance. 2 bad drivers caused 150 accidents and both died in their last fatal accidents. Scenario 2: 100 self-driving/autopilot-enabled cars drive the same road for the same distance. Everyone has one accident and one of them is fatal. Statistical numbers: Self-driving/autopilot-enabled cars cut both the accident rate and death rate by half. Death rate per accident is reduced from 2/150 to 1/100. Conclusion: self-driving/autopilot is better.

deleted 9y ago
[deleted]
nikanj 9y ago
Human accidents, however, are not news. "Tesla also posted these photos that raise another important question: they show what's called a "crash attenuator" or safety barrier in the proper condition ... and the way it was the day before Walter Huang's crash ... collapsed after a different accident." Did you see any news coverage of the previous accident? I know I didn't.
CPLX 9y ago
If we're making up statistics out of thin air can we do a couple with people riding ponies too?

z3t4 9y ago

When users discover a bug or issue they tend to work around it. Even as a developer for example you are debugging something and stumbling on another unrelated issue you are likely to ignore it. Users very rarely report issues. Self driving cars testers should be paid per issue found, not per hour.

merb 9y ago

Well I found the comment of tesla, odd as well. but it is also odd that the driver still kept driving the same road WITH autopilot activated, besides he KNEW it was broken. It's a strange accident, especially because of the strangeness of tesla and the whole background story (broken safety barrier). I'm looking forward to the outcome of the whole story. At the end I guess more than one party could've been a bad actor. Tesla, The guy itself and even that the road had no kind of warning (safety barrier not replaced or some kind of other warning sinces that it will be replaced soonish)

Qwertious 9y ago

> The Tesla statement is the equivalent of "works in my laptop" at a bigger scale. Erhem: https://i.imgur.com/sY3Eeln.png https://i.imgur.com/sY3Eeln.png

bryanrasmussen 9y ago

perhaps whoever wrote this statement does understand software but they also understand lying with numbers and knows the general public understands neither.