9 ms·

It's really quite sad that the regulation of autonomous vehicles been so slow to come along. Public roads are filled with other drivers, passengers, and pedestr

by benzor 5y ago

It's really quite sad that the regulation of autonomous vehicles been so slow to come along. Public roads are filled with other drivers, passengers, and pedestrians that did not consent to be a part of a large scale beta test for partial driving automation that could fail at any time. I believe this is a case where self driving software should be default illegal until proven safe. Most companies in this industry, thankfully, seem to be moving carefully and rolling out their products conservatively; Tesla seems to think "move fast and break things" is an appropriate motto for 5000 lbs projectiles on public roads.

bob33212 5y ago
If every driver on the road today was never drunk/distracted/enraged I would agree, but the reality is that humans driving cars kill other people every single day. We should fix this with a better system. Tesla and Waymo seem to be making progress. I don't expect them to be perfect but in the long run this will save lives.
- Swenrekcah 5y ago
  If FSD is accurately described as “not always worse than a drunk driver” then please remove it from the roads.
- wpietri 5y ago
  You seem to be implying that in the short run it's ok for them to kill some extra people. One, I don't think that's necessary; Waymo is a good example of how a safer approach is also apparently no less effective. Two, you're presuming that we will get to self-driving cars that are economically viable and safer than current human-driven ones, something that is not a given. And three, it's not clear to me who gets to decide exactly how many unwilling people should be sacrificed on the altar of technological progress, but I hope it's not us and it sure shouldn't be Musk.
  - kjksf 5y ago
    https://www.youtube.com/watch?v=zdKCQKBvH-A https://www.youtube.com/watch?v=zdKCQKBvH-A Here's a video of Waymo car in Phoenix. Around min. 17 it gets confused by a cone and stops in the middle of two lane road, hugging both lanes. Then google sends support but just when they arrive, the car drives away from them. They were lucky it didn't lead to an accident and as long as they keep the fleet to 600 cars then yeah, the accident rate will be much lower than Tesla's AutoPilot, shipped in more than a million cars. My point is not to rag on Waymo just to inject some reality. We don't have a choice between "safe and unsafe" way of developing self-driving software. We have a choice between "test software we know can't handle all situation on real roads and make it better based on that testing" or "we'll never have self-driving software". Except the other option will rather be: "U.S. doesn't allow testing of self-driving software on real roads and a Chinese company will develop it and will capture a trillion-dollar market in U.S."
    - heavyset_go 5y ago
      > We have a choice between "test software we know can't handle all situation on real roads and make it better based on that testing" or "we'll never have self-driving software". This is like saying that we'll never have a cure for cancer if we can't experiment on the public without their consent. The bar for medication, at least, is proving safety first before testing on large amounts of people and allowing the public to buy it.
      - ghoward 5y ago
        > The bar for medication, at least, is proving safety first before testing on large amounts of people and allowing the public to buy it. Except for vaccines, apparently...
        deleted 5y ago
        [deleted]
    - dataflow 5y ago
      > We don't have a choice between "safe and unsafe" way of developing self-driving software. It's also entirely possible Waymo hasn't achieved peak perfection in its software development practices despite doing better than Tesla, and that another entity could do it more safely.
  - cbozeman 5y ago
    How many unwilling people are sacrificed because we won't ban alcohol and all impairing drugs without exception and then enforce those bans with immediate, Judge Dredd-style summary execution? I bet you that'd reduce traffic fatalities dramatically too. How far do you want to go to 'save lives'. I guarantee you with 100% certainty I can design a society that will 'save lives' at every turn for every single activity, and I can guarantee you with 100% certainty you wouldn't want to live in it.
    - wpietri 5y ago
      You seem to ignore that we've already tried banning alcohol and currently ban many drugs. We scrapped banning alcohol for a reason.
  - fitzie 5y ago
    you are arguing a fake hypothetical. Tesla accident data shows that their autopilot feature reduces accidents. https://www.tesla.com/VehicleSafetyReport https://www.tesla.com/VehicleSafetyReport fortunately our regulators don't take a one size fits all approach for how new technologies can be developed
    - wpietri 5y ago
      Your notion is that everything is fine because according to Tesla, a company with a leader known for telling whoppers, they are killing less people on net? Even if we trust them on that stat, which I certainly don't, that still doesn't mean they aren't killing people unnecessarily.
    - avereveard 5y ago
      It doesn't, as it compare general rates with self selected "good driving conditions" as defined by the software - only highways, only good enough weather, only good enough maintenance state of the car.
- steelframe 5y ago
  I can't withdraw my consent to share the road with intoxicated or distracted drivers. That's just a fact of life. That doesn't mean I shouldn't be able to withdraw my consent to share the road with Beta software. Plenty of businesses are able to build very effective safety mechanisms for motor vehicles without subjecting the general public to Beta software. To me this is a case where the ends do not justify the means.
  - davrosthedalek 5y ago
    I understand your point, but I would frame it stronger: Indeed, we, as a community, have withdrawn our consent to share the road with intoxicated drivers. You break laws if you drive intoxicated. That beta software doesn't break any laws, but maybe it should.
- rantwasp 5y ago
  nah. just because some drivers do stupid things does not mean that we should allow this "FSD" travesty. The thing is: if you're a human driver and you screw up you pay the consequences. It's codified in the law and you are fully aware that there are consequences if you don't follow the rules. Now, if you drive one of these "FDS" cars and it ends up in an accident where people die, who is responsible? Are you responsible? Is the car manufacturer responsible? Do we just write it off as a freak accident with 1 in a million chance of happening again?
  - mgarfias 5y ago
    FSD != auto pilot
    - rantwasp 5y ago
      yes of course. fsd is only 3 letters but IMHO they’re both lies.
- HWR_14 5y ago
  > We should fix this with a better system. What if we made driving tests far more difficult (otherwise, you can drive a 50(?)CC moped that tops out at 35mph)? What if we had government subsidize rides home after going out drinking? There are a lot of solutions that don't involve "let companies test 5000lbs autonomous vehicles on public roads." Heck, those companies have enough money let them buy a city and test it on the now private streets. > Tesla and Waymo seem to be making progress. I don't expect them to be perfect but in the long run this will save lives. Let's assume for a second your unstated assumptions I was alluding to above are correct. This is the only way to make a better system and that system has to be tested on public roads. Why do we allow two private companies to reap the inevitable huge monetary rewards when we all pay for that system with a higher increased risk of dying in the meantime?
- deleted 5y ago
  [deleted]
iaw 5y ago
Innovation always precedes regulation/safety. Look at the early automobile and the road marker[0] [0] https://en.wikipedia.org/wiki/Road_surface_marking https://en.wikipedia.org/wiki/Road_surface_marking
- benzor 5y ago
  I agree with your point that government regulation is fundamentally reactive to private sector innovation, hence lagging. That being said, this particular issue of autonomous driving has been a hot topic for the better part of a decade now and I would like our governments to tackle it.
wpietri 5y ago
I partly agree, but it's also sad that we need to add to the bureaucracy just to insist that professionals act professionally. As you point out, most companies are doing fine. But it wasn't just Tesla playing fast and loose; as far as I'm concerned Uber execs should be doing time for negligent homicide: https://www.npr.org/2019/11/07/777438412/feds-say-self-driving-uber-suv-did-not-recognize-jaywalking-pedestrian-in-fatal- https://www.npr.org/2019/11/07/777438412/feds-say-self-drivi...
- studentrob 5y ago
  Tesla is the big fish here. If Uber has one instance, then given the size of their program, Tesla may have hundreds.
  - AKSucks 5y ago
    Thing is, it wasn't just one incident, it was just one incident that resulted in a death. When Ubers started self-driving it took just a few hours before there were videos on twitter and youtube of them driving right though red lights without a care in the world.
    - studentrob 5y ago
      Uber: 1 death and halted the program Tesla: many deaths, no halt, no improvement in the program. See: phantom braking
      - wpietri 5y ago
        Let's not give them too much credit here. I think Uber's halt had more to do with Uber's change in CEO and the indictment of the guy who ran their self-driving car program. Plus the fact that it was a giant money sink with no short-term return being run in a company that has never been profitable and can no longer raise infinite investor money.
        studentrob 5y ago
abledon 5y ago
no one in government is qualified to really understand ML and to regulate that stuff. Does the government employ people at the level of Tesla in ML? Its quite the problem... when the best minds are in the tech enclaves are not working for the public interest.
- benzor 5y ago
  We don't need any understanding of AI/ML in government to effectively regulate the autonomous vehicle industry. Design an appropriate set of tests, make companies run the gauntlet, only approve the ones that pass. The test criteria is simple: does this software meaningfully and statistically significantly reduce the risk of accident/harm/death compared to the average human driver? Add caveats and conditionals as you wish for conditions/weather etc. but it's fundamentally a black box test with no knowledge of technical internals required.
  - kjksf 5y ago
    > statistically significantly reduce And how does one prove something is true "statistically"? By doing it many times. So many times that you can be statistically confident of the result. Statistically, there is ~1.4 deaths per 100 MILLION miles driven. To prove, statistically, that software is as good, it would have to drive at least 1 billion miles. And yes, it would kill 14 people in the process (or more, if it's worse than humans; or less, if its better). Tesla is already doing what you say car companies should be doing i.e. statistically testing the software Except you don't like it and think there's a magic fairy test that will show something "statistically" without driving statistically significant number of miles.
slg 5y ago
>I believe this is a case where self driving software should be default illegal until proven safe. If we go with your suggestion, what would you consider "proven safe"? Autopilot has been running in hundreds of thousands of cars, has driven several billion miles, and we still don't have enough data to prove whether it is safer or more dangerous then an average human driver. Accidents are rare and we therefore need a huge amount of data before we can be confident that the accident rates we are seeing are predictive. I have no idea how you collect that type of data without them being tested on real roads with other real drivers.
- dragontamer 5y ago
  How about actually passing some basic tests before they're deployed? (Warning: unnecessarily loud video for some reason): https://twitter.com/finance_degen/status/1307529357951467531 https://twitter.com/finance_degen/status/1307529357951467531 Lets start with passing these basic tests before selling them as working features. I don't think we're asking for a very high bar.
  - slg 5y ago
    There is zero context on that video that tells us what is happening there. We have no idea if that is the Autopilot failing or the emergency braking failing. We also have no control group to tell us what percentage of humans would stop short of that dummy. It is inexact due to Twitter's video player not showing fractions of a second, but it looks like there were approximately 2 seconds between when the dummy started moving forward and when it was hit by the car. The average human time to braking is 2.2-2.3 seconds[1]. Is the car even failing that test in comparison to a human? It also isn't clear from watching that video what the safest and therefore desired behavior should be in that situation. A self driving car is obviously not going to prevent all accidents, so it is a question of minimizing potential harm. We don't want a car to aggressively brake whenever someone at a street corner takes a step towards the road. We therefore need to balance the chance of a person stepping into the path of the car with the risk of braking when it is unnecessary and causing a rear end collision. The problem in the linked thread is overaggressive braking so forcing the car to pass a test that rewards overaggressive braking would only make that specific example worse. That leads back to my point about needing a huge amount of data. You can't just run a car through an obstacle course to know whether it is safer or more dangerous than a human. You need to have it interacting with unpredictable humans and you need to do it repeatedly before you can confidentially predict whether it is safer or more dangerous than a human. [1] - https://copradar.com/redlight/factors/IEA2000_ABS51.pdf https://copradar.com/redlight/factors/IEA2000_ABS51.pdf
jsight 5y ago
> Most companies in this industry, thankfully, seem to be moving carefully and rolling out their products conservatively; Having used other products, I think this is objectively not true. I've seen "Pro" pilot accelerate itself into its own collision warning and randomly fail at basic curves. I've seen video of Ford Copilot failing due to glare and jerking toward trees. For some reason, Tesla just gets more attention. Some of the other beta-like behaviors are actually worse.
- AlexandrB 5y ago
  Tesla gets more attention because their CEO keeps promising the world[1] and selling these features as "Full Self Driving" instead of "Copilot". [1] https://mobile.twitter.com/elonmusk/status/686279251293777920 https://mobile.twitter.com/elonmusk/status/68627925129377792...
  - jsight 5y ago
    The others claim that you can drive without your hands on the wheel, despite the fact that the time required for takeover can be milliseconds. I'm really not convinced that is objectively better, even with whitelisted roads.
    - beardyw 5y ago
      Quite a lot of milliseconds in fact. For a visual stimulus it is 1/4 of a second minimum. That's 22 feet at 50 mph.
    - amanaplanacanal 5y ago
      I haven’t seen this, but I’m out of the loop. Which ones are claiming this?
      - jsight 5y ago
        GM SuperCruise right now, and Ford is claiming that their "Blue Cruise" product will be able to do it soon. There may be others, but those are the two that I'm familiar with. I've also been observing a general trend of reviewers rating the performance of new vehicles based in part on how long it can go without nagging the user to hold the wheel. sigh
TearsInTheRain 5y ago
I think Tesla should definitely be accountable for their software and damage that it causes but involving overly risk adverse regulations will push out self driving cars by a decade, maybe indefinitely.
Reason077 5y ago
> ”I believe this is a case where self driving software should be default illegal until proven safe.” You have to remember that human drivers are extremely dangerous, and cause millions of deaths and serious, life-changing injuries worldwide every year. Any regulations that hold back the development of self-driving software would likely be counter-productive. It’s a bit like arguing that we should have held back Covid-19 vaccines because a small number of people had blood clots or heart inflammation.
Boritanian 5y ago
>where self driving software should be default illegal until proven safe. Would you apply the same standard to human drivers?
- papaf 5y ago
  The same standard is applied to human drivers. I think it is called a "driving test".
- benhurmarcel 5y ago
  Yes. It's called a driving license, in many countries it's only delivered after you show you can drive safely. On a more similar note, that's what we apply to aircraft. They are illegal to fly until proven safe to the authorities (ie. certified).
jacquesm 5y ago
It's quite simple, really: any software that pretends that it can be in control of a car should be subject to the same kind of test that ordinary drivers have to do before being allowed to take to the road. Then, the manufacturer should assume all liability for errors made by their product. Just like a real person would. They can choose to insure or self insure.
- Gys 5y ago
  Those successfully tested ordinary drivers also kill many people every year. In 2019 in the US alone more than 35.000! https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in_U.S._by_year https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...
  - jacquesm 5y ago
    Under all circumstances, over all of the miles driven. Tesla is very good at spinning things but in an apples-to-apples comparison they are doing much worse than those ordinary drivers.
    - Isinlor 5y ago
      Where do you have apples-to-apples comparison? As far as I know, this does not exist. NHTSA is investigating Tesla, some 20-30 accidents. If they find that Tesla is doing something seriously dangerous, I'm sure they will force Tesla to take corrective actions. The same applies to European regulators.
      - jacquesm 5y ago
        > Where do you have apples-to-apples comparison? https://www.forbes.com/sites/bradtempleton/2020/07/28/teslas-arent-safer-on-autopilot-so-researchers-calling-for-driver-monitoring-may-be-right/ https://www.forbes.com/sites/bradtempleton/2020/07/28/teslas...
        Isinlor 5y ago
fitzie 5y ago
it is a bit scary to think we'd have to lobby government to allow any new invention to prove "safety". Tesla has indicated why it needs to enable these features in order to collect data to improve them, it has also shown that driving under autopilot are already reducing accidents (compared to without and national stats for all vehicles), and yet you call for a entire ban instead of thinking constructively. with the vision stack there will be improvements to the driver attentiveness checks. that would seem to mitigate abuse of the features, which is clearly meant to be supervised at all times by the operator.

by benzor 5y ago

bob33212 5y ago

If every driver on the road today was never drunk/distracted/enraged I would agree, but the reality is that humans driving cars kill other people every single day. We should fix this with a better system. Tesla and Waymo seem to be making progress. I don't expect them to be perfect but in the long run this will save lives.

Swenrekcah 5y ago
If FSD is accurately described as “not always worse than a drunk driver” then please remove it from the roads.
wpietri 5y ago
You seem to be implying that in the short run it's ok for them to kill some extra people. One, I don't think that's necessary; Waymo is a good example of how a safer approach is also apparently no less effective. Two, you're presuming that we will get to self-driving cars that are economically viable and safer than current human-driven ones, something that is not a given. And three, it's not clear to me who gets to decide exactly how many unwilling people should be sacrificed on the altar of technological progress, but I hope it's not us and it sure shouldn't be Musk.
- kjksf 5y ago
  https://www.youtube.com/watch?v=zdKCQKBvH-A https://www.youtube.com/watch?v=zdKCQKBvH-A Here's a video of Waymo car in Phoenix. Around min. 17 it gets confused by a cone and stops in the middle of two lane road, hugging both lanes. Then google sends support but just when they arrive, the car drives away from them. They were lucky it didn't lead to an accident and as long as they keep the fleet to 600 cars then yeah, the accident rate will be much lower than Tesla's AutoPilot, shipped in more than a million cars. My point is not to rag on Waymo just to inject some reality. We don't have a choice between "safe and unsafe" way of developing self-driving software. We have a choice between "test software we know can't handle all situation on real roads and make it better based on that testing" or "we'll never have self-driving software". Except the other option will rather be: "U.S. doesn't allow testing of self-driving software on real roads and a Chinese company will develop it and will capture a trillion-dollar market in U.S."
  - heavyset_go 5y ago
    > We have a choice between "test software we know can't handle all situation on real roads and make it better based on that testing" or "we'll never have self-driving software". This is like saying that we'll never have a cure for cancer if we can't experiment on the public without their consent. The bar for medication, at least, is proving safety first before testing on large amounts of people and allowing the public to buy it.
    - ghoward 5y ago
      > The bar for medication, at least, is proving safety first before testing on large amounts of people and allowing the public to buy it. Except for vaccines, apparently...
      - deleted 5y ago
        [deleted]
  - dataflow 5y ago
    > We don't have a choice between "safe and unsafe" way of developing self-driving software. It's also entirely possible Waymo hasn't achieved peak perfection in its software development practices despite doing better than Tesla, and that another entity could do it more safely.
- cbozeman 5y ago
  How many unwilling people are sacrificed because we won't ban alcohol and all impairing drugs without exception and then enforce those bans with immediate, Judge Dredd-style summary execution? I bet you that'd reduce traffic fatalities dramatically too. How far do you want to go to 'save lives'. I guarantee you with 100% certainty I can design a society that will 'save lives' at every turn for every single activity, and I can guarantee you with 100% certainty you wouldn't want to live in it.
  - wpietri 5y ago
    You seem to ignore that we've already tried banning alcohol and currently ban many drugs. We scrapped banning alcohol for a reason.
- fitzie 5y ago
  you are arguing a fake hypothetical. Tesla accident data shows that their autopilot feature reduces accidents. https://www.tesla.com/VehicleSafetyReport https://www.tesla.com/VehicleSafetyReport fortunately our regulators don't take a one size fits all approach for how new technologies can be developed
  - wpietri 5y ago
    Your notion is that everything is fine because according to Tesla, a company with a leader known for telling whoppers, they are killing less people on net? Even if we trust them on that stat, which I certainly don't, that still doesn't mean they aren't killing people unnecessarily.
  - avereveard 5y ago
    It doesn't, as it compare general rates with self selected "good driving conditions" as defined by the software - only highways, only good enough weather, only good enough maintenance state of the car.
steelframe 5y ago
I can't withdraw my consent to share the road with intoxicated or distracted drivers. That's just a fact of life. That doesn't mean I shouldn't be able to withdraw my consent to share the road with Beta software. Plenty of businesses are able to build very effective safety mechanisms for motor vehicles without subjecting the general public to Beta software. To me this is a case where the ends do not justify the means.
- davrosthedalek 5y ago
  I understand your point, but I would frame it stronger: Indeed, we, as a community, have withdrawn our consent to share the road with intoxicated drivers. You break laws if you drive intoxicated. That beta software doesn't break any laws, but maybe it should.
rantwasp 5y ago
nah. just because some drivers do stupid things does not mean that we should allow this "FSD" travesty. The thing is: if you're a human driver and you screw up you pay the consequences. It's codified in the law and you are fully aware that there are consequences if you don't follow the rules. Now, if you drive one of these "FDS" cars and it ends up in an accident where people die, who is responsible? Are you responsible? Is the car manufacturer responsible? Do we just write it off as a freak accident with 1 in a million chance of happening again?
- mgarfias 5y ago
  FSD != auto pilot
  - rantwasp 5y ago
    yes of course. fsd is only 3 letters but IMHO they’re both lies.
HWR_14 5y ago
> We should fix this with a better system. What if we made driving tests far more difficult (otherwise, you can drive a 50(?)CC moped that tops out at 35mph)? What if we had government subsidize rides home after going out drinking? There are a lot of solutions that don't involve "let companies test 5000lbs autonomous vehicles on public roads." Heck, those companies have enough money let them buy a city and test it on the now private streets. > Tesla and Waymo seem to be making progress. I don't expect them to be perfect but in the long run this will save lives. Let's assume for a second your unstated assumptions I was alluding to above are correct. This is the only way to make a better system and that system has to be tested on public roads. Why do we allow two private companies to reap the inevitable huge monetary rewards when we all pay for that system with a higher increased risk of dying in the meantime?
deleted 5y ago
[deleted]

iaw 5y ago

Innovation always precedes regulation/safety. Look at the early automobile and the road marker[0] [0] https://en.wikipedia.org/wiki/Road_surface_marking https://en.wikipedia.org/wiki/Road_surface_marking

benzor 5y ago
I agree with your point that government regulation is fundamentally reactive to private sector innovation, hence lagging. That being said, this particular issue of autonomous driving has been a hot topic for the better part of a decade now and I would like our governments to tackle it.

wpietri 5y ago

I partly agree, but it's also sad that we need to add to the bureaucracy just to insist that professionals act professionally. As you point out, most companies are doing fine. But it wasn't just Tesla playing fast and loose; as far as I'm concerned Uber execs should be doing time for negligent homicide: https://www.npr.org/2019/11/07/777438412/feds-say-self-driving-uber-suv-did-not-recognize-jaywalking-pedestrian-in-fatal- https://www.npr.org/2019/11/07/777438412/feds-say-self-drivi...

studentrob 5y ago
Tesla is the big fish here. If Uber has one instance, then given the size of their program, Tesla may have hundreds.
- AKSucks 5y ago
  Thing is, it wasn't just one incident, it was just one incident that resulted in a death. When Ubers started self-driving it took just a few hours before there were videos on twitter and youtube of them driving right though red lights without a care in the world.
  - studentrob 5y ago
    Uber: 1 death and halted the program Tesla: many deaths, no halt, no improvement in the program. See: phantom braking
    - wpietri 5y ago
      Let's not give them too much credit here. I think Uber's halt had more to do with Uber's change in CEO and the indictment of the guy who ran their self-driving car program. Plus the fact that it was a giant money sink with no short-term return being run in a company that has never been profitable and can no longer raise infinite investor money.
      - studentrob 5y ago

abledon 5y ago

no one in government is qualified to really understand ML and to regulate that stuff. Does the government employ people at the level of Tesla in ML? Its quite the problem... when the best minds are in the tech enclaves are not working for the public interest.

benzor 5y ago
We don't need any understanding of AI/ML in government to effectively regulate the autonomous vehicle industry. Design an appropriate set of tests, make companies run the gauntlet, only approve the ones that pass. The test criteria is simple: does this software meaningfully and statistically significantly reduce the risk of accident/harm/death compared to the average human driver? Add caveats and conditionals as you wish for conditions/weather etc. but it's fundamentally a black box test with no knowledge of technical internals required.
- kjksf 5y ago
  > statistically significantly reduce And how does one prove something is true "statistically"? By doing it many times. So many times that you can be statistically confident of the result. Statistically, there is ~1.4 deaths per 100 MILLION miles driven. To prove, statistically, that software is as good, it would have to drive at least 1 billion miles. And yes, it would kill 14 people in the process (or more, if it's worse than humans; or less, if its better). Tesla is already doing what you say car companies should be doing i.e. statistically testing the software Except you don't like it and think there's a magic fairy test that will show something "statistically" without driving statistically significant number of miles.

slg 5y ago

>I believe this is a case where self driving software should be default illegal until proven safe. If we go with your suggestion, what would you consider "proven safe"? Autopilot has been running in hundreds of thousands of cars, has driven several billion miles, and we still don't have enough data to prove whether it is safer or more dangerous then an average human driver. Accidents are rare and we therefore need a huge amount of data before we can be confident that the accident rates we are seeing are predictive. I have no idea how you collect that type of data without them being tested on real roads with other real drivers.

dragontamer 5y ago
How about actually passing some basic tests before they're deployed? (Warning: unnecessarily loud video for some reason): https://twitter.com/finance_degen/status/1307529357951467531 https://twitter.com/finance_degen/status/1307529357951467531 Lets start with passing these basic tests before selling them as working features. I don't think we're asking for a very high bar.
- slg 5y ago
  There is zero context on that video that tells us what is happening there. We have no idea if that is the Autopilot failing or the emergency braking failing. We also have no control group to tell us what percentage of humans would stop short of that dummy. It is inexact due to Twitter's video player not showing fractions of a second, but it looks like there were approximately 2 seconds between when the dummy started moving forward and when it was hit by the car. The average human time to braking is 2.2-2.3 seconds[1]. Is the car even failing that test in comparison to a human? It also isn't clear from watching that video what the safest and therefore desired behavior should be in that situation. A self driving car is obviously not going to prevent all accidents, so it is a question of minimizing potential harm. We don't want a car to aggressively brake whenever someone at a street corner takes a step towards the road. We therefore need to balance the chance of a person stepping into the path of the car with the risk of braking when it is unnecessary and causing a rear end collision. The problem in the linked thread is overaggressive braking so forcing the car to pass a test that rewards overaggressive braking would only make that specific example worse. That leads back to my point about needing a huge amount of data. You can't just run a car through an obstacle course to know whether it is safer or more dangerous than a human. You need to have it interacting with unpredictable humans and you need to do it repeatedly before you can confidentially predict whether it is safer or more dangerous than a human. [1] - https://copradar.com/redlight/factors/IEA2000_ABS51.pdf https://copradar.com/redlight/factors/IEA2000_ABS51.pdf

jsight 5y ago

> Most companies in this industry, thankfully, seem to be moving carefully and rolling out their products conservatively; Having used other products, I think this is objectively not true. I've seen "Pro" pilot accelerate itself into its own collision warning and randomly fail at basic curves. I've seen video of Ford Copilot failing due to glare and jerking toward trees. For some reason, Tesla just gets more attention. Some of the other beta-like behaviors are actually worse.

AlexandrB 5y ago
Tesla gets more attention because their CEO keeps promising the world[1] and selling these features as "Full Self Driving" instead of "Copilot". [1] https://mobile.twitter.com/elonmusk/status/686279251293777920 https://mobile.twitter.com/elonmusk/status/68627925129377792...
- jsight 5y ago
  The others claim that you can drive without your hands on the wheel, despite the fact that the time required for takeover can be milliseconds. I'm really not convinced that is objectively better, even with whitelisted roads.
  - beardyw 5y ago
    Quite a lot of milliseconds in fact. For a visual stimulus it is 1/4 of a second minimum. That's 22 feet at 50 mph.
  - amanaplanacanal 5y ago
    I haven’t seen this, but I’m out of the loop. Which ones are claiming this?
    - jsight 5y ago
      GM SuperCruise right now, and Ford is claiming that their "Blue Cruise" product will be able to do it soon. There may be others, but those are the two that I'm familiar with. I've also been observing a general trend of reviewers rating the performance of new vehicles based in part on how long it can go without nagging the user to hold the wheel. sigh

TearsInTheRain 5y ago

I think Tesla should definitely be accountable for their software and damage that it causes but involving overly risk adverse regulations will push out self driving cars by a decade, maybe indefinitely.

Reason077 5y ago

> ”I believe this is a case where self driving software should be default illegal until proven safe.” You have to remember that human drivers are extremely dangerous, and cause millions of deaths and serious, life-changing injuries worldwide every year. Any regulations that hold back the development of self-driving software would likely be counter-productive. It’s a bit like arguing that we should have held back Covid-19 vaccines because a small number of people had blood clots or heart inflammation.

Boritanian 5y ago

>where self driving software should be default illegal until proven safe. Would you apply the same standard to human drivers?

papaf 5y ago
The same standard is applied to human drivers. I think it is called a "driving test".
benhurmarcel 5y ago
Yes. It's called a driving license, in many countries it's only delivered after you show you can drive safely. On a more similar note, that's what we apply to aircraft. They are illegal to fly until proven safe to the authorities (ie. certified).

jacquesm 5y ago

It's quite simple, really: any software that pretends that it can be in control of a car should be subject to the same kind of test that ordinary drivers have to do before being allowed to take to the road. Then, the manufacturer should assume all liability for errors made by their product. Just like a real person would. They can choose to insure or self insure.

Gys 5y ago
Those successfully tested ordinary drivers also kill many people every year. In 2019 in the US alone more than 35.000! https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in_U.S._by_year https://en.wikipedia.org/wiki/Motor_vehicle_fatality_rate_in...
- jacquesm 5y ago
  Under all circumstances, over all of the miles driven. Tesla is very good at spinning things but in an apples-to-apples comparison they are doing much worse than those ordinary drivers.
  - Isinlor 5y ago
    Where do you have apples-to-apples comparison? As far as I know, this does not exist. NHTSA is investigating Tesla, some 20-30 accidents. If they find that Tesla is doing something seriously dangerous, I'm sure they will force Tesla to take corrective actions. The same applies to European regulators.
    - jacquesm 5y ago
      > Where do you have apples-to-apples comparison? https://www.forbes.com/sites/bradtempleton/2020/07/28/teslas-arent-safer-on-autopilot-so-researchers-calling-for-driver-monitoring-may-be-right/ https://www.forbes.com/sites/bradtempleton/2020/07/28/teslas...
      - Isinlor 5y ago

fitzie 5y ago

it is a bit scary to think we'd have to lobby government to allow any new invention to prove "safety". Tesla has indicated why it needs to enable these features in order to collect data to improve them, it has also shown that driving under autopilot are already reducing accidents (compared to without and national stats for all vehicles), and yet you call for a entire ban instead of thinking constructively. with the vision stack there will be improvements to the driver attentiveness checks. that would seem to mitigate abuse of the features, which is clearly meant to be supervised at all times by the operator.