6 ms·

"As you are undoubtedly aware, there is a high likelihood that some drivers will use your product in a manner that exceeds its intended purpose." The governmen

by abduhl 10y ago

"As you are undoubtedly aware, there is a high likelihood that some drivers will use your product in a manner that exceeds its intended purpose." The government appears well aware of this hand waving garbage that companies are trying with "it isn't really autopilot, keep your hands on the wheel wink." This is exactly government's job and is a fabulous example of a government reacting to ongoing events that have proven that so-called autopilot systems are inherently a danger to the public at this stage.

erikpukinskis 10y ago
Proof that so-called autopilot systems are a danger to the public would be evidence that they perform worse than the average driver. I haven't seen such evidence, what are you referencing?
- TeMPOraL 10y ago
  It's the other way around - you need a proof that they perform comparable to an average driver during expected use, and they do not have absurdly stupid corner cases during actually expected use.
  - lutorm 10y ago
    Well, humans have absurdly stupid corner cases during actual use, too. It depends on how small the corner is, I think.
    - thfuran 10y ago
      Yes and humans require certification before they are permitted to drive vehicles on public roadways.
      - strathmeyer 10y ago
        To get that certification, do they test you first, or do they wait for you to screw up and then take it away from you??
        azinman2 10y ago
        We know, being humans, and that we have the ability to process lots of complex information in a way that's very difficult for computers to replicate. Hard AI doesn't actually exist (yet). We also have a 100 years of humans driving cars worldwide so we understand well what they're good at and what they're not, so laws & safety designs take all of this into consideration. Each computer system will be encountering new, diverse things in the real world without a good understanding of how they'll perform. There are lots of crazy hard problems here that no one has solved yet. So to suggest we just automatically trust it because humans make mistakes is foolish when the consequences are so high. If someone came out with a surgery "autopilot" tomorrow, would you suggest it start giving triple bypass surgeries right away without FDA approval because humans make errors too?
        TeMPOraL 10y ago
        One of the features of the common human firmware is self-preservation instinct. It lets us trust that our fellow drivers, while still prone to mistakes, won't generally make obviously suicidal errors. Can one say the same about a new ML algorithm running on some board designed half a decade ago? How exactly would one know, without a thorough audit?
        dragonwriter 10y ago
        They test you first, but if despite passing the test you screw up sufficiently badly later they decertify you. It's a belt-and-suspenders system.
    - TeMPOraL 10y ago
      We've been dealing with those corner cases for thousands of years and we know them pretty well. Given that we all run on pretty much the same hardware and firmware (with minor differences), you could say humans have been thoroughly tested for few millennia, including a hundred or so years of road testing. So yeah, I a bit of thorough testing of a completely new hardware running completely new software isn't too much to ask for.
- deleted 10y ago
  [deleted]
- digler999 10y ago
  so we should wait till a statistically-significant percentage of people die before we tell everyone, "yep, definitely dangerous. Lets add some regulation" ? I wonder if they could shut him down on the basis of car insurance requirements and the legal definition of "driver". If "driver" is defined as the "person" inputting control commands, then either you dont consider the computer a person, and then nobody is legally "driving" the car, or else the software somehow meets the definition of a "person" or driver, in which case its an unlicensed driver operating the vehicle and insurance may not have to cover it.
  - smallnamespace 10y ago
    The NHTSA has been a strong proponent of self-driving technology and has gone out of its way to make sure there is a path to doing it legally, without pulling the legal 'gotcha' that you described here. The device would probably be SAE Level 2, and should be regulated as such.
- saosebastiao 10y ago
  When you measure the safety of autonomous cars by counting the miles they travel, and not counting the miles that they intentionally avoid or defer to a human, you have succeeded in creating a metric that looks like it's useful for comparison, but is actually completely meaningless. It's like measuring typing speed while ignoring typos. Until autonomous vehicles are subject to the full spectrum of conditions that human drivers face, incidents/mile is not a meaningful measure of comparative safety. And until then, the burden of proof should be on the creators.
  - stale2002 10y ago
    Not true. Lots of accidents happen in "good" conditions because humans suck at driving. The way you compare, is you give a self driving car to a person, and compare actually usage for that person, and see if owning a self driving car increases or reduces accidents for that person. Being perfect only half the time is still revolutionary.
    - saosebastiao 10y ago
      I have a 100% perfect safety record over probably hundreds of thousands of miles of driving a class B truck with my knees instead of my hands, using cruise control, on straight empty highways in Nevada, Idaho, and Utah, with perfect weather and good road conditions. There's nothing revolutionary about my ability to perfectly drive with my knees...I just very carefully selected the conditions where I was willing to do it.
      - stale2002 10y ago
        You may have that perfect safety record, but other people don't. People still crash every day in good conditions. That's thousands of lives that could be saved with our imperfect, sunny weather on the highway only self driving car. You underestimate how bad human drivers are even in perfect conditions.
        saosebastiao 10y ago
- dmritard96 10y ago
  beyond the obviousness of what you have overlooked, its also worth pointing out that not all autonomous car usage will be substitutive. What if i just send my autonomous car to go take a picture of something? I might not have done it before, but now I will because it only costs a few cents/mile and none of my time.
  - uw_rob 10y ago
    Another really cool use case will be sending your car to drive around the block for an hour or go back to your house and park, while you are going out to places. Beats paying the $10 to park for an hour.
    - Kadin 10y ago
      Lots of interesting effects there, all the way up to basic urban land-use decisions. Suddenly the price people will pay for parking is going to be closely related to the price of gas. The maximum value of a parking space in the city is basically the cost to send the car back out to some suburban parking garage during the day and have it come back in later for you. My guess is that the market-clearing price is going to be a lot lower than it is today. Lots of urban parking structures might end up getting repurposed for other uses, if you can't fill them at hundreds of dollars per space per month. In the short run I could see the average self-driving car's gas usage being higher than a conventional car's, but in the longer run the same self-driving features might make fully electric cars more palatable. Long charging times don't really matter if you can let the car go and charge itself while you're at work, for e.g.
      - tonyarkles 10y ago
        And if that place that it goes to charge itself happens to have a pile of solar and wind, even better! Way easier to do a bit installation in a less populated area than right in the centre of downtown.
komali2 10y ago
>inherently a danger Can you expand on how they are an inherent danger, rather than a possible danger? I only challenge you because Tesla has published several occasions where Autopilot has possibly saved lives by auto-braking.
- sangnoir 10y ago
  > Can you expand on how they are an inherent danger, rather than a possible danger? I do not believe those words (inherent & possible) are in any way mutually exclusive. They might even mean the same thing when describing a latent danger. So I do not quite understand the basis of your challenge. Also, I'm not parent poster.
  - komali2 10y ago
    My understanding of inherent might not be right, then. I think of it as "it is definitely a danger, and thus can only be described as dangerous," which I would argue is not mutually exclusive with possible but is mutually exclusive with "not always dangerous and maybe even anti-dangerous." So what I mean is, the autopilot could turn out to be safer than manual, in my mind, so how is it "inherently" dangerous.
    - dpark 10y ago
      A swimming pool in your backyard is an inherent danger. That doesn't mean anyone will be injured or killed, but a deep body of water with sheer walls is inherently dangerous. It would nonetheless also be accurate to describe it as a possible danger, because the danger only manifests in certain circumstances. > So what I mean is, the autopilot could turn out to be safer than manual, in my mind, so how is it "inherently" dangerous. That could well be the case but I don't think it eliminates the inherent danger. Maybe it's less dangerous, but cars on the road are just kind of dangerous in general. I think it's also very generous to assume that the early auto pilot car systems we have today are necessarily safer than human drivers. Emergency braking is almost certainly a net gain. Self driving might be or might not be. A self driving add-on cooked up by geohot? I'm doubtful it's yet gone through the kind of engineering rigor we'd expect of a system like this.
ryanmarsh 10y ago
Yes. There are a number of federal agencies that are a shit-show at best but I can't remember a time in my adult life when I was disappointed in NHTSA.
- ryanwaggoner 10y ago
  The NTSB also does amazing work, though it's the smallest Federal agency (~400 people), which I think probably helps greatly.
  - deleted 10y ago
    [deleted]
  - optionalparens 10y ago
    NTSB has awesome investigators. Some of the smartest people I've met and worked with in my life. They are very serious and dedicated, while having to put up with a lot of roadblocks, nonsense, and other issues from both corporate America and other government agencies. Can confirm at least though many years ago, the internal NTSB IT staff/programmers/DB were a joke. I guess that's just government though and like much of government, they relied a lot on outside contractors. For years they were a ColdFusion shop, so I'll let your mind start there.

by abduhl 10y ago

erikpukinskis 10y ago

Proof that so-called autopilot systems are a danger to the public would be evidence that they perform worse than the average driver. I haven't seen such evidence, what are you referencing?

TeMPOraL 10y ago
It's the other way around - you need a proof that they perform comparable to an average driver during expected use, and they do not have absurdly stupid corner cases during actually expected use.
- lutorm 10y ago
  Well, humans have absurdly stupid corner cases during actual use, too. It depends on how small the corner is, I think.
  - thfuran 10y ago
    Yes and humans require certification before they are permitted to drive vehicles on public roadways.
    - strathmeyer 10y ago
      To get that certification, do they test you first, or do they wait for you to screw up and then take it away from you??
      - azinman2 10y ago
        We know, being humans, and that we have the ability to process lots of complex information in a way that's very difficult for computers to replicate. Hard AI doesn't actually exist (yet). We also have a 100 years of humans driving cars worldwide so we understand well what they're good at and what they're not, so laws & safety designs take all of this into consideration. Each computer system will be encountering new, diverse things in the real world without a good understanding of how they'll perform. There are lots of crazy hard problems here that no one has solved yet. So to suggest we just automatically trust it because humans make mistakes is foolish when the consequences are so high. If someone came out with a surgery "autopilot" tomorrow, would you suggest it start giving triple bypass surgeries right away without FDA approval because humans make errors too?
        TeMPOraL 10y ago
        One of the features of the common human firmware is self-preservation instinct. It lets us trust that our fellow drivers, while still prone to mistakes, won't generally make obviously suicidal errors. Can one say the same about a new ML algorithm running on some board designed half a decade ago? How exactly would one know, without a thorough audit?
      - dragonwriter 10y ago
        They test you first, but if despite passing the test you screw up sufficiently badly later they decertify you. It's a belt-and-suspenders system.
  - TeMPOraL 10y ago
    We've been dealing with those corner cases for thousands of years and we know them pretty well. Given that we all run on pretty much the same hardware and firmware (with minor differences), you could say humans have been thoroughly tested for few millennia, including a hundred or so years of road testing. So yeah, I a bit of thorough testing of a completely new hardware running completely new software isn't too much to ask for.
deleted 10y ago
[deleted]
digler999 10y ago
so we should wait till a statistically-significant percentage of people die before we tell everyone, "yep, definitely dangerous. Lets add some regulation" ? I wonder if they could shut him down on the basis of car insurance requirements and the legal definition of "driver". If "driver" is defined as the "person" inputting control commands, then either you dont consider the computer a person, and then nobody is legally "driving" the car, or else the software somehow meets the definition of a "person" or driver, in which case its an unlicensed driver operating the vehicle and insurance may not have to cover it.
- smallnamespace 10y ago
  The NHTSA has been a strong proponent of self-driving technology and has gone out of its way to make sure there is a path to doing it legally, without pulling the legal 'gotcha' that you described here. The device would probably be SAE Level 2, and should be regulated as such.
saosebastiao 10y ago
When you measure the safety of autonomous cars by counting the miles they travel, and not counting the miles that they intentionally avoid or defer to a human, you have succeeded in creating a metric that looks like it's useful for comparison, but is actually completely meaningless. It's like measuring typing speed while ignoring typos. Until autonomous vehicles are subject to the full spectrum of conditions that human drivers face, incidents/mile is not a meaningful measure of comparative safety. And until then, the burden of proof should be on the creators.
- stale2002 10y ago
  Not true. Lots of accidents happen in "good" conditions because humans suck at driving. The way you compare, is you give a self driving car to a person, and compare actually usage for that person, and see if owning a self driving car increases or reduces accidents for that person. Being perfect only half the time is still revolutionary.
  - saosebastiao 10y ago
    I have a 100% perfect safety record over probably hundreds of thousands of miles of driving a class B truck with my knees instead of my hands, using cruise control, on straight empty highways in Nevada, Idaho, and Utah, with perfect weather and good road conditions. There's nothing revolutionary about my ability to perfectly drive with my knees...I just very carefully selected the conditions where I was willing to do it.
    - stale2002 10y ago
      You may have that perfect safety record, but other people don't. People still crash every day in good conditions. That's thousands of lives that could be saved with our imperfect, sunny weather on the highway only self driving car. You underestimate how bad human drivers are even in perfect conditions.
      - saosebastiao 10y ago
dmritard96 10y ago
beyond the obviousness of what you have overlooked, its also worth pointing out that not all autonomous car usage will be substitutive. What if i just send my autonomous car to go take a picture of something? I might not have done it before, but now I will because it only costs a few cents/mile and none of my time.
- uw_rob 10y ago
  Another really cool use case will be sending your car to drive around the block for an hour or go back to your house and park, while you are going out to places. Beats paying the $10 to park for an hour.
  - Kadin 10y ago
    Lots of interesting effects there, all the way up to basic urban land-use decisions. Suddenly the price people will pay for parking is going to be closely related to the price of gas. The maximum value of a parking space in the city is basically the cost to send the car back out to some suburban parking garage during the day and have it come back in later for you. My guess is that the market-clearing price is going to be a lot lower than it is today. Lots of urban parking structures might end up getting repurposed for other uses, if you can't fill them at hundreds of dollars per space per month. In the short run I could see the average self-driving car's gas usage being higher than a conventional car's, but in the longer run the same self-driving features might make fully electric cars more palatable. Long charging times don't really matter if you can let the car go and charge itself while you're at work, for e.g.
    - tonyarkles 10y ago
      And if that place that it goes to charge itself happens to have a pile of solar and wind, even better! Way easier to do a bit installation in a less populated area than right in the centre of downtown.

komali2 10y ago

>inherently a danger Can you expand on how they are an inherent danger, rather than a possible danger? I only challenge you because Tesla has published several occasions where Autopilot has possibly saved lives by auto-braking.

sangnoir 10y ago
> Can you expand on how they are an inherent danger, rather than a possible danger? I do not believe those words (inherent & possible) are in any way mutually exclusive. They might even mean the same thing when describing a latent danger. So I do not quite understand the basis of your challenge. Also, I'm not parent poster.
- komali2 10y ago
  My understanding of inherent might not be right, then. I think of it as "it is definitely a danger, and thus can only be described as dangerous," which I would argue is not mutually exclusive with possible but is mutually exclusive with "not always dangerous and maybe even anti-dangerous." So what I mean is, the autopilot could turn out to be safer than manual, in my mind, so how is it "inherently" dangerous.
  - dpark 10y ago
    A swimming pool in your backyard is an inherent danger. That doesn't mean anyone will be injured or killed, but a deep body of water with sheer walls is inherently dangerous. It would nonetheless also be accurate to describe it as a possible danger, because the danger only manifests in certain circumstances. > So what I mean is, the autopilot could turn out to be safer than manual, in my mind, so how is it "inherently" dangerous. That could well be the case but I don't think it eliminates the inherent danger. Maybe it's less dangerous, but cars on the road are just kind of dangerous in general. I think it's also very generous to assume that the early auto pilot car systems we have today are necessarily safer than human drivers. Emergency braking is almost certainly a net gain. Self driving might be or might not be. A self driving add-on cooked up by geohot? I'm doubtful it's yet gone through the kind of engineering rigor we'd expect of a system like this.

ryanmarsh 10y ago

Yes. There are a number of federal agencies that are a shit-show at best but I can't remember a time in my adult life when I was disappointed in NHTSA.

ryanwaggoner 10y ago
The NTSB also does amazing work, though it's the smallest Federal agency (~400 people), which I think probably helps greatly.
- deleted 10y ago
  [deleted]
- optionalparens 10y ago
  NTSB has awesome investigators. Some of the smartest people I've met and worked with in my life. They are very serious and dedicated, while having to put up with a lot of roadblocks, nonsense, and other issues from both corporate America and other government agencies. Can confirm at least though many years ago, the internal NTSB IT staff/programmers/DB were a joke. I guess that's just government though and like much of government, they relied a lot on outside contractors. For years they were a ColdFusion shop, so I'll let your mind start there.