6 ms·

And it’d be morally unacceptable to allow public deployment of software that has not been sufficiently tested. As other comments have pointed out, self-driving

And it’d be morally unacceptable to allow public deployment of software that has not been sufficiently tested. As other comments have pointed out, self-driving cars have undergone ridiculously little testing. In fact, based on only the objective statistics it is very unlikely that they are anywhere near as good at driving as humans. There is no reason self-driving cars can’t be tested in private. Companies can hire pedestrians to interact with the cars, and the software can go through the same certification process that buildings and vehicles currently go through. It’s a false dichotomy to say that you can either have self driving cars or minimally safe and accountable development, but not both. EDIT: Here is a link to a thread replying to a parent which is now auto-collapsed. https://news.ycombinator.com/item?id=16620968 https://news.ycombinator.com/item?id=16620968

jamiequint 9y ago
> "it’d be morally unacceptable to allow public deployment of software that has not been sufficiently tested" Does "sufficiently tested" mean you get the end-product a year later? What if the end-product saves 10,000 lives a year? In that case what cost is morally acceptable to get there a year faster?
- thorell 9y ago
  You can't project the actual cost without sufficient testing.
- rootlocus 9y ago
  > What if the end-product saves 10,000 lives a year? You can't have the certainty to make such statements without "sufficiently testing" the product.
  - jamiequint 9y ago
    > You can't have the certainty to make such statements Can you point me to where I made a statement? I only see questions.
    - rootlocus 9y ago
      You're misinterpreting me. I meant "nobody" can make that such statements. As in, if you (somebody) have a a product and you (somebody) haven't tested it enough, you (somebody) can't state that it will save N lives per year.
stormbrew 9y ago
I'm just going to point out that nearly every time someone gets their learners permit, or graduated from a learners permit to a full license, an insufficiently tested driver is allowed on the road in order to develop more skills and become a better driver. Often they kill people in the process of learning. Should we require years of private training for every human driver as well? They can't even share what they learn with each other effectively.
- jonathanyc 9y ago
  The difference is we have a lot of experience with the human brain, and not much at all with self-driving cars. All we have to go on are the statistics, which say that although humans are bad drivers, self-driving cars are even worse (or at best that we don’t have sufficient information). Again, why not just test them in private and hire people to be pedestrians?
  - lhorie 9y ago
    > why not just test them in private and hire people to be pedestrians? And get these employees to do what exactly? You obviously can't tell them to put their lives at risk to test what happens if they walk onto a car's path, or if they ride a bike on a car's blind spot wearing dark clothes at night, or if they fall from a motorcycle in front of a car after hitting a raccoon. The dichotomy here is that private courses are inherently orderly and the real world is inherently chaotic.
    - jonathanyc 9y ago
      That is exactly what those employees would have to be paid and consent to do—just like test drivers. In fact, someone else linked a Waymo blogpost saying that they have employees do exactly those things (walking into cars’ paths, lying down on skateboards...) How is it not OK to have consenting people do these things but OK to have random people on the street participate in exactly the same tests?
      - 9y ago
nightski 9y ago
Sure you could test it privately. Probably quite extensively. But if you have any inclination as to how deep learning works you might agree that this would not be sufficient in any way. The only way that will really work is on real world streets.
- jonathanyc 9y ago
  And if you had any inclination as to how robotics (or indeed engineering!) works I’m sure you would know the value of testing dangerous and extremely immature equipment with people who consented to be your test subjects and in controlled environments. “Deep learning” is no excuse. Certainly in the long run we would want to test in the real world. The argument is whether or not we are there yet. Other commenters have made a compelling argument, again based on the statistics, that we are not there yet. Do you disagree, and if so, based on what facts?
  - nightski 9y ago
    I never made an excuse nor said we should be testing these on live streets. What I did say is that with deep learning it simply is not possible to "test" in a simulated environment to any level of certainty. You can use simulated environments with test subjects to help develop such as system (and should do so). But you will never be able to adequately test such a system in this environment. Why? The state space of traditional robotics and engineering problems is far more constrained. So much so that you can't even compare the fields in my opinion.
    - c22 9y ago
      But you can still gather learning data from sensor-covered cars "in the wild" and then test it with contrived tests.
danso 9y ago
Can't speak for Uber, but Waymo/Google apparently has billions of miles tested on closed track, vs just 4+ million miles on the open road: https://medium.com/waymo/waymos-fleet-reaches-4-million-self-driven-miles-b28f32de495a https://medium.com/waymo/waymos-fleet-reaches-4-million-self...
wetpaws 9y ago
Every day self driving car is not tested in real life environment is a day self driving technology delayed and a day worth of hundreds of human lives. If anything, we are morally obliged to double down.
- more_corn 9y ago
  Yes, we're obliged to double down. On safety. The Uber self driving cars are known to be unsafe around pedestrians. Or another way of saying it "people walking". Or just "people". Like you and like me. I had one try to run me down in the crosswalk (utterly failed to yield to pedestrians in the crosswalk) while Uber was running their ill-fated test in San Francisco. I walked into their offices on Harrison and asked to file a bug report. They laughed it off. Saw the exact thing happen a couple days later. They need to stop till they can figure it out how to do it safely.
- LoSboccacc 9y ago
  Absolutely not. Current data point at self driving car being more dangerous than human driven cars. Double down in real environment testing and you’ll expose the public to increased danger until software becomes good enough - you’re basically telling random people to share publicly the risks so other people in the future may or may not be safer while private companies profit from the fruit of the research and, sincerely, screw that: public test should start after controlled testing proves the system to be at least not more dangerous than the status quo and the companies should use testers that opt in and are adeguately compensated for their liabilities to injuries instead of killing off random bystanders for the “greater good” which in trith is how they name their pyroll savings
- notahacker 9y ago
  When considering whether they're net saving lives we are now doing so from the starting point that self driving cars presently have a worse fatality record per real world mile driven than humans, including the portion of miles driven by humans who are unfit to drive or wilfully reckless. Self driving cars can also be tested in real life environments with safety drivers behind the wheel whose own driving shortcomings are unlikely to coincide with software error. This is believed from the OEM's own telemetry to have prevented several accidents with early generation Waymo (Edit: and apparently one was present in this vehicle and was unable to prevent the accident) Of course, there's a PR benefit of taking the drivers out of the car at this stage of their development, and it appears this has taken priority...

jamiequint 9y ago

> "it’d be morally unacceptable to allow public deployment of software that has not been sufficiently tested" Does "sufficiently tested" mean you get the end-product a year later? What if the end-product saves 10,000 lives a year? In that case what cost is morally acceptable to get there a year faster?

thorell 9y ago
You can't project the actual cost without sufficient testing.
rootlocus 9y ago
> What if the end-product saves 10,000 lives a year? You can't have the certainty to make such statements without "sufficiently testing" the product.
- jamiequint 9y ago
  > You can't have the certainty to make such statements Can you point me to where I made a statement? I only see questions.
  - rootlocus 9y ago
    You're misinterpreting me. I meant "nobody" can make that such statements. As in, if you (somebody) have a a product and you (somebody) haven't tested it enough, you (somebody) can't state that it will save N lives per year.

stormbrew 9y ago

I'm just going to point out that nearly every time someone gets their learners permit, or graduated from a learners permit to a full license, an insufficiently tested driver is allowed on the road in order to develop more skills and become a better driver. Often they kill people in the process of learning. Should we require years of private training for every human driver as well? They can't even share what they learn with each other effectively.

jonathanyc 9y ago
The difference is we have a lot of experience with the human brain, and not much at all with self-driving cars. All we have to go on are the statistics, which say that although humans are bad drivers, self-driving cars are even worse (or at best that we don’t have sufficient information). Again, why not just test them in private and hire people to be pedestrians?
- lhorie 9y ago
  > why not just test them in private and hire people to be pedestrians? And get these employees to do what exactly? You obviously can't tell them to put their lives at risk to test what happens if they walk onto a car's path, or if they ride a bike on a car's blind spot wearing dark clothes at night, or if they fall from a motorcycle in front of a car after hitting a raccoon. The dichotomy here is that private courses are inherently orderly and the real world is inherently chaotic.
  - jonathanyc 9y ago
    That is exactly what those employees would have to be paid and consent to do—just like test drivers. In fact, someone else linked a Waymo blogpost saying that they have employees do exactly those things (walking into cars’ paths, lying down on skateboards...) How is it not OK to have consenting people do these things but OK to have random people on the street participate in exactly the same tests?
    - 9y ago

nightski 9y ago

Sure you could test it privately. Probably quite extensively. But if you have any inclination as to how deep learning works you might agree that this would not be sufficient in any way. The only way that will really work is on real world streets.

jonathanyc 9y ago
And if you had any inclination as to how robotics (or indeed engineering!) works I’m sure you would know the value of testing dangerous and extremely immature equipment with people who consented to be your test subjects and in controlled environments. “Deep learning” is no excuse. Certainly in the long run we would want to test in the real world. The argument is whether or not we are there yet. Other commenters have made a compelling argument, again based on the statistics, that we are not there yet. Do you disagree, and if so, based on what facts?
- nightski 9y ago
  I never made an excuse nor said we should be testing these on live streets. What I did say is that with deep learning it simply is not possible to "test" in a simulated environment to any level of certainty. You can use simulated environments with test subjects to help develop such as system (and should do so). But you will never be able to adequately test such a system in this environment. Why? The state space of traditional robotics and engineering problems is far more constrained. So much so that you can't even compare the fields in my opinion.
  - c22 9y ago
    But you can still gather learning data from sensor-covered cars "in the wild" and then test it with contrived tests.

danso 9y ago

Can't speak for Uber, but Waymo/Google apparently has billions of miles tested on closed track, vs just 4+ million miles on the open road: https://medium.com/waymo/waymos-fleet-reaches-4-million-self-driven-miles-b28f32de495a https://medium.com/waymo/waymos-fleet-reaches-4-million-self...

wetpaws 9y ago

Every day self driving car is not tested in real life environment is a day self driving technology delayed and a day worth of hundreds of human lives. If anything, we are morally obliged to double down.

more_corn 9y ago
Yes, we're obliged to double down. On safety. The Uber self driving cars are known to be unsafe around pedestrians. Or another way of saying it "people walking". Or just "people". Like you and like me. I had one try to run me down in the crosswalk (utterly failed to yield to pedestrians in the crosswalk) while Uber was running their ill-fated test in San Francisco. I walked into their offices on Harrison and asked to file a bug report. They laughed it off. Saw the exact thing happen a couple days later. They need to stop till they can figure it out how to do it safely.
LoSboccacc 9y ago
Absolutely not. Current data point at self driving car being more dangerous than human driven cars. Double down in real environment testing and you’ll expose the public to increased danger until software becomes good enough - you’re basically telling random people to share publicly the risks so other people in the future may or may not be safer while private companies profit from the fruit of the research and, sincerely, screw that: public test should start after controlled testing proves the system to be at least not more dangerous than the status quo and the companies should use testers that opt in and are adeguately compensated for their liabilities to injuries instead of killing off random bystanders for the “greater good” which in trith is how they name their pyroll savings
notahacker 9y ago
When considering whether they're net saving lives we are now doing so from the starting point that self driving cars presently have a worse fatality record per real world mile driven than humans, including the portion of miles driven by humans who are unfit to drive or wilfully reckless. Self driving cars can also be tested in real life environments with safety drivers behind the wheel whose own driving shortcomings are unlikely to coincide with software error. This is believed from the OEM's own telemetry to have prevented several accidents with early generation Waymo (Edit: and apparently one was present in this vehicle and was unable to prevent the accident) Of course, there's a PR benefit of taking the drivers out of the car at this stage of their development, and it appears this has taken priority...