5 ms·

I think it's unacceptable for automation to produce a worse result than a human in the same situation, with the same information. i.e. it's not acceptable for a

by cmurf 8y ago

I think it's unacceptable for automation to produce a worse result than a human in the same situation, with the same information. i.e. it's not acceptable for automation to fail danger, it must fail safe, including even if all it can do is give up (disconnects, warning tone, hands control over to the human driver). I think it's reasonable, in the narrow case where primary control is asserted and competency is claimed to be as good or better than a human, to hold automation accountable the same as a human. And in this case, if this driver acted the way autopilot did based on the same available information, we would say the driver committed suicide or was somehow incompetent. I see this as possibly a case of automation committing involuntary manslaughter (unintentional homicide from criminally negligent or reckless conduct).

Skunkleton 8y ago
> ... it's not acceptable for automation to fail danger, it must fail safe... The scary thing is, by definition of what is being automated there can be no fail-safe. The only "safe" is to stop the car. If the car detected it was going to crash, then it wouldn't have had the failure in the first place. By the time the failure might be detected, it is already too late for "safe".
- emiliobumachar 8y ago
  It's worse than that. Sudden breaking is also dangerous.
1123581321 8y ago
We don’t expect all kinds of drivers and vehicles to fit the same safety bell curve. You’ve made an assertion, but what legal framework are you are using to treat this particular human-machine interaction differently without introducing a whole new class of liability for humans and traditional manufacturers?
- kadendogthing 8y ago
  >but what legal framework are you are using to treat this particular human-machine interaction differently without introducing a whole new class of liability for humans and traditional manufacturers? Humans are not robots?
stevenjohns 8y ago
> or was somehow incompetent If they survived, they would have lost their license and been charged with some form of reckless driving. The underlying issue is that manufacturers are beta testing on the road and almost literally playing with peoples lives in the process. Legislation, although would hurt innovation, in this case is appropriate to ensure that the race to the finish doesn’t have a blood-stained legacy.
- threeseed 8y ago
  Tesla is the only one claiming to have a working solution and is selling it to the general public. Everyone else is far more cautious.
modzu 8y ago
you cant expect the human driver to be ready to take over autopilot in any meaningful way. if the driver is alert enough to the conditions and physically ready at the appropriate controls to do that, then they are already driving. still, despite it not being based on science, but legal reasons, companies pretend that humans can "take over" when alerted to. maybe a lawsuit will change that idiocy.
- jimmaswell 8y ago
  A few people dying of their own irresponsibility does not warrant ruining it for the rest of us.
  - patejam 8y ago
    There are other people on the road, it's not just the drivers that are impacted by this. Calling it "Autopilot" is what's irresponsible.
    - jimmaswell 8y ago
      Pilots still need to take over sometimes despite the autopilot. The name is fine.
  - protomyth 8y ago
    A lot of people said the exact same thing about lawn darts and a whole host of products and activities, but sadly, society doesn't go that way.
    - modzu 8y ago
      im all for the technology and accepting it will be fallible, just not for putting the blame for its errors onto the human non-driver
corodra 8y ago
Well, this brings to light the concept from the movie I, Robot. Is it a crime when an ai kills someone, unintentional or not, or is it an industrial accident? If it's suppose to replace a human task due to a "level of intelligence", is it still taxed as equipment or as an employee? Is the company (due to equipment failures) or the ai (choosing to do an action on its own) at fault? To be fair, these questions need to be hard lined pretty soon.
- Bendingo 8y ago
  Don't know why this is downvoted - it's a good statement of the essential issue.
- mr_toad 8y ago
  > these questions need to be hard lined pretty soon I’d argue that they’re a long way from being an issue. Firstly, current AI are far less ‘intelligent’ than almost any animal. Even a mosquito has more brain power, and we don’t think twice about killing mosquitoes. Secondly, even if an AI had similar intelligence to a human, there is no reason to believe it would be a moral creature, capable of making moral judgments, and being judged as such. Our morality evolved over thousands, probably millions, of years (or if you prefer, it was granted by some divine power). Either way, intelligence and morality aren’t synonymous.
lucb1e 8y ago
> it's unacceptable for automation to produce a worse result than a human in the same situation, with the same information I don't think anyone disagrees on that. The question is: is the software worse than a human driver? Do we have enough data for a statistically significant judgement on that? Is it even autonomous enough to say anything either way, like, if the driver is required to be at attention anyway, can the software be blamed for anything in the first place? Those are the questions, I don't think there is a point saying "software must be good!"
- jessriedel 8y ago
  >> it's unacceptable for automation to produce a worse result than a human in the same situation, with the same information > I don't think anyone disagrees on that. I disagree on that. If there's an autonomous vehicle that is better than a human in most situations, and worse in a few situations, such that the overall accident/death rate is lower, and there is no reasonable away to identify the rare dangerous situations in time to disable the autopilot, I would want to drive that car and would advise others to do so. In fact, if there was an autonomous vehicle that was almost exactly as safe as a human but slightly more dangerous (say, a 10% higher death/accident rate), I would frequently use it because the large benefits outweigh the minor statistical costs. (Indeed, I use a car at all because of its benefits over walking, busing, or staying at home, despite the higher rate of death.) If other people understood the risks, I would also suggest that they to do likewise.
btilly 8y ago
This sounds good, but is impractical. First, a human who is paying attention makes better decision than a human who is surprised by being given control during a crisis. Therefore if automation sees a crisis, it's standard has to be that it is worse than a surprised human, and not how it compares to an alert driver. Second, the analysis that concludes that we're now in a specific situation where a human would do better is often beyond the powers of automation. For example it can recognize that we are now in a class of cases where, on average, automation does better. But whether it will in any particular case may be beyond its reasoning power.
- salawat 8y ago
  This is called the Halting Problem. We don't waste time trying to make machines that do that because it's theoretically undecidable. Furthermore, you're then throwing more neural network (that has to be trained improvably across the dataset which we have no proof is the full dataset required to fully enumerate the problem space), to successfully detect the problem areas in the data going into the driving NN for edge cases. Repeat ad absurdum or until most of your self-driving car's energy is consumed trying to solve the Silicon equivalent of Plato's Cave. Or... You just teach people to drive the blessed car, which is what the absolute best a theoretical NN implementation would converge to anyway. The only benefit to a theoretically perfect NN which simulates the average driver being the car doesn't run on a processing system that is adversely effected through the recreational and voluntary (by the processing hardware) application of ethanol to the processing matrix. Well... Thermally speaking, it could end up causing serious problems if someone were dumb enough to build the capability to do that into the car for some daft reason, but we're talking existential failure of computing hardware rather than gradual degradation of functionality like you'd see with a brain, but still I think everyone gets the point and I can stop torturing this poor analogy.
  - btilly 8y ago
    What you say has just enough technical detail to seem entirely plausible to someone who doesn't understand technology. It is also entirely wrong. The problem that I am describing is not the Halting Problem. A theoretically perfect NN is also able to do a lot better than any person could. The ability to have binocular vision in multiple directions, and mix that with radar awareness of the local environment and electronic communication with other cars to be aware of obstacles that are out of sight. This technology can drive more quickly and safely than humans ever could. This sort of thing is admittedly some time off. But it is doable, and it is almost certain to happpen within our lifetimes.

by cmurf 8y ago

Skunkleton 8y ago

> ... it's not acceptable for automation to fail danger, it must fail safe... The scary thing is, by definition of what is being automated there can be no fail-safe. The only "safe" is to stop the car. If the car detected it was going to crash, then it wouldn't have had the failure in the first place. By the time the failure might be detected, it is already too late for "safe".

emiliobumachar 8y ago
It's worse than that. Sudden breaking is also dangerous.

1123581321 8y ago

We don’t expect all kinds of drivers and vehicles to fit the same safety bell curve. You’ve made an assertion, but what legal framework are you are using to treat this particular human-machine interaction differently without introducing a whole new class of liability for humans and traditional manufacturers?

kadendogthing 8y ago
>but what legal framework are you are using to treat this particular human-machine interaction differently without introducing a whole new class of liability for humans and traditional manufacturers? Humans are not robots?

stevenjohns 8y ago

> or was somehow incompetent If they survived, they would have lost their license and been charged with some form of reckless driving. The underlying issue is that manufacturers are beta testing on the road and almost literally playing with peoples lives in the process. Legislation, although would hurt innovation, in this case is appropriate to ensure that the race to the finish doesn’t have a blood-stained legacy.

threeseed 8y ago
Tesla is the only one claiming to have a working solution and is selling it to the general public. Everyone else is far more cautious.

modzu 8y ago

you cant expect the human driver to be ready to take over autopilot in any meaningful way. if the driver is alert enough to the conditions and physically ready at the appropriate controls to do that, then they are already driving. still, despite it not being based on science, but legal reasons, companies pretend that humans can "take over" when alerted to. maybe a lawsuit will change that idiocy.

jimmaswell 8y ago
A few people dying of their own irresponsibility does not warrant ruining it for the rest of us.
- patejam 8y ago
  There are other people on the road, it's not just the drivers that are impacted by this. Calling it "Autopilot" is what's irresponsible.
  - jimmaswell 8y ago
    Pilots still need to take over sometimes despite the autopilot. The name is fine.
- protomyth 8y ago
  A lot of people said the exact same thing about lawn darts and a whole host of products and activities, but sadly, society doesn't go that way.
  - modzu 8y ago
    im all for the technology and accepting it will be fallible, just not for putting the blame for its errors onto the human non-driver

corodra 8y ago

Well, this brings to light the concept from the movie I, Robot. Is it a crime when an ai kills someone, unintentional or not, or is it an industrial accident? If it's suppose to replace a human task due to a "level of intelligence", is it still taxed as equipment or as an employee? Is the company (due to equipment failures) or the ai (choosing to do an action on its own) at fault? To be fair, these questions need to be hard lined pretty soon.

Bendingo 8y ago
Don't know why this is downvoted - it's a good statement of the essential issue.
mr_toad 8y ago
> these questions need to be hard lined pretty soon I’d argue that they’re a long way from being an issue. Firstly, current AI are far less ‘intelligent’ than almost any animal. Even a mosquito has more brain power, and we don’t think twice about killing mosquitoes. Secondly, even if an AI had similar intelligence to a human, there is no reason to believe it would be a moral creature, capable of making moral judgments, and being judged as such. Our morality evolved over thousands, probably millions, of years (or if you prefer, it was granted by some divine power). Either way, intelligence and morality aren’t synonymous.

lucb1e 8y ago

> it's unacceptable for automation to produce a worse result than a human in the same situation, with the same information I don't think anyone disagrees on that. The question is: is the software worse than a human driver? Do we have enough data for a statistically significant judgement on that? Is it even autonomous enough to say anything either way, like, if the driver is required to be at attention anyway, can the software be blamed for anything in the first place? Those are the questions, I don't think there is a point saying "software must be good!"

jessriedel 8y ago
>> it's unacceptable for automation to produce a worse result than a human in the same situation, with the same information > I don't think anyone disagrees on that. I disagree on that. If there's an autonomous vehicle that is better than a human in most situations, and worse in a few situations, such that the overall accident/death rate is lower, and there is no reasonable away to identify the rare dangerous situations in time to disable the autopilot, I would want to drive that car and would advise others to do so. In fact, if there was an autonomous vehicle that was almost exactly as safe as a human but slightly more dangerous (say, a 10% higher death/accident rate), I would frequently use it because the large benefits outweigh the minor statistical costs. (Indeed, I use a car at all because of its benefits over walking, busing, or staying at home, despite the higher rate of death.) If other people understood the risks, I would also suggest that they to do likewise.

btilly 8y ago

This sounds good, but is impractical. First, a human who is paying attention makes better decision than a human who is surprised by being given control during a crisis. Therefore if automation sees a crisis, it's standard has to be that it is worse than a surprised human, and not how it compares to an alert driver. Second, the analysis that concludes that we're now in a specific situation where a human would do better is often beyond the powers of automation. For example it can recognize that we are now in a class of cases where, on average, automation does better. But whether it will in any particular case may be beyond its reasoning power.

salawat 8y ago
This is called the Halting Problem. We don't waste time trying to make machines that do that because it's theoretically undecidable. Furthermore, you're then throwing more neural network (that has to be trained improvably across the dataset which we have no proof is the full dataset required to fully enumerate the problem space), to successfully detect the problem areas in the data going into the driving NN for edge cases. Repeat ad absurdum or until most of your self-driving car's energy is consumed trying to solve the Silicon equivalent of Plato's Cave. Or... You just teach people to drive the blessed car, which is what the absolute best a theoretical NN implementation would converge to anyway. The only benefit to a theoretically perfect NN which simulates the average driver being the car doesn't run on a processing system that is adversely effected through the recreational and voluntary (by the processing hardware) application of ethanol to the processing matrix. Well... Thermally speaking, it could end up causing serious problems if someone were dumb enough to build the capability to do that into the car for some daft reason, but we're talking existential failure of computing hardware rather than gradual degradation of functionality like you'd see with a brain, but still I think everyone gets the point and I can stop torturing this poor analogy.
- btilly 8y ago
  What you say has just enough technical detail to seem entirely plausible to someone who doesn't understand technology. It is also entirely wrong. The problem that I am describing is not the Halting Problem. A theoretically perfect NN is also able to do a lot better than any person could. The ability to have binocular vision in multiple directions, and mix that with radar awareness of the local environment and electronic communication with other cars to be aware of obstacles that are out of sight. This technology can drive more quickly and safely than humans ever could. This sort of thing is admittedly some time off. But it is doable, and it is almost certain to happpen within our lifetimes.