7 ms·

For all of the theoretical moral questions that have been brought up around self driving cars (the proverbial lever that lets you kill one person instead of a h

by suresk 9y ago

For all of the theoretical moral questions that have been brought up around self driving cars (the proverbial lever that lets you kill one person instead of a handful, for example), it's interesting/inevitable that this one came up first. I'm sure lots of people will argue that self-driving cars will, in the long run, save a lot of lives, and therefore we can justify some loss of life in the process. They are probably right about self-driving cars saving more lives in the long run, but I don't know that the conclusion follows. From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing and would not have been at-fault in the collision. But self-driving cars have more sensors and, in theory, more capabilities to prevent something like this, and we absolutely should expect them to do better. As someone with a little bit of experience with the machine learning side of things, I understand how this stuff is really hard, and I can imagine a handful of ways the system could have been confused, which makes it all the more important to understand what went wrong and how we could prevent it from happening in the future. As self-driving cars become more prevalent, things like this will happen and it will be tough to walk the line between over-reacting to every incident and killing progress in this area and designing safe cars that don't make the problem worse. I think it is prudent to have Uber stop testing on public roads until they can explain how this happened and how they can keep it from happening again.

ars 9y ago
> From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing No, a human would have easily avoided this (unless they were not looking). The video is much darker than what a person would see. Even when you can see the bike on that dark video, the car still does not react!
MBCook 9y ago
Based on what came out of the last few days I don’t think it’s clear at all, I think a human would not have hit the woman. The footage we saw was from a terrible dashcam and did not represent how dark the street actually was. Other people who have driven down the street with cameras show it to be perfectly reasonably lit, with streetlights and everything. Second, the woman was crossing across two lanes, so any driver watching would have already seen her cross the left-hand lane. It’s not like she popped out from behind a tree suddenly. She wasn’t even going that fast. This seems to be a massive failure of Uber’s system as well as a complete failure of the ‘safety’ driver. I don’t think anyone’s overreacting. I think they’re under reacting. I think it’s pretty clear over was heavily negligent. There’s no way they don’t get sued. What comes out at trial is going to be up amazing, I think. I think if this had been a “real” accident that would’ve been almost impossible to avoid, around it WAS impossible to avoid, or one where the human stepped in and it wasn’t good enough… We would be having a discussion we’re like what you’re saying. Maybe if this was what emai I think if this had been a “real“ accident that would’ve been almost impossible to avoid, around it WAS impossible to avoid, or one where the human stepped in and it wasn’t good enough… We would be having a discussion we’re like what you’re saying. Maybe if this was Waymo or one of the other comapnies. But it looks like the dubious honor of first death by a autonomous car was totally unnecessary and preventable.
- sokoloff 9y ago
  The pedestrian also has a significant contribution of negligence to this collision, IMO.
  - mjmahone17 9y ago
    Why? They crossed a 35 mph (so in Tempe this is a “slow” road) at a corner (unmarked corners are also crosswalks, legally) on a well lit road. Where the car was many seconds away when she started crossing. Would a human driver be deemed at fault in a court of law? Usually not (though in this case, maybe) unless they kept going after the collision, or were under the influence. But that’s because we give human drivers a lot of leeway to kill people. Not because it isn’t their fault.
    - bendauphinee 9y ago
      As a pedestrian, it's still our responsibility to make sure approaching vehicles are actually stopped before we start crossing. She did not, or she would be alive.
      - lutorm 9y ago
        It's certainly prudent to do so, and any amount of self-preservation says you should, but is not checking if you're about to be run over actually negligent? Edit: Also, if you as a pedestrian, stand at a 35mph road, at night, and wait for all approaching cars to stop before you start crossing, you will be standing there a very long time indeed.
        sokoloff 9y ago
        One legal definition of negligence is A failure to behave with the level of care that someone of ordinary prudence would have exercised under the same circumstances. If you've established that doing so is "certainly prudent", then I think it's reasonable that not doing so is negligent.
dontreact 9y ago
"From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing and would not have been at-fault in the collision." The camera footage is A LOT darker than what people have filmed there just a day or two after and is probably not a good representation of what it looked like to the human eye. Even if it was fairly accurate, the car would be HEAVILY and EXTREMELY dangerously overdriving its headlights in a way that humans would not do.
takeda 9y ago
> From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing and would not have been at-fault in the collision. But self-driving cars have more sensors and, in theory, more capabilities to prevent something like this, and we absolutely should expect them to do better. The footage was artificially darkened, or it was extremely crappy dash cam or both. I suspect Uber just tampered with it (to get some sympathy) Someone posted a footage from that place few days later and that is very well lit road. On Uber footage you can see lamps on the sky, yet it is very dark. Also note this dashcam is not what the car uses, and the car is also equipped with LIDAR so it should be able to spot pedestrian even in pitch black. > As self-driving cars become more prevalent, things like this will happen and it will be tough to walk the line between over-reacting to every incident and killing progress in this area and designing safe cars that don't make the problem worse. I think it is prudent to have Uber stop testing on public roads until they can explain how this happened and how they can keep it from happening again. Your are missing the point here. Preventing accidents like this should be thing that AI would excel at, even today. The fact that this failed means that there is something seriously wrong with how Uber is doing it.
- Aspyre 9y ago
  How would the footage be tampered when it was taken by the police directly from the dashcam itself at the scene? Also, there was no moon on that night and all of the video comparisons that have been put out are low-light cameras. They are not very realistic. Also, noone seems to think it's a possibility that the agencies responsible for the lighting in this area to have made improvements immediately after the inicident before the public speculation started.
  - ceejayoz 9y ago
    > all of the video comparisons that have been put out are low-light cameras If a self-driving car can't see 50 feet at night, maybe they should buy some low-light cameras? > Also, noone seems to think it's a possibility that the agencies responsible for the lighting in this area to have made improvements immediately after the inicident before the public speculation started. None of this matters. (I also find it pretty unlikely - the lights shown in Uber's video look plenty close together already to safely light the road. Even on an unlit road, the car's own headlights should've revealed the pedestrian far sooner.) A self-driving car should be able to see (via the car's headlights, infrared, LIDAR, etc.) a safe distance ahead of the car at all times. The car should detect the unsafe condition and refuse to continue - not mow down pedestrians. If it can't cope with a moonless, unlit road, it should not be on that road.
zer00eyz 9y ago
I'm going to take your post apart because I enjoyed it very much... > For all of the theoretical moral questions that have been brought up around self driving cars (the proverbial lever that lets you kill one person instead of a handful, for example), it's interesting/inevitable that this one came up first. Your talking about the "trolley problem" and it is a ethical question as opposed to moral. This is literally a fuckup, bad programing or poor design. We are going to have real ethical concerns because at some point a car is going to have to make a choice pedestrian or occupant - and I know that my elderly father would choose himself if he were in the car, but I might make a different choice.... will the cars ethics ever reflect my own? > I'm sure lots of people will argue that self-driving cars will, in the long run, save a lot of lives, and therefore we can justify some loss of life in the process. They are probably right about self-driving cars saving more lives in the long run, but I don't know that the conclusion follows. It is a tough one, and if there is more loss of life does using a self driving car a bit of fruit off a poison tree. https://en.wikipedia.org/wiki/Eduard_Pernkopf https://en.wikipedia.org/wiki/Eduard_Pernkopf springs to mind when you bring this up. > From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing Except the fact that when you look at footage of the road when driven on with a decent camera recording rather than the crapy low quality and very much artificially dark uber footage this looks to have been very avoidable. If the guy who was supposed to be there to take over had been looking up right then, we might not be talking about this at all... > As someone with a little bit of experience with the machine learning side of things, I understand how this stuff is really hard, and I can imagine a handful of ways the system could have been confused, which makes it all the more important to understand what went wrong and how we could prevent it from happening in the future. Were talking about uber here... it is also just as likely that they are in way over their heads. > As self-driving cars become more prevalent, things like this will happen and it will be tough to walk the line between over-reacting to every incident and killing progress in this area and designing safe cars that don't make the problem worse. When I read this bit I felt compelled to respond - the life of one victim is too many. In the early days of aviation people knew the risks... But this wasn't the test pilot who died this was a woman walking down the road. I look forward to the NTSB report - they tend to be the adults in these situations.
- 9y ago
shakna 9y ago
> From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing and would not have been at-fault in the collision. But self-driving cars have more sensors and, in theory, more capabilities to prevent something like this, and we absolutely should expect them to do better. Intel and Volvo tested against the terrible dashcam footage. If Uber hadn't disabled the XC90's safety systems, this wouldn't have happened. [0] The systems today can do better. [0] https://www.bloomberg.com/news/articles/2018-03-26/uber-disabled-volvo-suv-s-standard-safety-system-before-fatality https://www.bloomberg.com/news/articles/2018-03-26/uber-disa...

by suresk 9y ago

ars 9y ago

> From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing No, a human would have easily avoided this (unless they were not looking). The video is much darker than what a person would see. Even when you can see the bike on that dark video, the car still does not react!

MBCook 9y ago

Based on what came out of the last few days I don’t think it’s clear at all, I think a human would not have hit the woman. The footage we saw was from a terrible dashcam and did not represent how dark the street actually was. Other people who have driven down the street with cameras show it to be perfectly reasonably lit, with streetlights and everything. Second, the woman was crossing across two lanes, so any driver watching would have already seen her cross the left-hand lane. It’s not like she popped out from behind a tree suddenly. She wasn’t even going that fast. This seems to be a massive failure of Uber’s system as well as a complete failure of the ‘safety’ driver. I don’t think anyone’s overreacting. I think they’re under reacting. I think it’s pretty clear over was heavily negligent. There’s no way they don’t get sued. What comes out at trial is going to be up amazing, I think. I think if this had been a “real” accident that would’ve been almost impossible to avoid, around it WAS impossible to avoid, or one where the human stepped in and it wasn’t good enough… We would be having a discussion we’re like what you’re saying. Maybe if this was what emai I think if this had been a “real“ accident that would’ve been almost impossible to avoid, around it WAS impossible to avoid, or one where the human stepped in and it wasn’t good enough… We would be having a discussion we’re like what you’re saying. Maybe if this was Waymo or one of the other comapnies. But it looks like the dubious honor of first death by a autonomous car was totally unnecessary and preventable.

sokoloff 9y ago
The pedestrian also has a significant contribution of negligence to this collision, IMO.
- mjmahone17 9y ago
  Why? They crossed a 35 mph (so in Tempe this is a “slow” road) at a corner (unmarked corners are also crosswalks, legally) on a well lit road. Where the car was many seconds away when she started crossing. Would a human driver be deemed at fault in a court of law? Usually not (though in this case, maybe) unless they kept going after the collision, or were under the influence. But that’s because we give human drivers a lot of leeway to kill people. Not because it isn’t their fault.
  - bendauphinee 9y ago
    As a pedestrian, it's still our responsibility to make sure approaching vehicles are actually stopped before we start crossing. She did not, or she would be alive.
    - lutorm 9y ago
      It's certainly prudent to do so, and any amount of self-preservation says you should, but is not checking if you're about to be run over actually negligent? Edit: Also, if you as a pedestrian, stand at a 35mph road, at night, and wait for all approaching cars to stop before you start crossing, you will be standing there a very long time indeed.
      - sokoloff 9y ago
        One legal definition of negligence is A failure to behave with the level of care that someone of ordinary prudence would have exercised under the same circumstances. If you've established that doing so is "certainly prudent", then I think it's reasonable that not doing so is negligent.

dontreact 9y ago

"From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing and would not have been at-fault in the collision." The camera footage is A LOT darker than what people have filmed there just a day or two after and is probably not a good representation of what it looked like to the human eye. Even if it was fairly accurate, the car would be HEAVILY and EXTREMELY dangerously overdriving its headlights in a way that humans would not do.

takeda 9y ago

> From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing and would not have been at-fault in the collision. But self-driving cars have more sensors and, in theory, more capabilities to prevent something like this, and we absolutely should expect them to do better. The footage was artificially darkened, or it was extremely crappy dash cam or both. I suspect Uber just tampered with it (to get some sympathy) Someone posted a footage from that place few days later and that is very well lit road. On Uber footage you can see lamps on the sky, yet it is very dark. Also note this dashcam is not what the car uses, and the car is also equipped with LIDAR so it should be able to spot pedestrian even in pitch black. > As self-driving cars become more prevalent, things like this will happen and it will be tough to walk the line between over-reacting to every incident and killing progress in this area and designing safe cars that don't make the problem worse. I think it is prudent to have Uber stop testing on public roads until they can explain how this happened and how they can keep it from happening again. Your are missing the point here. Preventing accidents like this should be thing that AI would excel at, even today. The fact that this failed means that there is something seriously wrong with how Uber is doing it.

Aspyre 9y ago
How would the footage be tampered when it was taken by the police directly from the dashcam itself at the scene? Also, there was no moon on that night and all of the video comparisons that have been put out are low-light cameras. They are not very realistic. Also, noone seems to think it's a possibility that the agencies responsible for the lighting in this area to have made improvements immediately after the inicident before the public speculation started.
- ceejayoz 9y ago
  > all of the video comparisons that have been put out are low-light cameras If a self-driving car can't see 50 feet at night, maybe they should buy some low-light cameras? > Also, noone seems to think it's a possibility that the agencies responsible for the lighting in this area to have made improvements immediately after the inicident before the public speculation started. None of this matters. (I also find it pretty unlikely - the lights shown in Uber's video look plenty close together already to safely light the road. Even on an unlit road, the car's own headlights should've revealed the pedestrian far sooner.) A self-driving car should be able to see (via the car's headlights, infrared, LIDAR, etc.) a safe distance ahead of the car at all times. The car should detect the unsafe condition and refuse to continue - not mow down pedestrians. If it can't cope with a moonless, unlit road, it should not be on that road.

zer00eyz 9y ago

I'm going to take your post apart because I enjoyed it very much... > For all of the theoretical moral questions that have been brought up around self driving cars (the proverbial lever that lets you kill one person instead of a handful, for example), it's interesting/inevitable that this one came up first. Your talking about the "trolley problem" and it is a ethical question as opposed to moral. This is literally a fuckup, bad programing or poor design. We are going to have real ethical concerns because at some point a car is going to have to make a choice pedestrian or occupant - and I know that my elderly father would choose himself if he were in the car, but I might make a different choice.... will the cars ethics ever reflect my own? > I'm sure lots of people will argue that self-driving cars will, in the long run, save a lot of lives, and therefore we can justify some loss of life in the process. They are probably right about self-driving cars saving more lives in the long run, but I don't know that the conclusion follows. It is a tough one, and if there is more loss of life does using a self driving car a bit of fruit off a poison tree. https://en.wikipedia.org/wiki/Eduard_Pernkopf https://en.wikipedia.org/wiki/Eduard_Pernkopf springs to mind when you bring this up. > From what has been released so far, this feels like a case where a human would have had a high likelihood of doing the same thing Except the fact that when you look at footage of the road when driven on with a decent camera recording rather than the crapy low quality and very much artificially dark uber footage this looks to have been very avoidable. If the guy who was supposed to be there to take over had been looking up right then, we might not be talking about this at all... > As someone with a little bit of experience with the machine learning side of things, I understand how this stuff is really hard, and I can imagine a handful of ways the system could have been confused, which makes it all the more important to understand what went wrong and how we could prevent it from happening in the future. Were talking about uber here... it is also just as likely that they are in way over their heads. > As self-driving cars become more prevalent, things like this will happen and it will be tough to walk the line between over-reacting to every incident and killing progress in this area and designing safe cars that don't make the problem worse. When I read this bit I felt compelled to respond - the life of one victim is too many. In the early days of aviation people knew the risks... But this wasn't the test pilot who died this was a woman walking down the road. I look forward to the NTSB report - they tend to be the adults in these situations.

9y ago

shakna 9y ago