5 ms·

I don't believe that partially automated driving, as shown by Autopilot is fit for human use. People will rely on it as if it's fully automated and more acciden

I don't believe that partially automated driving, as shown by Autopilot is fit for human use. People will rely on it as if it's fully automated and more accidents like this will happen. We see this same problem with face recognition software that says an innocent man's face matches a criminal. The cops just accept what the computer says and arrest people because the computer says it's a match. We can't have the machine do stuff for us unless it's superior to human judgment, or the stakes are much lower than ruining lives.

grecy 3y ago
> I don't believe that partially automated driving, as shown by Autopilot is fit for human use. People will rely on it as if it's fully automated and more accidents like this will happen. Keep in mind 125 people are killed on the roads every single day in the US. If all vehicles magically switched to this system overnight and the system only killed 100 people per day, that would be a huge win. The fact some people will be killed by automated driving systems is not enough to ban them. If it was, we would ban all cars. The only important statistic is if those systems kill less people than the current state of affairs.
- jmpman 3y ago
  Continuing your analysis - vehicles magically switched to this system overnight and the system only killed 100 people per day. Huge win. But then you find out that the drivers of those 100 daily accidents were all browsing their phone when the accidents happened. Would we ban phone usage while using autopilot? And if we did ban phone usage while using autopilot, and someone was using their phone… should they go to jail? I expect the 125 people who are killed today have a high factor of alcohol being involved, and our society doesn’t take drunk driving lightly. Those drivers would certainly go to jail. Why would phone use be any more accepted if the death rate suddenly dropped to 124, or, as in your hypothetical situation, 100?
  - wilg 3y ago
    What? Phone usage is already banned while driving, period. Also, along with other Autopilot distractions, it is also detected by the cabin camera and it can kick you from using it. So I guess the answer to your questions are yes and yes?
    - friend_and_foe 3y ago
      So the goal here is not 100% full autonomy?
      - wilg 3y ago
        No clue what we're talking about anymore.
        friend_and_foe 3y ago
        If the goal is full autonomy then people using their phones while the automobile is driving have no impact whatsoever on the accident when that goal is reached, if laws banning distraction suffice for you to solve problems then your goal is not full autonomy, in which case what's the point of any of it? When full autonomy is reached, I expect that 100% of accidents will occur when passengers are busy letting the car do the driving, so that correlation would be meaningless.
      - xboxnolifes 3y ago
        The goal is saving lives.
- friend_and_foe 3y ago
  I used to agree with this, I no longer do. There was an interesting read on HN yesterday coincidentally that articulates a part of my reasoning, https://www.devever.net/~hl/ruthlessness https://www.devever.net/~hl/ruthlessness The difference is that machines have no agency. Imagine a scenario where on roads with 100% human driving, the most of the people that are killed are PCP addled people running naked across freeways where they aren't expected to be, but with self driving you have 50% grandmas walking with babies in a stroller across crosswalks. We don't have to say that one life is more valuable than another to come to the conclusion that the latter is worse, all we have to do is acknowledge the pedestrian's portion of the responsibility for the outcome. With that in mind, your last sentence is false, the aggregate death count is not all that matters, what matters is the situations in which they occur. Now let's think about the link in this comment. We begin to form the rules of crossing a street around the behaviors of machines. Not unreasonable. But then it's just a small step from there to "you should've known how the machine was going to behave because it's behavior is simple and predictable." We wind up in a situation where we absolve the self driving car of all liability when an accident occurs because the only entity involved with agency was the pedestrian. We do this already with train tracks. So with that in mind, the machine systems killing less people is not enough, it has to be so good that the only deaths are suicides, unbelievably bad behavior by those killed, or freak accidents so unlikely that they genuinely shock everyone who learns of them. That's the only way to transition from humans to software operating the machines and be able to justify the ruthlessness with which machines operate themselves.
  - grecy 3y ago
    > Imagine a scenario where on roads with 100% human driving, the most of the people that are killed are PCP addled people running naked across freeways where they aren't expected to be I don't think it's helpful to introduce this scenario that is entirely not true in the real world and therefore doesn't teach us anything. You're saying that a huge number of road deaths are people that were asking for it anyway. Dozens and dozens of people were killed on the road today who were not drunk, who were not on drugs and were not breaking a single road rule. This is the reality of the world. Good people are being killed every day by the way things are now. If we can do anything to reduce that number, we should. ... Another way I like to think about this is from the ad in Victoria where a guy has to pick which of his family members will be killed on the roads [1]. Obviously none of us want any of our family to be killed, we all choose zero. Closer to zero is better than not closer to zero. [1] https://www.youtube.com/watch?v=k2tOye9DKdQ https://www.youtube.com/watch?v=k2tOye9DKdQ
simondotau 3y ago
Most new cars are sold with features roughly in parity with Autopilot. If this was a problem, we’d already see it in pedestrian injury statistics. (You may have a point with respect to systems like Supercruise and Tesla FSD, but that’s not the topic of this article.)
- chroma 3y ago
  Tesla offers three tiers of automated driving: Autopilot. This automatically accelerates, decelerates, and steers on roads. This works on all roads with lane markings, not just limited access freeways. It also obeys speed limit signs. Enhanced autopilot. This is autopilot but will also self drive from on-ramp to off-ramp, changing lanes to pass (and exit the passing lane), taking interchanges to other freeways, and exiting the offramp. Full self driving beta. This will drive on pretty much any street. Early on it was purely a gimmick that required constant intervention, but lately it’s gotten pretty good. My main complaint is that it drives too conservatively, causing impatient drivers to behave erratically around you. Imagine someone driving as if they're taking a driving test. Because of that I tend to drive manually unless I’m on a freeway.
- gnicholas 3y ago
  > Most new cars are sold with features roughly in parity with Autopilot. Really? My understanding is that many new vehicles have lane keep assist and dynamic cruise control, but that they are much more limited than Autopilot. For example, lots of cars have much tighter limits on the curves they will follow. They also don't have the UI that Autopilot has, which shows what cars it detects. This gives more of a sense of full autonomy (although I find it a bit creepy/worrisome when vehicles appear and disappear on the screen for no reason — one would think this would give people pause). There are definitely more expensive vehicles (Cadillac, Mercedes) that have more robust systems, but my sense is that your typical Toyota, Subaru, etc. isn't close to Autopilot.
  - simondotau 3y ago
    Autopilot really is just adaptive cruise control and lane centering. It can't change lanes by itself. It can't take an exit. It won't stop at an intersection. It can't turn corners. These things are blindingly obvious within a few minutes of use. Yes it's more slick than the cruise assist features on your typical Toyota, Subaru etc, but not dramatically so. Some people argue that Tesla customers think they've been sold autonomy. I can't see how. Tesla goes to pains to make you aware of the $12k you would need to spend to get the system that "self drives". You are painfully aware of the $12k you didn't spend.

grecy 3y ago

> I don't believe that partially automated driving, as shown by Autopilot is fit for human use. People will rely on it as if it's fully automated and more accidents like this will happen. Keep in mind 125 people are killed on the roads every single day in the US. If all vehicles magically switched to this system overnight and the system only killed 100 people per day, that would be a huge win. The fact some people will be killed by automated driving systems is not enough to ban them. If it was, we would ban all cars. The only important statistic is if those systems kill less people than the current state of affairs.

jmpman 3y ago
Continuing your analysis - vehicles magically switched to this system overnight and the system only killed 100 people per day. Huge win. But then you find out that the drivers of those 100 daily accidents were all browsing their phone when the accidents happened. Would we ban phone usage while using autopilot? And if we did ban phone usage while using autopilot, and someone was using their phone… should they go to jail? I expect the 125 people who are killed today have a high factor of alcohol being involved, and our society doesn’t take drunk driving lightly. Those drivers would certainly go to jail. Why would phone use be any more accepted if the death rate suddenly dropped to 124, or, as in your hypothetical situation, 100?
- wilg 3y ago
  What? Phone usage is already banned while driving, period. Also, along with other Autopilot distractions, it is also detected by the cabin camera and it can kick you from using it. So I guess the answer to your questions are yes and yes?
  - friend_and_foe 3y ago
    So the goal here is not 100% full autonomy?
    - wilg 3y ago
      No clue what we're talking about anymore.
      - friend_and_foe 3y ago
        If the goal is full autonomy then people using their phones while the automobile is driving have no impact whatsoever on the accident when that goal is reached, if laws banning distraction suffice for you to solve problems then your goal is not full autonomy, in which case what's the point of any of it? When full autonomy is reached, I expect that 100% of accidents will occur when passengers are busy letting the car do the driving, so that correlation would be meaningless.
    - xboxnolifes 3y ago
      The goal is saving lives.
friend_and_foe 3y ago
I used to agree with this, I no longer do. There was an interesting read on HN yesterday coincidentally that articulates a part of my reasoning, https://www.devever.net/~hl/ruthlessness https://www.devever.net/~hl/ruthlessness The difference is that machines have no agency. Imagine a scenario where on roads with 100% human driving, the most of the people that are killed are PCP addled people running naked across freeways where they aren't expected to be, but with self driving you have 50% grandmas walking with babies in a stroller across crosswalks. We don't have to say that one life is more valuable than another to come to the conclusion that the latter is worse, all we have to do is acknowledge the pedestrian's portion of the responsibility for the outcome. With that in mind, your last sentence is false, the aggregate death count is not all that matters, what matters is the situations in which they occur. Now let's think about the link in this comment. We begin to form the rules of crossing a street around the behaviors of machines. Not unreasonable. But then it's just a small step from there to "you should've known how the machine was going to behave because it's behavior is simple and predictable." We wind up in a situation where we absolve the self driving car of all liability when an accident occurs because the only entity involved with agency was the pedestrian. We do this already with train tracks. So with that in mind, the machine systems killing less people is not enough, it has to be so good that the only deaths are suicides, unbelievably bad behavior by those killed, or freak accidents so unlikely that they genuinely shock everyone who learns of them. That's the only way to transition from humans to software operating the machines and be able to justify the ruthlessness with which machines operate themselves.
- grecy 3y ago
  > Imagine a scenario where on roads with 100% human driving, the most of the people that are killed are PCP addled people running naked across freeways where they aren't expected to be I don't think it's helpful to introduce this scenario that is entirely not true in the real world and therefore doesn't teach us anything. You're saying that a huge number of road deaths are people that were asking for it anyway. Dozens and dozens of people were killed on the road today who were not drunk, who were not on drugs and were not breaking a single road rule. This is the reality of the world. Good people are being killed every day by the way things are now. If we can do anything to reduce that number, we should. ... Another way I like to think about this is from the ad in Victoria where a guy has to pick which of his family members will be killed on the roads [1]. Obviously none of us want any of our family to be killed, we all choose zero. Closer to zero is better than not closer to zero. [1] https://www.youtube.com/watch?v=k2tOye9DKdQ https://www.youtube.com/watch?v=k2tOye9DKdQ

simondotau 3y ago

Most new cars are sold with features roughly in parity with Autopilot. If this was a problem, we’d already see it in pedestrian injury statistics. (You may have a point with respect to systems like Supercruise and Tesla FSD, but that’s not the topic of this article.)

chroma 3y ago
Tesla offers three tiers of automated driving: Autopilot. This automatically accelerates, decelerates, and steers on roads. This works on all roads with lane markings, not just limited access freeways. It also obeys speed limit signs. Enhanced autopilot. This is autopilot but will also self drive from on-ramp to off-ramp, changing lanes to pass (and exit the passing lane), taking interchanges to other freeways, and exiting the offramp. Full self driving beta. This will drive on pretty much any street. Early on it was purely a gimmick that required constant intervention, but lately it’s gotten pretty good. My main complaint is that it drives too conservatively, causing impatient drivers to behave erratically around you. Imagine someone driving as if they're taking a driving test. Because of that I tend to drive manually unless I’m on a freeway.
gnicholas 3y ago
> Most new cars are sold with features roughly in parity with Autopilot. Really? My understanding is that many new vehicles have lane keep assist and dynamic cruise control, but that they are much more limited than Autopilot. For example, lots of cars have much tighter limits on the curves they will follow. They also don't have the UI that Autopilot has, which shows what cars it detects. This gives more of a sense of full autonomy (although I find it a bit creepy/worrisome when vehicles appear and disappear on the screen for no reason — one would think this would give people pause). There are definitely more expensive vehicles (Cadillac, Mercedes) that have more robust systems, but my sense is that your typical Toyota, Subaru, etc. isn't close to Autopilot.
- simondotau 3y ago
  Autopilot really is just adaptive cruise control and lane centering. It can't change lanes by itself. It can't take an exit. It won't stop at an intersection. It can't turn corners. These things are blindingly obvious within a few minutes of use. Yes it's more slick than the cruise assist features on your typical Toyota, Subaru etc, but not dramatically so. Some people argue that Tesla customers think they've been sold autonomy. I can't see how. Tesla goes to pains to make you aware of the $12k you would need to spend to get the system that "self drives". You are painfully aware of the $12k you didn't spend.