10 ms·

Try replacing "acting unexpectedly" in your thought process (which superficially I agree with) with the words "acting safely." It remains to be seen if autonom

by sounds 3y ago

Try replacing "acting unexpectedly" in your thought process (which superficially I agree with) with the words "acting safely." It remains to be seen if autonomous driving systems are actually safe. But if the other driver does something that is safe, there's then an onus on the first driver to have accounted for that.

bumby 3y ago
>But if the other driver does something that is safe, there's then an onus on the first driver to have accounted for that. I don't disagree, but in order for the first driver to "account" for the actions of the second, they have to have some reasonable ability to predict what that driver will do. That gets us back to the theory-of-mind question.
- taneq 3y ago
  Are we still talking about a car getting rear ended because it braked? Because you’re meant to leave enough room between you and the car in front to stop safely even if it unexpectedly brakes as hard as possible. Running into the back of a car in front of you (that didn’t just pull out) is always your fault.
  - bumby 3y ago
    I think people are often missing point. Yes, in a rear end collision the fault is almost always the following driver. Having a framework for assigning liability is not the same as having a safety framework. Consider an AV that is consistently brake-checking those behind them due to nuisance alarms. Now I have a harder time predicting what the car in front of me is going to do. Is that a safer or less-safe scenario? Sure, I can mitigate it by giving more trailing distance, but now we've traded traffic flow/congestion for an equal level of safety.
    - shkkmo 3y ago
      > now we've traded traffic flow/congestion for an equal level of safety. If you don't maintain a safe following distance for your speed, you are the one creating the dangerous driving environment. Tailgating is worse for both traffic flow and safety.
      - bumby 3y ago
        This, again, gets to missing the point. If a disproportionate amount of cars are nuisance brake checking, it increases the level of uncertainty in driving behavior. I now have to overcompensate on average to maintain the same level of safety.
        UncleMeat 3y ago
        But you aren't overcompensating. Instead you are driving safely. If sudden braking is so rare that you feel comfortable riding right behind a car in front of you then when they do suddenly brake (which will happen eventually), you are now in a very dangerous situation.
        shkkmo 3y ago
        > I now have to overcompensate on average to maintain the same level of safety. Tailgating is bad, regardless of if people brake check or not. If automous vehicles are what it takes to get you to stop tailgating and follow at a safe distance, then that is just an added bonus.
        bumby 3y ago
        There is no single definition of tailgaiting other than being able to not being able to stop at a reasonable distance. So it is impossible to declare what constitutes tailing, especially in the mixed case of human drivers and robot drivers (who have a reputation for nuisance braking). Why, for example, do you think trainers post "Student Driver" stickers on their cars? It's because it signals the driver may be more unpredictable and people (rightly) tend to give them wide berth. You're essentially advocating that everyone treat everyone else (and every robot) as a student driver. That's fine for a dichotomous safety mindset, but other people would prefer to recognize the tradeoffs with that approach. Or maybe you're just deliberately bent on misunderstanding my point, I can't read your mind :)
    - zehaeva 3y ago
      To add onto the sibling's point, the "safe following distance" has a rule of thumb of "3 seconds behind". At 65MPH, I'm assuming you're in the US, is approximately 300 feet. I'm willing to bet that's around 10 times what you were considering as a safe following distance in your head, and probably still 5 times more than what you were picturing for the safe distance behind a brake checking AV.
      - bombcar 3y ago
        The problem with the “safe distance” (my new car has it built into the cruise control) is that it’s more than large enough for a vehicle to merge into, which repeatedly happens. As more and more cars default to this safely, however, it’ll start to equal out.
    - taneq 3y ago
      If it were doing it consistently then that would be less of an issue. The problem is that complex systems make decisions for complex reasons and are very difficult to predict as a result. That said, if you routinely tailgate the driver in front on the assumption that nothing will go wrong then you've chosen to accept the consequences when an otter runs out in front of them (hey I didn't see that coming either) and they suddenly limit brake and you're sitting in their back seat. Or a steering tie breaks in a classic car coming the other way, which hits the car in front of you head on and now they're stationary. The question of how much additional caution (in terms of lower speed limits, longer following distances etc.) is optimal in terms of overall QALYs is, I feel, vastly under-considered and under-discussed.
- strangattractor 3y ago
  The rules for every vehicle I have ever operated on land, air and water require the overtaking vehicle to maintain separation.
  - bumby 3y ago
    I’m not sure this is the blanket case. Hot air balloons, for example, get right-of-way regardless, on the assumption they have less maneuverability. Weird edge case, I know, but just throwing it out there to underscore the danger of absolute statements.
    - strangattractor 3y ago
      I have to admit I have not piloted a ballon but even a sailing vessel over taking a power boat has to avoid the vessel in front. I would also question the rationality of operating an aircraft you cannot steer:)
    - mannykannot 3y ago
      This discussion appears to have run off the road with an unexpected turn.
    - ang_cire 3y ago
      But who gets the right of way between 2 hot air balloons? Funny story, I actually "crashed" in a hot air balloon as a kid, when the hill we were landing on had a draft running up over it that caught the balloon after the basket had touched down, and dragged us along the ground sideways for a good quarter mile.
      - bumby 3y ago
        Haha well in that case, I’d say the ground had right-of-way :-)
      - shagie 3y ago
        If I recall correctly, the lower one has right of way because visibility from lower to higher is blocked compared to higher to lower. It is easier for the one further up in the air column to spot and react to the lower one. Going up is also likely a less dangerous proposition than going down. This is half remembered from a Snowmass balloon rally conversation.
- mannykannot 3y ago
  None of these three cases involved the Waymo car behaving in ways that are not that uncommon among human drivers, and our theory of mind does not make us nearly-infallible predictors of what another driver is going to do. Your objection becomes essentially hypothetical unless these cars are behaving in ways that are both outside of the norms established by the driving public, and dangerous.
  - bumby 3y ago
    That’s true, but also one of the selling points of some AI tasks. As a non hypothetical example, the DoD hired a company to train a software dogfighting simulator with RL. What surprised the pilots was how many “best practices” it broke and how it essentially behaved like a pilot with a death wish. Possibly good in war, maybe not so good on a public road.
    - dcow 3y ago
      Your comparison is RL dogfighting? I think you need to brush up on recent autonomous driving systems…
      - bumby 3y ago
        Do you not feel they are related? In general, aviation tech tends to outpace automotive tech (I’ve worked in both). Maybe you could enlighten me?
        dcow 3y ago
        Modern AVs are not driving like they have a death wish by any stretch of the imagination (and these systems are not developed using raw RL). They are driving safer than humans. Any concern that they are not following established best practices is entirely unfounded and strictly grounded in FUD.
        bumby 3y ago
        >They are driving safer than humans. I don’t think you can make this strong of a claim with the available data. Likewise, someone can’t make strong claim in the opposite direction. The best data I’ve seen is from NTSB investigations, and it clearly shows some very dangerous behavior. But it’s just a snapshot of data. And I think you’ve taken away the wrong point. It was about unexpected behavior, not “driving with a death wish.” The ability of people to consistently miss/twist the point to fit their own predetermined viewpoint is tiresome.
strangattractor 3y ago
From the article : "But it’s not enough to answer the most important safety question: whether Waymo’s technology makes fatal crashes less likely." Wouldn't the most important safety question be "whether Waymo’s technology makes fatal crashes MORE likely?" Why assume Autonomous Vehicles have to perform better? Waymo less fatalities == Win Waymo same fatalities == Draw Waymo more fatalities == Bad Waymo
- nkingsy 3y ago
  because everyone thinks they're above average. If you're only better than the below average drivers, then everyone will think they're better off driving themselves even if that's not true. Also, we've accepted and made legal frameworks around the concept that people sometimes kill each other with cars. Robots killing people with cars does not benefit from this carve out.
  - strangattractor 3y ago
    It is only a matter of time before that changes.
  - throwaway2037 3y ago
    I am a terrible driver -- so easily distracted. Almost zero men I have met in my life are willing to admit the same. (However, some women in my life do.) As a result, I try to avoid driving as much as possible and use mass transit.
  - whatever1 3y ago
    The average data include literal drug addicts and people driving under alcohol influence or people who drive distracted while texting. I am neither, hence I will not trust any system that does not perform at least as well as I do. The bar is much higher than the average.
    - throw10920 3y ago
      > hence I will not trust any system that does not perform at least as well as I do. ...which is a pretty big fallacy and, if implemented as law (which I am working to make sure it is not), will get a lot of people killed. Literal drug addicts, people driving under the influence of alcohol, and people driving distracted are driving right now. Even AV that performs at the level of the median driver will massively reduce the number of those people actually making driving decisions, and therefore directly reduce road deaths. Why? Because most accidents and deaths are caused by drivers performing at the low end of the spectrum. In the US, a full third of accidents are caused by drunk driving - which would be virtually nonexistent with AVs ("virtually" because the AVs aren't going to be perfect and are going to need manual intervention). The bar is literally the average, because if everyone drove average, deaths would plummet. And, unlike the American public, it's massively easier to improve AV driving behavior than average human driver behavior. A system that starts out eliminating the lower end of driving performance while improving over time is a deal too good to turn down if you care about human life.
lukas099 3y ago
You're right. Also, if AI drivers are safer, there will be knock-on effects of making humans drive more safely as well.
throwaway2037 3y ago
It remains to be seen if autonomous driving systems are actually safe. What a strange, leading expression. The phrase "[i]t remains to be seen if" is almost meaningless and highly editorial. Alternative example: It remains to be seen if microplastics will cause higher rates of cancer. It remains to be seen if organic peanut butter will cause lower rates of cancer. What is your reaction to this data published by Waymo? Do you agree that many would reasonably conclude that this particular AV system was safe in these cities? I say yes. I write this as someone who is cautiously optimistic about AV. Waymo seems genuinely lower on the hyper compared to other AV companies. I hope they continue this path of higher transparency to encourage other AV companies to do the same.

by sounds 3y ago

bumby 3y ago

>But if the other driver does something that is safe, there's then an onus on the first driver to have accounted for that. I don't disagree, but in order for the first driver to "account" for the actions of the second, they have to have some reasonable ability to predict what that driver will do. That gets us back to the theory-of-mind question.

taneq 3y ago
Are we still talking about a car getting rear ended because it braked? Because you’re meant to leave enough room between you and the car in front to stop safely even if it unexpectedly brakes as hard as possible. Running into the back of a car in front of you (that didn’t just pull out) is always your fault.
- bumby 3y ago
  I think people are often missing point. Yes, in a rear end collision the fault is almost always the following driver. Having a framework for assigning liability is not the same as having a safety framework. Consider an AV that is consistently brake-checking those behind them due to nuisance alarms. Now I have a harder time predicting what the car in front of me is going to do. Is that a safer or less-safe scenario? Sure, I can mitigate it by giving more trailing distance, but now we've traded traffic flow/congestion for an equal level of safety.
  - shkkmo 3y ago
    > now we've traded traffic flow/congestion for an equal level of safety. If you don't maintain a safe following distance for your speed, you are the one creating the dangerous driving environment. Tailgating is worse for both traffic flow and safety.
    - bumby 3y ago
      This, again, gets to missing the point. If a disproportionate amount of cars are nuisance brake checking, it increases the level of uncertainty in driving behavior. I now have to overcompensate on average to maintain the same level of safety.
      - UncleMeat 3y ago
        But you aren't overcompensating. Instead you are driving safely. If sudden braking is so rare that you feel comfortable riding right behind a car in front of you then when they do suddenly brake (which will happen eventually), you are now in a very dangerous situation.
      - shkkmo 3y ago
        > I now have to overcompensate on average to maintain the same level of safety. Tailgating is bad, regardless of if people brake check or not. If automous vehicles are what it takes to get you to stop tailgating and follow at a safe distance, then that is just an added bonus.
        bumby 3y ago
        There is no single definition of tailgaiting other than being able to not being able to stop at a reasonable distance. So it is impossible to declare what constitutes tailing, especially in the mixed case of human drivers and robot drivers (who have a reputation for nuisance braking). Why, for example, do you think trainers post "Student Driver" stickers on their cars? It's because it signals the driver may be more unpredictable and people (rightly) tend to give them wide berth. You're essentially advocating that everyone treat everyone else (and every robot) as a student driver. That's fine for a dichotomous safety mindset, but other people would prefer to recognize the tradeoffs with that approach. Or maybe you're just deliberately bent on misunderstanding my point, I can't read your mind :)
  - zehaeva 3y ago
    To add onto the sibling's point, the "safe following distance" has a rule of thumb of "3 seconds behind". At 65MPH, I'm assuming you're in the US, is approximately 300 feet. I'm willing to bet that's around 10 times what you were considering as a safe following distance in your head, and probably still 5 times more than what you were picturing for the safe distance behind a brake checking AV.
    - bombcar 3y ago
      The problem with the “safe distance” (my new car has it built into the cruise control) is that it’s more than large enough for a vehicle to merge into, which repeatedly happens. As more and more cars default to this safely, however, it’ll start to equal out.
  - taneq 3y ago
    If it were doing it consistently then that would be less of an issue. The problem is that complex systems make decisions for complex reasons and are very difficult to predict as a result. That said, if you routinely tailgate the driver in front on the assumption that nothing will go wrong then you've chosen to accept the consequences when an otter runs out in front of them (hey I didn't see that coming either) and they suddenly limit brake and you're sitting in their back seat. Or a steering tie breaks in a classic car coming the other way, which hits the car in front of you head on and now they're stationary. The question of how much additional caution (in terms of lower speed limits, longer following distances etc.) is optimal in terms of overall QALYs is, I feel, vastly under-considered and under-discussed.
strangattractor 3y ago
The rules for every vehicle I have ever operated on land, air and water require the overtaking vehicle to maintain separation.
- bumby 3y ago
  I’m not sure this is the blanket case. Hot air balloons, for example, get right-of-way regardless, on the assumption they have less maneuverability. Weird edge case, I know, but just throwing it out there to underscore the danger of absolute statements.
  - strangattractor 3y ago
    I have to admit I have not piloted a ballon but even a sailing vessel over taking a power boat has to avoid the vessel in front. I would also question the rationality of operating an aircraft you cannot steer:)
  - mannykannot 3y ago
    This discussion appears to have run off the road with an unexpected turn.
  - ang_cire 3y ago
    But who gets the right of way between 2 hot air balloons? Funny story, I actually "crashed" in a hot air balloon as a kid, when the hill we were landing on had a draft running up over it that caught the balloon after the basket had touched down, and dragged us along the ground sideways for a good quarter mile.
    - bumby 3y ago
      Haha well in that case, I’d say the ground had right-of-way :-)
    - shagie 3y ago
      If I recall correctly, the lower one has right of way because visibility from lower to higher is blocked compared to higher to lower. It is easier for the one further up in the air column to spot and react to the lower one. Going up is also likely a less dangerous proposition than going down. This is half remembered from a Snowmass balloon rally conversation.
mannykannot 3y ago
None of these three cases involved the Waymo car behaving in ways that are not that uncommon among human drivers, and our theory of mind does not make us nearly-infallible predictors of what another driver is going to do. Your objection becomes essentially hypothetical unless these cars are behaving in ways that are both outside of the norms established by the driving public, and dangerous.
- bumby 3y ago
  That’s true, but also one of the selling points of some AI tasks. As a non hypothetical example, the DoD hired a company to train a software dogfighting simulator with RL. What surprised the pilots was how many “best practices” it broke and how it essentially behaved like a pilot with a death wish. Possibly good in war, maybe not so good on a public road.
  - dcow 3y ago
    Your comparison is RL dogfighting? I think you need to brush up on recent autonomous driving systems…
    - bumby 3y ago
      Do you not feel they are related? In general, aviation tech tends to outpace automotive tech (I’ve worked in both). Maybe you could enlighten me?
      - dcow 3y ago
        Modern AVs are not driving like they have a death wish by any stretch of the imagination (and these systems are not developed using raw RL). They are driving safer than humans. Any concern that they are not following established best practices is entirely unfounded and strictly grounded in FUD.
        bumby 3y ago
        >They are driving safer than humans. I don’t think you can make this strong of a claim with the available data. Likewise, someone can’t make strong claim in the opposite direction. The best data I’ve seen is from NTSB investigations, and it clearly shows some very dangerous behavior. But it’s just a snapshot of data. And I think you’ve taken away the wrong point. It was about unexpected behavior, not “driving with a death wish.” The ability of people to consistently miss/twist the point to fit their own predetermined viewpoint is tiresome.

strangattractor 3y ago

From the article : "But it’s not enough to answer the most important safety question: whether Waymo’s technology makes fatal crashes less likely." Wouldn't the most important safety question be "whether Waymo’s technology makes fatal crashes MORE likely?" Why assume Autonomous Vehicles have to perform better? Waymo less fatalities == Win Waymo same fatalities == Draw Waymo more fatalities == Bad Waymo

nkingsy 3y ago
because everyone thinks they're above average. If you're only better than the below average drivers, then everyone will think they're better off driving themselves even if that's not true. Also, we've accepted and made legal frameworks around the concept that people sometimes kill each other with cars. Robots killing people with cars does not benefit from this carve out.
- strangattractor 3y ago
  It is only a matter of time before that changes.
- throwaway2037 3y ago
  I am a terrible driver -- so easily distracted. Almost zero men I have met in my life are willing to admit the same. (However, some women in my life do.) As a result, I try to avoid driving as much as possible and use mass transit.
- whatever1 3y ago
  The average data include literal drug addicts and people driving under alcohol influence or people who drive distracted while texting. I am neither, hence I will not trust any system that does not perform at least as well as I do. The bar is much higher than the average.
  - throw10920 3y ago
    > hence I will not trust any system that does not perform at least as well as I do. ...which is a pretty big fallacy and, if implemented as law (which I am working to make sure it is not), will get a lot of people killed. Literal drug addicts, people driving under the influence of alcohol, and people driving distracted are driving right now. Even AV that performs at the level of the median driver will massively reduce the number of those people actually making driving decisions, and therefore directly reduce road deaths. Why? Because most accidents and deaths are caused by drivers performing at the low end of the spectrum. In the US, a full third of accidents are caused by drunk driving - which would be virtually nonexistent with AVs ("virtually" because the AVs aren't going to be perfect and are going to need manual intervention). The bar is literally the average, because if everyone drove average, deaths would plummet. And, unlike the American public, it's massively easier to improve AV driving behavior than average human driver behavior. A system that starts out eliminating the lower end of driving performance while improving over time is a deal too good to turn down if you care about human life.

lukas099 3y ago

You're right. Also, if AI drivers are safer, there will be knock-on effects of making humans drive more safely as well.

throwaway2037 3y ago

It remains to be seen if autonomous driving systems are actually safe. What a strange, leading expression. The phrase "[i]t remains to be seen if" is almost meaningless and highly editorial. Alternative example: It remains to be seen if microplastics will cause higher rates of cancer. It remains to be seen if organic peanut butter will cause lower rates of cancer. What is your reaction to this data published by Waymo? Do you agree that many would reasonably conclude that this particular AV system was safe in these cities? I say yes. I write this as someone who is cautiously optimistic about AV. Waymo seems genuinely lower on the hyper compared to other AV companies. I hope they continue this path of higher transparency to encourage other AV companies to do the same.