13 ms·

>Waymo getting hit from behind, which is the other driver’s fault even if the Waymo acted “unexpectedly”. Yes, but...there is something else to be said here. O

by bumby 3y ago

>Waymo getting hit from behind, which is the other driver’s fault even if the Waymo acted “unexpectedly”. Yes, but...there is something else to be said here. One of the things we have evolved to do, without necessarily appreciating it, is to intuit the behavior of other humans through the theory-of-mind. If AVs consistent act "unexpectedly", this injects a lot more uncertainty into the system, especially when interacting with other humans. "Acting unexpectedly" is one of the aspects that makes dealing with mentally ill people anxiety-producing. I don't think most of us would automatically want to share the roads with a bunch of mentally ill drivers, even if, statistically, they were better than neurotypical drivers. There's something to be said about these scenarios regarding trust being derived from understanding what someone else is likely thinking. Edit: the other aspect that needs to be said is that tech in society is governed by policy. People don't generally just accept policy based on statistical arguments. If you think that you can expect people to accept policies that allow AVs without addressing the trust issue, it might be a painful ride.

closewith 3y ago
> I don't think most of us would automatically want to share the roads with a bunch of mentally ill drivers, even if, statistically, they were better than neurotypical drivers. I've got some bad news for you.
- kelseyfrog 3y ago
  Is it that 1 in 5 adults in the US live with mental illness[1]? 1. https://www.nimh.nih.gov/health/statistics/mental-illness https://www.nimh.nih.gov/health/statistics/mental-illness
sounds 3y ago
Try replacing "acting unexpectedly" in your thought process (which superficially I agree with) with the words "acting safely." It remains to be seen if autonomous driving systems are actually safe. But if the other driver does something that is safe, there's then an onus on the first driver to have accounted for that.
- bumby 3y ago
  >But if the other driver does something that is safe, there's then an onus on the first driver to have accounted for that. I don't disagree, but in order for the first driver to "account" for the actions of the second, they have to have some reasonable ability to predict what that driver will do. That gets us back to the theory-of-mind question.
  - taneq 3y ago
    Are we still talking about a car getting rear ended because it braked? Because you’re meant to leave enough room between you and the car in front to stop safely even if it unexpectedly brakes as hard as possible. Running into the back of a car in front of you (that didn’t just pull out) is always your fault.
    - bumby 3y ago
      I think people are often missing point. Yes, in a rear end collision the fault is almost always the following driver. Having a framework for assigning liability is not the same as having a safety framework. Consider an AV that is consistently brake-checking those behind them due to nuisance alarms. Now I have a harder time predicting what the car in front of me is going to do. Is that a safer or less-safe scenario? Sure, I can mitigate it by giving more trailing distance, but now we've traded traffic flow/congestion for an equal level of safety.
      - shkkmo 3y ago
        > now we've traded traffic flow/congestion for an equal level of safety. If you don't maintain a safe following distance for your speed, you are the one creating the dangerous driving environment. Tailgating is worse for both traffic flow and safety.
        bumby 3y ago
        This, again, gets to missing the point. If a disproportionate amount of cars are nuisance brake checking, it increases the level of uncertainty in driving behavior. I now have to overcompensate on average to maintain the same level of safety.
rudasn 3y ago
For sure. As a driver, one of my main inputs is the other drivers overall driving behaviour that I'm aware of. A good indicator of their next move is their previous move.
uluyol 3y ago
I don't think this applies to any of the incidents mentioned in the article.
- __loam 3y ago
  In fact, if you don't have enough room to react to "unexpected" behavior, you are at fault lol.
  - trgn 3y ago
    When a human driver must emergency break for a downed branch, it'd ok. When an AI does it, it's unexpected and needs to be hyperanalyzed. I swear, the trolley-car problem is absurd, it's poisoned all debate. 99% of crashes is people not being able to stop in time because people don't drive defensively and can't stop in time when they are called to do so.
    - bumby 3y ago
      >it's unexpected and needs to be hyperanalyzed There's a good reason for this. It's because the human can be interrogated into what was going through their mind whereas many ML models cannot. That means we can't ascertain if the ML accident is part of a latent issue that may rear its ugly head again (or in a slightly different manner) or just a one-off. That is the original point: a theory-of-mind is important to risk management. That means we will struggle to mitigate the risk if we don't "hyperanalyze" it.
      - shagie 3y ago
        Cory Doctorow - Car Wars https://web.archive.org/web/20170301224942/http://this.deakin.edu.au/culture/car-wars https://web.archive.org/web/20170301224942/http://this.deaki... (Linking to the web.archive version because the graphics are better / more understandable when in the context of some of the text) Chapter 6 is the most relevant here, but it's all a thought provoking story.
      - parineum 3y ago
adwn 3y ago
Braking to avoid hitting an obstacle (like that tree branch in the first example) is hardly "acting unexpectedly". > "Acting unexpectedly" is one of the aspects that makes dealing with mentally ill people anxiety-producing. Wat.
- bumby 3y ago
  But this certainly could be construed as unexpected: >"In August, a Waymo at an intersection “began to proceed forward” but then “slowed to a stop” and was hit from behind by an SUV." >"Wat." Are you saying you don't understand why unexpected behavior causes anxiety? It's a pretty well documented effect, from rats to humans.
  - robertlagrant 3y ago
    It was: > > "Acting unexpectedly" is one of the aspects that makes dealing with mentally ill people anxiety-producing. > Wat. The "Wat" probably refers to the fact that this seems unrelated. Dealing with mentally ill people is anxiety-inducing because they act unexpectedly... so what? Lots of things are unexpected. They shouldn't all be drawn into the analogy that says "well, that thing, plus a load of other things, can induce anxiety, therefore that thing should be tarnished with the same brush as all of those things." People drive in unexpected ways all the time. Of all the criticisms to level at them, "the thing you're doing plus a load of other stuff can induce anxiety" probably isn't top of the list.
    - bumby 3y ago
      I am not painting them with the same brush, I'm drawing an analogy to help people understand the context better. In this case, public policy will dictate to what extent AVs are allowed on public roadways. That, in turn, is dictated by trust. I'm pointing out that "trust" may be incompatible with "unpredictability." I'm not sure what throughline you're drawing, but you seem overly hung up on the use of the word "anxiety," and it's causing you to miss the real point. So to put a finer point on it, people need to acknowledge that public trust is necessary to wide-scale adoption of AV tech. Plenty of psychological research shows how we aren't intuitively wired to understand statistics. So all the bleating about statistics may be necessary, but not sufficient, to get wide-scale adoption of AVs on public roadways.
seandoe 3y ago
I would imagine that, with more experience, anticipating the (more consistent) actions of a machine would be easier than anticipating the actions of an unknown human in an unknown state.
- mrguyorama 3y ago
  The entire point of this line of discussion is that an ML based system with extremely weird and unexpected failure conditions and failure states ISN'T "more consistent" than a human who might follow more closely than physics says but otherwise is ACTUALLY predictable because they have a mind that we have evolved to predict. ML having completely unpredictable failure modes is like the entire case against putting them anywhere. What would you call a vision system that mis-identifies a stop sigh because of a couple unrelated lines painted on it, other than "unpredictable"?
shadowgovt 3y ago
Quite frankly, the vehicle in front of you is allowed to stop at any time for any reason (legally and practically, as front vehicle has better visibility on road conditions than follower), and it is always incumbent upon the driver to the rear to leave room. If humans can't do that, the solution is probably more automation.
- bumby 3y ago
  Has the nuisance braking problem been completetly solved? If not, I don't know that I'd agree that more automation is necessarily the answer. More good automation, maybe, but there's a logical jump there. The Uber fatality from years back showed that the software used "action suppression" to mitigate nuisance braking. The idea that that would be acceptable on a safety-critical software application should give us pause to consider that more automation is the knee-jerk solution.
  - shadowgovt 3y ago
    "Nuisance braking" is, like jaywalking, a phrase that prioritizes one party's use of the road over another party's. The best policy is still to leave the vehicle in front enough room to brake for any reason. Mostly because "nuisance braking" hasn't been solved in humans either (ever been behind someone who panicked when they realized they were falling asleep behind the wheel? I have.)
    - bumby 3y ago
      I think you are mischaracterizing the problem. Nuisance braking seems to be far more prevalent in AVs than in humans, partly because image classification has more uncertainty. Now, couple that with: 1) a general approach of "when in doubt, brake" and the hacky workaround to suppress braking if it's occurring too much, and you've got a bad nuisance braking problem that's primed for a safety incident. But again, this is disjointed from reality. I'd agree that if we all left the recommended 4.5 seconds of stopping distance, we'd all be safer. But that's not how our roads were built and that's not how humans drive. You're tilting at windmills here to make a point that doesn't need to be made because it doesn't apply to reality.
Justsignedup 3y ago
> I don't think most of us would automatically want to share the roads with a bunch of mentally ill drivers, even if, statistically, they were better than neurotypical drivers. I'm not scared of mentally ill drivers. I'm scared of rich 16 year olds. I'm scared of drunk drivers. I'm scared of drivers sitting so low they can't see most of what is happening around them. I'm scared of drivers having seizures while driving (my mom was hit a while ago by a man who lost his license due to seizures, and still refused to stop driving). I'm scared of drivers who drive without a license because "fuck them, I drive when I want to". I'm scared of people mixing up gas and break pedals (just got hit by one), in cars which can go 0-60 in 2.6 seconds weighing 6000lbs.
- Dalewyn 3y ago
  >I'm scared of people mixing up gas and break pedals This specific problem is just as much a design flaw as it is a PEBCAC issue. You have two very similar pedals that perform polar opposite functions right next to each other, and they are both operated by the same foot. I'm surprised this isn't a bigger problem.
  - bumby 3y ago
    I had to look up the 'PEBKAC' acronym, but I think you allude to the problem of human factors engineering. It's commonplace in aerospace, where safety-critical, time-sensitive decisions must be made, and humans are in the loop. I would extend this to autonomous driving systems, particularly when you expand the system boundaries beyond the car itself. Humans are part of that human-car-environment system, whether as pedestrians, passengers, or other drivers and we should give them consideration.
  - Justsignedup 3y ago
    an average of 44 per day in the US. So... common, but relatively, uncommon. And apparently there is software that can help mitigate this.
- throwaway2037 3y ago
  Slight trolling here: That is a lot of scared. Do you also drive wearing a helmet and protective body suit? man who lost his license due to seizures, and still refused to stop driving If I have to guess, probably the man's livelihood depends upon driving to a job. That is probably why he kept driving.
UncleMeat 3y ago
Ordinary neurotypical drivers act "unexpectedly" on the road all the time. I know that I would brake if I saw a downed branch. And people do much much much more. Suddenly change lanes on the highway with no signaling? Check. Brake suddenly because you almost missed your turn without looking to see if anybody was following close behind? Check. Drive over non-lane portions of the road because you were late seeing your highway exit? Check. Swerve suddenly because you dropped your phone between the seats while texting? Check. I've seen people reverse up a highway onramp.
- wddkcs 3y ago
  I think both points are true- people act unexpectedly all the time, so much so that we've come to expect the unexpected, by the sophistication of our theory of mind. I live in a city where erratic driving is commonplace, but that's in relation to the legal norms. New local norms become accepted, which you can anticipate once you adapt. Some of these 'norms' are handy time savers, others are incredibly dangerous and result in frequent accidents, but persist anyway. How will AI drivers navigate these 'cultural' differences? Will they insist on following the letter of the law (presumably, for liability) or will they adapt to local practices and potentially lower their overall collision rate with humans driving poorly. Interesting near term question.
  - droopyEyelids 3y ago
    You’re being selectively reductive. The unexpected is, by definition, what was not predicted.
    - pastage 3y ago
      The point is unexpected is supposed to be in norm, self driving cars can not tell what a norm is. The idiom is not about the word unexpected, more about the relatioship between chaos and order
- jjav 3y ago
  > Ordinary neurotypical drivers act "unexpectedly" on the road all the time. "all the time" means it is not unexpected. Humans do mistakes and omissions that you've subconsciously learned that humans do, so you can adjust to deal with that. The difficulty with AI is that it behaves like a completely alien being, making errors that are alien and unpredictable to a human driver.
waythenewsgoes 3y ago
Others may not necessarily agree, but at least anecdotally, a sizeable portion of drivers I see make all kinds of mistakes (law of averages dictates more of them are so called "neurotypical" than not no?). "Acting Unexpectedly" can often mean following the actual laws and general guidelines for safe and/or defensive driving. I would hazard a guess that sometimes doing the intuitive thing is, in reality, unsafe and/or against the law. If the car does this in 99% of circumstances, and still gets rear-ended, who is really the problem here?
- lukas099 3y ago
  My wife gets triggered every time I drive 'only' the speed limit.
  - bobsmooth 3y ago
    Driving the speed limit with people whizzing past you is more dangerous than following the speed of everyone around you.
    - UncleMeat 3y ago
      If somebody is whizzing past you and you are going the speed limit then they aren't going 5mph faster than you. I've never been on a road where everybody is consistently speeding by so much that it could be perceived as "whizzing."
    - lukas099 3y ago
      I've heard that. In driving safety class they taught us that the speed limit is safer. I tried to look up a credible source either way and came up blank. Do you have one? I go by feel. If it feels much safer to speed, I do. Otherwise I'm in the right hand lane, usually a safe stopping distance behind a big rig for aero gains.
      - dcow 3y ago
        I don’t think you get the aero gains at safe stopping distances, though.
lukas099 3y ago
Humans are nothing if not adaptable. We will adjust our expectations.
- bumby 3y ago
  True, but that doesn't mean it's always a better result. I've adjusted my expectations that software on my smart tv will be glitchy and have interface errors with a lot of apps, but that doesn't mean that's the only or best way to program a system. We adjust to poor quality all the time, but I prefer not to lower my expectations on a safety-critical system.
throwaway2037 3y ago
These two phrases are strange to me in the context of OP: intuit the behavior of other humans through the theory-of-mind and: If AVs consistently act "unexpectedly" Are you implying that the three specific incidents quoted by the OP are acting unexpectedly? I hope not. Most good, aware drivers would do the same. If you disagree, please provide alternative driving actions that you consider expected. And, would the safety outcomes be better? Unlikely.
- bumby 3y ago
  No, I'm making a broader claim about the nature of AI in safety critical domains. I am also saying we should be careful about drawing strong conclusions from a sample size of n=3 from a potentially questionable dataset.
Nashenas 3y ago
Getting hit from behind in all of these scenarios means the people behind did not maintain proper distancing between vehicles. Especially the 3 car pile up. That's why you're supposed to leave one car length for every 10mph you're going, so you have enough space to cancel out your momentum. What if instead of a branch it was a dog or a child? And I've been to MANY places in the US where people drive terribly and completely unexpectedly. I've seen people put on a left turn signal then go into the RIGHT lane. I've seen someone pass on a one lane overpass, where the car in front was towing a heavy load What's good about these cars is they will ALL drive consistently, and the more people get used to them the easier it well be to drive with them because you will know how waymo cars react.
frogpelt 3y ago
Which raises questions for me about how traffic behaves if 25-50% of cars are self-driving. What "feedback loops" might occur? I'd be interested to see large scale tests that demonstrate how self-driving cars deal with each other in high traffic areas.
rad_gruchalski 3y ago
> Yes, but...there is something else to be said here. One of the things we have evolved to do, without necessarily appreciating it, is to intuit the behavior of other humans through the theory-of-mind. One of the first thing my driving instructor told me was: never trust anyone on the road except yourself. That’s why you see people waiting for a car with the indicator on to visually hint they really intended to turn.

by bumby 3y ago

closewith 3y ago

> I don't think most of us would automatically want to share the roads with a bunch of mentally ill drivers, even if, statistically, they were better than neurotypical drivers. I've got some bad news for you.

kelseyfrog 3y ago
Is it that 1 in 5 adults in the US live with mental illness[1]? 1. https://www.nimh.nih.gov/health/statistics/mental-illness https://www.nimh.nih.gov/health/statistics/mental-illness

sounds 3y ago

Try replacing "acting unexpectedly" in your thought process (which superficially I agree with) with the words "acting safely." It remains to be seen if autonomous driving systems are actually safe. But if the other driver does something that is safe, there's then an onus on the first driver to have accounted for that.

bumby 3y ago
>But if the other driver does something that is safe, there's then an onus on the first driver to have accounted for that. I don't disagree, but in order for the first driver to "account" for the actions of the second, they have to have some reasonable ability to predict what that driver will do. That gets us back to the theory-of-mind question.
- taneq 3y ago
  Are we still talking about a car getting rear ended because it braked? Because you’re meant to leave enough room between you and the car in front to stop safely even if it unexpectedly brakes as hard as possible. Running into the back of a car in front of you (that didn’t just pull out) is always your fault.
  - bumby 3y ago
    I think people are often missing point. Yes, in a rear end collision the fault is almost always the following driver. Having a framework for assigning liability is not the same as having a safety framework. Consider an AV that is consistently brake-checking those behind them due to nuisance alarms. Now I have a harder time predicting what the car in front of me is going to do. Is that a safer or less-safe scenario? Sure, I can mitigate it by giving more trailing distance, but now we've traded traffic flow/congestion for an equal level of safety.
    - shkkmo 3y ago
      > now we've traded traffic flow/congestion for an equal level of safety. If you don't maintain a safe following distance for your speed, you are the one creating the dangerous driving environment. Tailgating is worse for both traffic flow and safety.
      - bumby 3y ago
        This, again, gets to missing the point. If a disproportionate amount of cars are nuisance brake checking, it increases the level of uncertainty in driving behavior. I now have to overcompensate on average to maintain the same level of safety.

rudasn 3y ago

For sure. As a driver, one of my main inputs is the other drivers overall driving behaviour that I'm aware of. A good indicator of their next move is their previous move.

uluyol 3y ago

I don't think this applies to any of the incidents mentioned in the article.

__loam 3y ago
In fact, if you don't have enough room to react to "unexpected" behavior, you are at fault lol.
- trgn 3y ago
  When a human driver must emergency break for a downed branch, it'd ok. When an AI does it, it's unexpected and needs to be hyperanalyzed. I swear, the trolley-car problem is absurd, it's poisoned all debate. 99% of crashes is people not being able to stop in time because people don't drive defensively and can't stop in time when they are called to do so.
  - bumby 3y ago
    >it's unexpected and needs to be hyperanalyzed There's a good reason for this. It's because the human can be interrogated into what was going through their mind whereas many ML models cannot. That means we can't ascertain if the ML accident is part of a latent issue that may rear its ugly head again (or in a slightly different manner) or just a one-off. That is the original point: a theory-of-mind is important to risk management. That means we will struggle to mitigate the risk if we don't "hyperanalyze" it.
    - shagie 3y ago
      Cory Doctorow - Car Wars https://web.archive.org/web/20170301224942/http://this.deakin.edu.au/culture/car-wars https://web.archive.org/web/20170301224942/http://this.deaki... (Linking to the web.archive version because the graphics are better / more understandable when in the context of some of the text) Chapter 6 is the most relevant here, but it's all a thought provoking story.
    - parineum 3y ago

adwn 3y ago

Braking to avoid hitting an obstacle (like that tree branch in the first example) is hardly "acting unexpectedly". > "Acting unexpectedly" is one of the aspects that makes dealing with mentally ill people anxiety-producing. Wat.

bumby 3y ago
But this certainly could be construed as unexpected: >"In August, a Waymo at an intersection “began to proceed forward” but then “slowed to a stop” and was hit from behind by an SUV." >"Wat." Are you saying you don't understand why unexpected behavior causes anxiety? It's a pretty well documented effect, from rats to humans.
- robertlagrant 3y ago
  It was: > > "Acting unexpectedly" is one of the aspects that makes dealing with mentally ill people anxiety-producing. > Wat. The "Wat" probably refers to the fact that this seems unrelated. Dealing with mentally ill people is anxiety-inducing because they act unexpectedly... so what? Lots of things are unexpected. They shouldn't all be drawn into the analogy that says "well, that thing, plus a load of other things, can induce anxiety, therefore that thing should be tarnished with the same brush as all of those things." People drive in unexpected ways all the time. Of all the criticisms to level at them, "the thing you're doing plus a load of other stuff can induce anxiety" probably isn't top of the list.
  - bumby 3y ago
    I am not painting them with the same brush, I'm drawing an analogy to help people understand the context better. In this case, public policy will dictate to what extent AVs are allowed on public roadways. That, in turn, is dictated by trust. I'm pointing out that "trust" may be incompatible with "unpredictability." I'm not sure what throughline you're drawing, but you seem overly hung up on the use of the word "anxiety," and it's causing you to miss the real point. So to put a finer point on it, people need to acknowledge that public trust is necessary to wide-scale adoption of AV tech. Plenty of psychological research shows how we aren't intuitively wired to understand statistics. So all the bleating about statistics may be necessary, but not sufficient, to get wide-scale adoption of AVs on public roadways.

seandoe 3y ago

I would imagine that, with more experience, anticipating the (more consistent) actions of a machine would be easier than anticipating the actions of an unknown human in an unknown state.

mrguyorama 3y ago
The entire point of this line of discussion is that an ML based system with extremely weird and unexpected failure conditions and failure states ISN'T "more consistent" than a human who might follow more closely than physics says but otherwise is ACTUALLY predictable because they have a mind that we have evolved to predict. ML having completely unpredictable failure modes is like the entire case against putting them anywhere. What would you call a vision system that mis-identifies a stop sigh because of a couple unrelated lines painted on it, other than "unpredictable"?

shadowgovt 3y ago

Quite frankly, the vehicle in front of you is allowed to stop at any time for any reason (legally and practically, as front vehicle has better visibility on road conditions than follower), and it is always incumbent upon the driver to the rear to leave room. If humans can't do that, the solution is probably more automation.

bumby 3y ago
Has the nuisance braking problem been completetly solved? If not, I don't know that I'd agree that more automation is necessarily the answer. More good automation, maybe, but there's a logical jump there. The Uber fatality from years back showed that the software used "action suppression" to mitigate nuisance braking. The idea that that would be acceptable on a safety-critical software application should give us pause to consider that more automation is the knee-jerk solution.
- shadowgovt 3y ago
  "Nuisance braking" is, like jaywalking, a phrase that prioritizes one party's use of the road over another party's. The best policy is still to leave the vehicle in front enough room to brake for any reason. Mostly because "nuisance braking" hasn't been solved in humans either (ever been behind someone who panicked when they realized they were falling asleep behind the wheel? I have.)
  - bumby 3y ago
    I think you are mischaracterizing the problem. Nuisance braking seems to be far more prevalent in AVs than in humans, partly because image classification has more uncertainty. Now, couple that with: 1) a general approach of "when in doubt, brake" and the hacky workaround to suppress braking if it's occurring too much, and you've got a bad nuisance braking problem that's primed for a safety incident. But again, this is disjointed from reality. I'd agree that if we all left the recommended 4.5 seconds of stopping distance, we'd all be safer. But that's not how our roads were built and that's not how humans drive. You're tilting at windmills here to make a point that doesn't need to be made because it doesn't apply to reality.

Justsignedup 3y ago

> I don't think most of us would automatically want to share the roads with a bunch of mentally ill drivers, even if, statistically, they were better than neurotypical drivers. I'm not scared of mentally ill drivers. I'm scared of rich 16 year olds. I'm scared of drunk drivers. I'm scared of drivers sitting so low they can't see most of what is happening around them. I'm scared of drivers having seizures while driving (my mom was hit a while ago by a man who lost his license due to seizures, and still refused to stop driving). I'm scared of drivers who drive without a license because "fuck them, I drive when I want to". I'm scared of people mixing up gas and break pedals (just got hit by one), in cars which can go 0-60 in 2.6 seconds weighing 6000lbs.

Dalewyn 3y ago
>I'm scared of people mixing up gas and break pedals This specific problem is just as much a design flaw as it is a PEBCAC issue. You have two very similar pedals that perform polar opposite functions right next to each other, and they are both operated by the same foot. I'm surprised this isn't a bigger problem.
- bumby 3y ago
  I had to look up the 'PEBKAC' acronym, but I think you allude to the problem of human factors engineering. It's commonplace in aerospace, where safety-critical, time-sensitive decisions must be made, and humans are in the loop. I would extend this to autonomous driving systems, particularly when you expand the system boundaries beyond the car itself. Humans are part of that human-car-environment system, whether as pedestrians, passengers, or other drivers and we should give them consideration.
- Justsignedup 3y ago
  an average of 44 per day in the US. So... common, but relatively, uncommon. And apparently there is software that can help mitigate this.
throwaway2037 3y ago
Slight trolling here: That is a lot of scared. Do you also drive wearing a helmet and protective body suit? man who lost his license due to seizures, and still refused to stop driving If I have to guess, probably the man's livelihood depends upon driving to a job. That is probably why he kept driving.

UncleMeat 3y ago

Ordinary neurotypical drivers act "unexpectedly" on the road all the time. I know that I would brake if I saw a downed branch. And people do much much much more. Suddenly change lanes on the highway with no signaling? Check. Brake suddenly because you almost missed your turn without looking to see if anybody was following close behind? Check. Drive over non-lane portions of the road because you were late seeing your highway exit? Check. Swerve suddenly because you dropped your phone between the seats while texting? Check. I've seen people reverse up a highway onramp.

wddkcs 3y ago
I think both points are true- people act unexpectedly all the time, so much so that we've come to expect the unexpected, by the sophistication of our theory of mind. I live in a city where erratic driving is commonplace, but that's in relation to the legal norms. New local norms become accepted, which you can anticipate once you adapt. Some of these 'norms' are handy time savers, others are incredibly dangerous and result in frequent accidents, but persist anyway. How will AI drivers navigate these 'cultural' differences? Will they insist on following the letter of the law (presumably, for liability) or will they adapt to local practices and potentially lower their overall collision rate with humans driving poorly. Interesting near term question.
- droopyEyelids 3y ago
  You’re being selectively reductive. The unexpected is, by definition, what was not predicted.
  - pastage 3y ago
    The point is unexpected is supposed to be in norm, self driving cars can not tell what a norm is. The idiom is not about the word unexpected, more about the relatioship between chaos and order
jjav 3y ago
> Ordinary neurotypical drivers act "unexpectedly" on the road all the time. "all the time" means it is not unexpected. Humans do mistakes and omissions that you've subconsciously learned that humans do, so you can adjust to deal with that. The difficulty with AI is that it behaves like a completely alien being, making errors that are alien and unpredictable to a human driver.

waythenewsgoes 3y ago

Others may not necessarily agree, but at least anecdotally, a sizeable portion of drivers I see make all kinds of mistakes (law of averages dictates more of them are so called "neurotypical" than not no?). "Acting Unexpectedly" can often mean following the actual laws and general guidelines for safe and/or defensive driving. I would hazard a guess that sometimes doing the intuitive thing is, in reality, unsafe and/or against the law. If the car does this in 99% of circumstances, and still gets rear-ended, who is really the problem here?

lukas099 3y ago
My wife gets triggered every time I drive 'only' the speed limit.
- bobsmooth 3y ago
  Driving the speed limit with people whizzing past you is more dangerous than following the speed of everyone around you.
  - UncleMeat 3y ago
    If somebody is whizzing past you and you are going the speed limit then they aren't going 5mph faster than you. I've never been on a road where everybody is consistently speeding by so much that it could be perceived as "whizzing."
  - lukas099 3y ago
    I've heard that. In driving safety class they taught us that the speed limit is safer. I tried to look up a credible source either way and came up blank. Do you have one? I go by feel. If it feels much safer to speed, I do. Otherwise I'm in the right hand lane, usually a safe stopping distance behind a big rig for aero gains.
    - dcow 3y ago
      I don’t think you get the aero gains at safe stopping distances, though.

lukas099 3y ago

Humans are nothing if not adaptable. We will adjust our expectations.

bumby 3y ago
True, but that doesn't mean it's always a better result. I've adjusted my expectations that software on my smart tv will be glitchy and have interface errors with a lot of apps, but that doesn't mean that's the only or best way to program a system. We adjust to poor quality all the time, but I prefer not to lower my expectations on a safety-critical system.

throwaway2037 3y ago

These two phrases are strange to me in the context of OP: intuit the behavior of other humans through the theory-of-mind and: If AVs consistently act "unexpectedly" Are you implying that the three specific incidents quoted by the OP are acting unexpectedly? I hope not. Most good, aware drivers would do the same. If you disagree, please provide alternative driving actions that you consider expected. And, would the safety outcomes be better? Unlikely.

bumby 3y ago
No, I'm making a broader claim about the nature of AI in safety critical domains. I am also saying we should be careful about drawing strong conclusions from a sample size of n=3 from a potentially questionable dataset.

Nashenas 3y ago

Getting hit from behind in all of these scenarios means the people behind did not maintain proper distancing between vehicles. Especially the 3 car pile up. That's why you're supposed to leave one car length for every 10mph you're going, so you have enough space to cancel out your momentum. What if instead of a branch it was a dog or a child? And I've been to MANY places in the US where people drive terribly and completely unexpectedly. I've seen people put on a left turn signal then go into the RIGHT lane. I've seen someone pass on a one lane overpass, where the car in front was towing a heavy load What's good about these cars is they will ALL drive consistently, and the more people get used to them the easier it well be to drive with them because you will know how waymo cars react.

frogpelt 3y ago

Which raises questions for me about how traffic behaves if 25-50% of cars are self-driving. What "feedback loops" might occur? I'd be interested to see large scale tests that demonstrate how self-driving cars deal with each other in high traffic areas.

rad_gruchalski 3y ago

> Yes, but...there is something else to be said here. One of the things we have evolved to do, without necessarily appreciating it, is to intuit the behavior of other humans through the theory-of-mind. One of the first thing my driving instructor told me was: never trust anyone on the road except yourself. That’s why you see people waiting for a car with the indicator on to visually hint they really intended to turn.