6 ms·

What strikes me odd is that once an unknown is detected on the road, the car should already be alerting the driver and slowing down. That buys you time for the

What strikes me odd is that once an unknown is detected on the road, the car should already be alerting the driver and slowing down. That buys you time for the ai classifier to do its thing and isn’t as dangerous as an emergency braking later on, so seems a sensible behavior all around.

tomkarlo 8y ago
I'd guess that "unknown objects" happen all the time - it seems like that's the default until something is classified, so tire scrap or plastic bag would also fall into that category. If the car slowed down every time it saw one it would never get anywhere, it should only slow if the object gets classified as something you can't hit and is clearly in the path of the vehicle. Seems like that decision happened too late here, requiring emergency braking... which was disabled (!).
- amenod 8y ago
  The missing bit here is "...unknown is detected on the road" - if there is a tire scrap or plastic bag or anything that looks suspicious a normal human driver would slow down and give it extra attention, then try to avoid it anyway. You don't drive over / through an object unless you know that it is not harmful, and even then you try to avoid it so you don't drive over a bag... of nails.
  - EpicEng 8y ago
    It's not that simple. Humans can also predict movement, and that is necessary because cars don't stop instantaneously. So you have a person walking toward the street and your dumb smart car is constantly hitting the brakes. This tech simply isn't there yet and I doubt it's all that close.
    - kolpa 8y ago
      > So you have a person walking toward the street > and your dumb smart car is constantly hitting the brakes. Doesn't sound dumb to me. The car should be going slow enough to emergency stop if the pedestrian enters the road.
      - LoSboccacc 8y ago
        human track gaze and understand desires, so you kinda know if someone exiting a shop will or will not proceed straight into the road. that said, as I said above whenever a car sense a situation it doesn't understand it should slow down, that's enough to be safe later on as the situation develops and is different than hitting the brakes full force. and anyway expecting autonomous car to drive full speed all the time is moronic, humans don't do that either, precisely because it's dangerous.
      - vbezhenar 8y ago
        People don't drive like that. People expect reasonable behavior from other people and that includes expectations that they won't jump into the road. If Uber will drive unreasonably, its passengers will prefer other taxi who drive more aggressively.
      - EpicEng 8y ago
        Except no one drives that way and no one would put up with it. Do you slow down every time someone on the sidewalk takes a step toward the street? I doubt it (and, if you do, please never get in front of me.) So, yeah: dumb.
        blattimwind 8y ago
        If I think they might walk on the street I of course slow down (as much as I deem necessary). What kind of question is that? If I think I saw children run around between cars on the parking lane, are the parents probably morons? Yes. Do I slow down and be prepare to slam the brakes in case a child suddenly runs in front of me? Ab-so-lute-ly. Even if people behave idiotically on the street, it is obviously still my fault if I run them over.
        EpicEng 8y ago
        >If I think they might walk on the street I of course slow down And that's what I'm talking about. Computers aren't all that good at determining whether or not a person is going to jump into the street.
    - CaptainZapp 8y ago
      This tech simply isn't there yet and I doubt it's all that close. In which case it's absolutely criminal by the government of Arizona to allow testing such tech on public roads.
- icebraining 8y ago
  How many plastic bags do you have crossing your streets?! Around here I doubt you'd slow down more than a couple of times per month.
  - tomkarlo 8y ago
    Not as many since they banned them here, but it used to be quite common given the combination between the fact it's pretty windy here in SF every afternoon, and there's lots of trash / debris around. It's still pretty common to see blowing paper or other debris (tire shreds) in the road at least a few times in my 12 mile commute. This has been a not-minor problem for autonomous cars and the Tesla-style autopilots / adaptive cruise controls that depend on vision only. You have to program it to ignore some types of things that seem like they might be an obstruction, such as road signs, debris in the road, etc. so they don't hit the brakes unnecessarily.
- LoSboccacc 8y ago
  > If the car slowed down every time it saw one [unknown object] it would never get anywhere well tough shit then, a car that plows trough situation it doesn't understand should never be on the public road
  - tomkarlo 8y ago
    Both adaptive cruise controls and human drivers do this by default. If you're doing 60 MPH on a highway and something pops into the periphery of your vision that you don't recognize, do you slam on the brakes? No.
    - LoSboccacc 8y ago
      of course I do slow down, there's a whole load of possibilities between slamming the brakes and get rear ended and driving at the posted limit trough a dangerous situation, you know? car can slow down gently. it's a 60 zone and rain or smoke impairs the visibility? slow down. it's a 45mph zone and something that's not a motor vehicle is in a lane that's supposed to only have motor vehicles on it? you slow down until you make sense of the situation. you're near a playground and a mother is walking a children on the other side of the road and you can't see if she's holding his hand? you slow down. a person walks near the kerb and it's not looking in your direction? you slow down. a bike is loaded with groceries? a car acting erratically? a person being pulled by his dog? a bus stopped unloading people? you don't drive past them at 30mph. driving safely: super simple stuff.
      - forapurpose 8y ago
        > super simple stuff If people did the super simple stuff, we'd have boundless peace, prosperity, liberty. I remember a story - stop me if you've heard this one - about a God helping out a group of desperate people, freeing them from slavery, parting seas, feeding them in the desert. They were camped at the foot of a mountain with the God right there on top - right there! And they built the golden calf anyway. And that rule seems easier than all the other 9. WTF did they even need a golden calf for? So sadly, the criteria of simplicity is irrelevant - people will find a hard way to do it.
- skykooler 8y ago
  As a human, if I saw a plastic bag blow into the street at night I'd slow down until I was sure it wasn't an animal or something. Seems like basically the same process.
  - tomkarlo 8y ago
    Sure, but if you were on a highway, would you slam on the brakes? I hope not. There's a calculation here of balancing the perceived risk of an obstruction with the consequences of avoiding it or braking in time. Drivers have to make this decision all the time, on a highway they will generally assume it's safer to hit most things than swerve or panic brake, because it's mostly likely not that dangerous to collide with. At least one stat I saw from AAA is that ~40% of the deaths from road debris result from drivers swerving to avoid them.
    - erobbins 8y ago
      That means 60% result from hitting them.. still seems better to swerve.
cc439 8y ago
The more after reports on accidents involving self driving cars I've read, the more I've become convinced that the current state of the art in this field is merely an illusion. Reading between the lines, the complete disregard for basic safety protocols like you've just described comes across as more than just a brazen continuation of the "move fast and break things" Silicon Valley culture. Viewed in this light, this entire niche of tech R&D begins to take on the appearance of a giant game of smoke and mirrors being played for the benefit of those recipients of Big Investment $$$ that wooed investors to bet big with promises of delivering a marketable self-driving car within the next decade. The way I see it,there is only way to make sense of a field where the most respectable R&D house (Google/Alphabet) limits their vehicle to a relative snail's pace while everyone else (including notoriously unethical shops like Uber) is taking a gung-ho, "the only limit is the speed limit" approach. That is to assume "everyone else" is cheating the game by choosing a development path that gives the appearance of being functional in the "rough but will get there with enough polish" sense while the truth is that it's merely a mountain of cheap and dirty hacks that will never achieve the goals demanded by investors. The only reason a company would overlook such a simple safety protocol as "slow down until a potentially dangerous object is positively identified" is if their "AI" fails to positively identify objects so frequently that the car could never achieve what a human passenger would consider a consistent, normal driving pace. The same can be said for any "AI" that can't be trusted to initiate panic braking in response to a positively identified collision scenario with a positively identified object. The fact that they specifically wrote an "AI" absolving workaround for that scenario their software means the frequency of false positives must be so high as to make the frequency of "false alarm" panic braking incidents unacceptable for human passengers.
mcguire 8y ago
I'm not sure why your comment seems to be grey-ed out. Here's the relevant section from the report: "As the vehicle and pedestrian paths converged, the self-driving system software classified the pedestrian as an unknown object, as a vehicle, and then as a bicycle with varying expectations of future travel path." 1. It seems like the classifier flipped state between pedestrian/unknown object/vehicle/bicycle; this seems like one of the well-known issues with machine learning. (I'm assuming the classifier is using ML simply because I have never heard of any other (semi-?) successful work on that problem.) I suggest that the problem is that the rest of the driving system went from 100% certainty of A to 100% certainty of B, etc., with a resulting complete recalculation of what do to about the current classification. I make this hypothesis on the basis of the 4+ seconds when the car did nothing, while a response to any of the individual possibilities would possibly have averted the accident. 2. If the classifier was flipping state, I assume the system interrupted the Decide-Act phases of an OODA loop, resulting in the car continuing its given path rather than executing any actions. This seems like a reasonable thing to do, if the system contains no moment-to-moment state. Which would be strange; it seems like the planning system should have some case for having obstacles A, B, C, and D rapidly and successively appearing in the same area of its path. 3. Assuming the classifier wasn't flipping state, but presenting multiple options with probabilities, I can see no reason why the car wouldn't have taken some action in the 4+ seconds. (I note that the trajectory of the vehicle seems to move towards the right of its lane, which is a rather inadequate response and likely the wrong thing to do for several of the classification options.) "According to Uber, emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior." That's just idiotic and would be nigh-criminally unprofessional in most engineering situations.

tomkarlo 8y ago

I'd guess that "unknown objects" happen all the time - it seems like that's the default until something is classified, so tire scrap or plastic bag would also fall into that category. If the car slowed down every time it saw one it would never get anywhere, it should only slow if the object gets classified as something you can't hit and is clearly in the path of the vehicle. Seems like that decision happened too late here, requiring emergency braking... which was disabled (!).

amenod 8y ago
The missing bit here is "...unknown is detected on the road" - if there is a tire scrap or plastic bag or anything that looks suspicious a normal human driver would slow down and give it extra attention, then try to avoid it anyway. You don't drive over / through an object unless you know that it is not harmful, and even then you try to avoid it so you don't drive over a bag... of nails.
- EpicEng 8y ago
  It's not that simple. Humans can also predict movement, and that is necessary because cars don't stop instantaneously. So you have a person walking toward the street and your dumb smart car is constantly hitting the brakes. This tech simply isn't there yet and I doubt it's all that close.
  - kolpa 8y ago
    > So you have a person walking toward the street > and your dumb smart car is constantly hitting the brakes. Doesn't sound dumb to me. The car should be going slow enough to emergency stop if the pedestrian enters the road.
    - LoSboccacc 8y ago
      human track gaze and understand desires, so you kinda know if someone exiting a shop will or will not proceed straight into the road. that said, as I said above whenever a car sense a situation it doesn't understand it should slow down, that's enough to be safe later on as the situation develops and is different than hitting the brakes full force. and anyway expecting autonomous car to drive full speed all the time is moronic, humans don't do that either, precisely because it's dangerous.
    - vbezhenar 8y ago
      People don't drive like that. People expect reasonable behavior from other people and that includes expectations that they won't jump into the road. If Uber will drive unreasonably, its passengers will prefer other taxi who drive more aggressively.
    - EpicEng 8y ago
      Except no one drives that way and no one would put up with it. Do you slow down every time someone on the sidewalk takes a step toward the street? I doubt it (and, if you do, please never get in front of me.) So, yeah: dumb.
      - blattimwind 8y ago
        If I think they might walk on the street I of course slow down (as much as I deem necessary). What kind of question is that? If I think I saw children run around between cars on the parking lane, are the parents probably morons? Yes. Do I slow down and be prepare to slam the brakes in case a child suddenly runs in front of me? Ab-so-lute-ly. Even if people behave idiotically on the street, it is obviously still my fault if I run them over.
        EpicEng 8y ago
        >If I think they might walk on the street I of course slow down And that's what I'm talking about. Computers aren't all that good at determining whether or not a person is going to jump into the street.
  - CaptainZapp 8y ago
    This tech simply isn't there yet and I doubt it's all that close. In which case it's absolutely criminal by the government of Arizona to allow testing such tech on public roads.
icebraining 8y ago
How many plastic bags do you have crossing your streets?! Around here I doubt you'd slow down more than a couple of times per month.
- tomkarlo 8y ago
  Not as many since they banned them here, but it used to be quite common given the combination between the fact it's pretty windy here in SF every afternoon, and there's lots of trash / debris around. It's still pretty common to see blowing paper or other debris (tire shreds) in the road at least a few times in my 12 mile commute. This has been a not-minor problem for autonomous cars and the Tesla-style autopilots / adaptive cruise controls that depend on vision only. You have to program it to ignore some types of things that seem like they might be an obstruction, such as road signs, debris in the road, etc. so they don't hit the brakes unnecessarily.
LoSboccacc 8y ago
> If the car slowed down every time it saw one [unknown object] it would never get anywhere well tough shit then, a car that plows trough situation it doesn't understand should never be on the public road
- tomkarlo 8y ago
  Both adaptive cruise controls and human drivers do this by default. If you're doing 60 MPH on a highway and something pops into the periphery of your vision that you don't recognize, do you slam on the brakes? No.
  - LoSboccacc 8y ago
    of course I do slow down, there's a whole load of possibilities between slamming the brakes and get rear ended and driving at the posted limit trough a dangerous situation, you know? car can slow down gently. it's a 60 zone and rain or smoke impairs the visibility? slow down. it's a 45mph zone and something that's not a motor vehicle is in a lane that's supposed to only have motor vehicles on it? you slow down until you make sense of the situation. you're near a playground and a mother is walking a children on the other side of the road and you can't see if she's holding his hand? you slow down. a person walks near the kerb and it's not looking in your direction? you slow down. a bike is loaded with groceries? a car acting erratically? a person being pulled by his dog? a bus stopped unloading people? you don't drive past them at 30mph. driving safely: super simple stuff.
    - forapurpose 8y ago
      > super simple stuff If people did the super simple stuff, we'd have boundless peace, prosperity, liberty. I remember a story - stop me if you've heard this one - about a God helping out a group of desperate people, freeing them from slavery, parting seas, feeding them in the desert. They were camped at the foot of a mountain with the God right there on top - right there! And they built the golden calf anyway. And that rule seems easier than all the other 9. WTF did they even need a golden calf for? So sadly, the criteria of simplicity is irrelevant - people will find a hard way to do it.
skykooler 8y ago
As a human, if I saw a plastic bag blow into the street at night I'd slow down until I was sure it wasn't an animal or something. Seems like basically the same process.
- tomkarlo 8y ago
  Sure, but if you were on a highway, would you slam on the brakes? I hope not. There's a calculation here of balancing the perceived risk of an obstruction with the consequences of avoiding it or braking in time. Drivers have to make this decision all the time, on a highway they will generally assume it's safer to hit most things than swerve or panic brake, because it's mostly likely not that dangerous to collide with. At least one stat I saw from AAA is that ~40% of the deaths from road debris result from drivers swerving to avoid them.
  - erobbins 8y ago
    That means 60% result from hitting them.. still seems better to swerve.

cc439 8y ago

The more after reports on accidents involving self driving cars I've read, the more I've become convinced that the current state of the art in this field is merely an illusion. Reading between the lines, the complete disregard for basic safety protocols like you've just described comes across as more than just a brazen continuation of the "move fast and break things" Silicon Valley culture. Viewed in this light, this entire niche of tech R&D begins to take on the appearance of a giant game of smoke and mirrors being played for the benefit of those recipients of Big Investment $$$ that wooed investors to bet big with promises of delivering a marketable self-driving car within the next decade. The way I see it,there is only way to make sense of a field where the most respectable R&D house (Google/Alphabet) limits their vehicle to a relative snail's pace while everyone else (including notoriously unethical shops like Uber) is taking a gung-ho, "the only limit is the speed limit" approach. That is to assume "everyone else" is cheating the game by choosing a development path that gives the appearance of being functional in the "rough but will get there with enough polish" sense while the truth is that it's merely a mountain of cheap and dirty hacks that will never achieve the goals demanded by investors. The only reason a company would overlook such a simple safety protocol as "slow down until a potentially dangerous object is positively identified" is if their "AI" fails to positively identify objects so frequently that the car could never achieve what a human passenger would consider a consistent, normal driving pace. The same can be said for any "AI" that can't be trusted to initiate panic braking in response to a positively identified collision scenario with a positively identified object. The fact that they specifically wrote an "AI" absolving workaround for that scenario their software means the frequency of false positives must be so high as to make the frequency of "false alarm" panic braking incidents unacceptable for human passengers.

mcguire 8y ago

I'm not sure why your comment seems to be grey-ed out. Here's the relevant section from the report: "As the vehicle and pedestrian paths converged, the self-driving system software classified the pedestrian as an unknown object, as a vehicle, and then as a bicycle with varying expectations of future travel path." 1. It seems like the classifier flipped state between pedestrian/unknown object/vehicle/bicycle; this seems like one of the well-known issues with machine learning. (I'm assuming the classifier is using ML simply because I have never heard of any other (semi-?) successful work on that problem.) I suggest that the problem is that the rest of the driving system went from 100% certainty of A to 100% certainty of B, etc., with a resulting complete recalculation of what do to about the current classification. I make this hypothesis on the basis of the 4+ seconds when the car did nothing, while a response to any of the individual possibilities would possibly have averted the accident. 2. If the classifier was flipping state, I assume the system interrupted the Decide-Act phases of an OODA loop, resulting in the car continuing its given path rather than executing any actions. This seems like a reasonable thing to do, if the system contains no moment-to-moment state. Which would be strange; it seems like the planning system should have some case for having obstacles A, B, C, and D rapidly and successively appearing in the same area of its path. 3. Assuming the classifier wasn't flipping state, but presenting multiple options with probabilities, I can see no reason why the car wouldn't have taken some action in the 4+ seconds. (I note that the trajectory of the vehicle seems to move towards the right of its lane, which is a rather inadequate response and likely the wrong thing to do for several of the classification options.) "According to Uber, emergency braking maneuvers are not enabled while the vehicle is under computer control, to reduce the potential for erratic vehicle behavior." That's just idiotic and would be nigh-criminally unprofessional in most engineering situations.