15 ms·

>> 5.6 seconds before impact, it classified her as a vehicle. Then it changed its mind to “other,” then to vehicle again, back to “other,” then to bicycle, then

>> 5.6 seconds before impact, it classified her as a vehicle. Then it changed its mind to “other,” then to vehicle again, back to “other,” then to bicycle, then to “other” again, and finally back to bicycle. The system should have started applying brake at this point. If a 3500lb vehicle can't decide what it is about to impact, it needs to slow down (to a stop if necessary). > That triggered what Uber called “action suppression,” in which the system held off braking for one second This is borderline criminal negligence. > with 0.2 seconds left before impact, the car sounded an audio alarm, and Vasquez took the steering wheel, disengaging the autonomous system. Nearly a full second after striking Herzberg, Vasquez hit the brakes. Why were there no alarms going off at 5.6 seconds when the vehicle was confused!!!?? SMH. This is just ... I'm flabbergasted.

gdulli 7y ago
> The system should have started applying brake at this point. If a 3500lb vehicle can't decide what it is about to impact, it needs to slow down (to a stop if necessary). I suspect there's a big problem in the other direction, too. If the system starts to brake every time it thinks it might need to, it will happen all the time. It might see false positives like this all the time and that's why it doesn't act on them right away. If it's (appropriately) conservative about starting to brake it will be braking/slowing all the time. If it's not conservative, people or cars will occasionally get hit. The former could make people uncomfortable or carsick or create some subtler danger by stopping short needlessly. The latter might mostly work and mostly not kill people, except for when it does.
- fmpwizard 7y ago
  If we apply this to a human, let's say I'm getting old and my eyes aren't as good any more, I don't just keep driving like nothing changed, I need to find a way to see better, because the risk of getting into an accident is higher. If the car's system cannot tell what an object is, you don't just assume it's going to be ok, you need to either get better sensors or find some other real solution.
- trhway 7y ago
  >If the system starts to brake every time it thinks it might need to, it will happen all the time. It might see false positives like this all the time and that's why it doesn't act on them right away. if your car always sees empty space ahead as non-empty space, you probably shouldn't let the car on the road until you fix that. Once you fixed that, if the car sees that the space ahead is non-empty, even it can't classify it, it should slow down well before and warn the driver and continue with the braking if the driver is asleep/watching youtube. It is AZ, there is no rain nor snowflakes falling which would mislead the lidar. An object ahead - slow down and stop if you can't navigate it. Presenting it as the AI-hard issue of misclassification is just an attention misdirection from and a whitewhashing/laundering of the foundational issue of knowingly letting car on the road with missing basic safety level of "don't hit objects in front of the car". Similar to the Boeing blaming 737 MAX crashes on the failed sensor.
  - dkonofalski 7y ago
    >It is AZ, there is no rain nor snowflakes falling This is so understated. This car was basically driving in ideal conditions at night for a self-driving car. If it can't avoid a hit under these conditions, it clearly wasn't ready to be on the road.
- nolok 7y ago
  Then your system is not ready to be on the road. We're taking about metal box weighting at least a ton, launched at dozen of miles an hour on roads where they can hit people. "They had to not react quickly because they keep mis-detecting things" is really not an acceptable justification. If that's the why (and it probably is) that car should never have been on the street.
  - gdulli 7y ago
    Exactly my point. I'm not apologizing for the software. I have severe doubt that software is anywhere close to making safe and appropriate decisions in the context of real world driving with open ended conditions about the makeup of the road, full range of obstacles and edge case conditions, weather, lighting, etc. The scariest thing is that it's so fucking familiar, this idea of technologists solving a range of simple cases and assuming the solution extrapolates to success in the real world. We cannot get into these cars if that kind of software development feels anything like everyday software development, where random web site bugs can be tolerated.
    - rhizome 7y ago
      No, the scariest thing is that capital may eventually push through a launch of these crappy devices before they're ready. And they'll never be ready.
      - shadowgovt 7y ago
        Why do you think they'll never be ready?
        rhizome 7y ago
        Because I don't think computer vision will ever be good enough to be safe for this purpose.
        shadowgovt 7y ago
- rhizome 7y ago
  >* suspect there's a big problem in the other direction, too. If the system starts to brake every time it thinks it might need to, it will happen all the time. It might see false positives like this all the time and that's why it doesn't act on them right away.* Oh no, it doesn't work and Travis & friends have been lying for years. Wake me when there are consequences for the follies of rich white guys.
- smileysteve 7y ago
  > If the system starts to brake every time it thinks it might need to, it will happen all the time. One thing that self driving will "give back" is time to the occupant. And the thing that most human driven accidents are correlated to damage; speed.
rayiner 7y ago
It all makes sense if you realize that Uber’s tech fundamentally doesn’t work, and if you programmed it react immediately or set off alarms immediately the false positives would make it unusable.
dkonofalski 7y ago
>This is borderline criminal negligence. Thank you for flatly and plainly stating this. That is unacceptable and anyone involved with approving that decision should be prosecuted in the death.
- trashface 7y ago
  I'd also argue that US government should shut down Uber's self-driving research division. If they want self-driving cars they can license from Waymo, or someone else who is actually taking it seriously. Its pretty clear that Uber's techbros have some serious culture issues. They have shown they are incompetent and should be disqualified by regulators.
  - uoaei 7y ago
    I think it'll be hard to come up with a legally defensible reason to demand that they shut down all research efforts without looking like it's a vendetta. Also, Uber's business plan is 100% to maintain share in the rideshare market until their cars drive themselves. That is the end game here. To shut down their self-driving research is to doom the company entirely. Not that I'd be against that. Would be great to see Uber die. It's already caused enough headache and pain.
    - _jal 7y ago
      > I think it'll be hard to come up with a legally defensible reason to demand that they shut down all research efforts without looking like it's a vendetta. They don't have to shut down their research. I don't care if they want to write software and play on private tracks. As I see it, Uber should be explaining exactly why their privilege to endanger the public by operating unsafe robots on public roads should not be revoked for criminal incompetence.
    - jacquesm 7y ago
      A fatality should do nicely. /s
    - michaelmrose 7y ago
mzs 7y ago
No it shouldn't have applied the brakes at that point. There are lots of vehicles around in most cases. If the car braked every time it spotted another vehicle it would almost never move. Only take corrective action if the paths look like they will intersect instead.
- chungus_khan 7y ago
  At five seconds from impact it absolutely should have. And if it couldn't tell that it was that close to the "vehicle" that's even more reason it shouldn't be anywhere near the road in this state. The system should be able to behave cautiously in situations where input is erratic. Good human drivers don't go full speed ahead when they can't see.
  - mzs 7y ago
    It took over four seconds for the system to decide swerving was not sufficient based on path prediction. The improvements made after this report would have initiated braking seconds earlier. -5.6s 44mph Classification: Vehicle - by radar Path prediction: None; not on the path of the SUV - Radar makes the first detection of the pedestrian and estimates its speed. ... -1.5s 44mph Classification: Unknown - by lidar Path prediction: Static; partially on the path of the SUV - Lidar detects an unknown object; since this is a changed classification, and an unknown object, it lacks tracking history and is not assigned a goal. ADS predicts the object’s path as static. - Although the detected object is partially in the SUV’s lane of travel, the ADS generates a motion plan around the object (maneuver to the right of the object); this motion plan remains valid—avoiding the object—for the next two data points. -1.2s 43mph Classification: Bicycle - by lidar Path prediction: The travel lane of the SUV; fully on the path of the SUV - Lidar detects a bicycle; although this is a changed classification and without a tracking history, it was assigned a goal. ADS predicts the bicycle to be on the path of the SUV. - The ADS motion plan—generated 300 msec earlier—for steering around the bicycle was no longer possible; as such, this situation becomes hazardous. - Action suppression begins. * The vehicle started decelerating due to the approaching intersection, where the pre-planned route includes a right turn at Curry Road. The deceleration plan was generated 3.6 seconds before impact https://dms.ntsb.gov/public/62500-62999/62978/629713.pdf https://dms.ntsb.gov/public/62500-62999/62978/629713.pdf
SamuelAdams 7y ago
> If a 3500lb vehicle can't decide what it is about to impact, it needs to slow down (to a stop if necessary). Don't be so sure. What if it's a bird, and you're traveling at 70 mph on a busy freeway? Slowing down or stopping could result in getting rear-ended or worse. Clearly more testing is needed, and more work needs to be put into identifying objects early.
- jacquesm 7y ago
  If you can't tell a bird from a car or a bike you have zero business being in the self driving car arena.
  - gugagore 7y ago
    It's not that I disagree with you, it's just by your standards, [almost] no one should be in the self-driving car arena.
    - jacquesm 7y ago
      Not on public roads.
    - bumby 7y ago
      ...yet? At least this doesn't seem to be that hard of a line to draw in terms of competency before allowing on public roads
    - patrickyeon 7y ago
      That's quite possibly a reasonable assessment of the situation.
  - Johnny555 7y ago
    What do you do when your radar says it could be a person, but your vision sensor tells you it's probably a bird?
    - jacquesm 7y ago
shadowgovt 7y ago
> Why were there no alarms going off at 5.6 seconds when the vehicle was confused!!!?? I wonder if it's because their system is not reliable enough yet; a continuous alarm isn't useful.
- rwbhn 7y ago
  Then it can't be deployed on public roadways!
  - jerf 7y ago
    You know, I don't usually like this quote when people deploy it, because it rarely applies. Frankly I find it questionable even in the original movie. But it applies here like gangbusters: "Your scientists were so preoccupied with whether or not they could that they didn't stop to think if they should." There are a lot of commenters here who seem to be operating on a subconscious presumption that being "the best we could do" is some sort of defense. It's not. If the best you can do is not good enough, don't deploy it.
- Nition 7y ago
  Considering the aforementioned one second "got a weird result, wait and see if it goes away" system, this seems highly likely. If everything's an alarm, nothing is.
theptip 7y ago
> This is borderline criminal negligence. Counterargument: the safety driver is supposed to be prepared to take control at any time, should the vehicle do something unsafe. It seems to me to be clearly criminal to use this code to run a fully automated environment (i.e. no safety driver). It's not clear to me what the expectations should be of the code when there is supposed to be an attentive safety driver in the vehicle. I think the safety driver is going to jail, because they were watching videos instead of watching the road at the time their vehicle killed a pedestrian.
- johnday 7y ago
  There is a clear subtext in everything put out by Tesla and Uber that their cars are self-driving. That is to say, they drive themselves. That is to say, nobody else needs to drive them in order for them to get from A to B. If they are truly trying to stick to the guns of "well the 'driver' should have been prepared to take over at zero notice", then their (potentially criminal) negligence is in how they present and market their product and not in the software. But it's one or the other.
  - dabbledash 7y ago
    Not to mention, the idea that a human failsafe will be attentive enough to respond instantly is ridiculous to begin with. If the car drives itself 99% of the time, it better drive itself 100% of the time, because the human is checking out.
    - jacquesm 7y ago
      > because the human is checking out. In some cases in the most literal sense of that.
  - close04 7y ago
    > There is a clear subtext in everything put out That's in marketing materials rather then the legal ones. Which is great to make regular people interpret it "look, it can drive itself, I'll just fiddle with my phone" but doesn't put any of the legal responsibility on the manufacturer's shoulders. I'm guessing regulators will eventually have to weigh in on this.
nathanvanfleet 7y ago
I think it's a bit vaguely described but I think the pedestrian wasn't in the path of the vehicle at 5.6 seconds since they were walking perpendicularly. The system couldn't classify the person or that they were walking into its path. Or at least that what I gathered from that. I guess if it classifies something as unknown it's basically would have no behavior? I am just speculating.
a_t48 7y ago
> The system should have started applying brake at this point. If a 3500lb vehicle can't decide what it is about to impact, it needs to slow down (to a stop if necessary). > Why were there no alarms going off at 5.6 seconds when the vehicle was confused!!!?? That sort of depends on the specifics of how their obstacle prediction and avoidance works - having a fuzzy view on what exactly something is at 5.6 seconds out is probably ok. The important bit is that it notices that the obstacle is moving out to the road and should stop it. Classification is not needed to avoid objects. The key words here are actually "at each change, the object's tracking history is unavailable" and "without a tracking history, ADS predicts the bicycle's path as static" which is a horrible oversight. >> That triggered what Uber called “action suppression,” in which the system held off braking for one second > This is borderline criminal negligence. Yeah, this is. Even though it was too late to swerve, there was still enough time to slow down and attempt to reduce the speed of impact. _This_ is probably where an alarm should have fired - the car is now in a really unsafe state and the safety driver should absolutely know about it. Disclaimer - Up until a few months ago I worked at a major (non Uber) self driving car company.
- gambiting 7y ago
  >>The important bit is that it notices that the obstacle is moving out to the road and should stop it The problem here is that if I'm doing 70mph+ on the motorway and I see a plastic bag flying in my path, the correct course of action isn't to brake hard or panically try to avoid it - but a computer cannot tell a difference between a soft plastic bag and something more rigid. If the only classifier is that "there's an object in my way, I need to stop" then that's also super dangerous. In fact, there are cases where unfortunately the correct course of action is to keep going straight and hit the object - any kind of small animal you shouldn't try to avoid, especially at higher speeds - yes they will wreck your car, but a panicked pull to the left or right can actually kill you. As a human you should do this instinctively - another human = avoid at all cost, small animal = brake hard but keep going straight, plastic bags = don't brake, don't move, keep going straight. How do you teach that to a machine if the machine can't reliably tell what it's looking at?
  - simion314 7y ago
dragonwriter 7y ago
> This is borderline criminal negligence. Yes, but on the border between “criminal negligence” and “depraved indifference” [0] not the border between “criminal negligence” and “innocent conduct”. [0] https://en.m.wikipedia.org/wiki/Depraved-heart_murder https://en.m.wikipedia.org/wiki/Depraved-heart_murder
thaumasiotes 7y ago
> If a 3500lb vehicle can't decide what it is about to impact, it needs to slow down (to a stop if necessary). Well, also, any of those categories should trigger braking themselves. You can't hit vehicles or bicycles. You can't hit "other" either. If you plan to hit something, it needs to be a piece of paper or some such. As a_t48 points out, classification is not needed to avoid objects, so being confused about what the object is is totally irrelevant.
- johannes1234321 7y ago
  > Well, also, any of those categories should trigger braking themselves. You can't hit vehicles or bicycles. A bicycle on the side of the same lane will typically move in the same direction and stay on the side. Thus the car has to overtake. That is different from a person crossing the street. You always have to make assumptions, about elements in the path. And it has to have an option to handle the worst case.
- smileysteve 7y ago
  This, whether it's a speed bump, trash, tire tread, tow hitch, bicycle, animal, or person in the road, it does significantly more damage at 35mph than it does at the reasonable decrease to 25mph. Basically, absent vehicles behind you and to each side, a driver should stop until the object can be identified and determined if it will do damage or be avoided.
throwaway35784 7y ago
> This is borderline criminal negligence. Uber had told the world who they are: criminals. They have flaunted many laws since inception. Will they ever stop? Who will stop them?
- perl4ever 7y ago
  They have flaunted their criminality. They have flouted the law.
ska 7y ago
This is borderline criminal negligence. I don't think there is much border.

gdulli 7y ago

> The system should have started applying brake at this point. If a 3500lb vehicle can't decide what it is about to impact, it needs to slow down (to a stop if necessary). I suspect there's a big problem in the other direction, too. If the system starts to brake every time it thinks it might need to, it will happen all the time. It might see false positives like this all the time and that's why it doesn't act on them right away. If it's (appropriately) conservative about starting to brake it will be braking/slowing all the time. If it's not conservative, people or cars will occasionally get hit. The former could make people uncomfortable or carsick or create some subtler danger by stopping short needlessly. The latter might mostly work and mostly not kill people, except for when it does.

fmpwizard 7y ago
If we apply this to a human, let's say I'm getting old and my eyes aren't as good any more, I don't just keep driving like nothing changed, I need to find a way to see better, because the risk of getting into an accident is higher. If the car's system cannot tell what an object is, you don't just assume it's going to be ok, you need to either get better sensors or find some other real solution.
trhway 7y ago
>If the system starts to brake every time it thinks it might need to, it will happen all the time. It might see false positives like this all the time and that's why it doesn't act on them right away. if your car always sees empty space ahead as non-empty space, you probably shouldn't let the car on the road until you fix that. Once you fixed that, if the car sees that the space ahead is non-empty, even it can't classify it, it should slow down well before and warn the driver and continue with the braking if the driver is asleep/watching youtube. It is AZ, there is no rain nor snowflakes falling which would mislead the lidar. An object ahead - slow down and stop if you can't navigate it. Presenting it as the AI-hard issue of misclassification is just an attention misdirection from and a whitewhashing/laundering of the foundational issue of knowingly letting car on the road with missing basic safety level of "don't hit objects in front of the car". Similar to the Boeing blaming 737 MAX crashes on the failed sensor.
- dkonofalski 7y ago
  >It is AZ, there is no rain nor snowflakes falling This is so understated. This car was basically driving in ideal conditions at night for a self-driving car. If it can't avoid a hit under these conditions, it clearly wasn't ready to be on the road.
nolok 7y ago
Then your system is not ready to be on the road. We're taking about metal box weighting at least a ton, launched at dozen of miles an hour on roads where they can hit people. "They had to not react quickly because they keep mis-detecting things" is really not an acceptable justification. If that's the why (and it probably is) that car should never have been on the street.
- gdulli 7y ago
  Exactly my point. I'm not apologizing for the software. I have severe doubt that software is anywhere close to making safe and appropriate decisions in the context of real world driving with open ended conditions about the makeup of the road, full range of obstacles and edge case conditions, weather, lighting, etc. The scariest thing is that it's so fucking familiar, this idea of technologists solving a range of simple cases and assuming the solution extrapolates to success in the real world. We cannot get into these cars if that kind of software development feels anything like everyday software development, where random web site bugs can be tolerated.
  - rhizome 7y ago
    No, the scariest thing is that capital may eventually push through a launch of these crappy devices before they're ready. And they'll never be ready.
    - shadowgovt 7y ago
      Why do you think they'll never be ready?
      - rhizome 7y ago
        Because I don't think computer vision will ever be good enough to be safe for this purpose.
        shadowgovt 7y ago
rhizome 7y ago
>* suspect there's a big problem in the other direction, too. If the system starts to brake every time it thinks it might need to, it will happen all the time. It might see false positives like this all the time and that's why it doesn't act on them right away.* Oh no, it doesn't work and Travis & friends have been lying for years. Wake me when there are consequences for the follies of rich white guys.
smileysteve 7y ago
> If the system starts to brake every time it thinks it might need to, it will happen all the time. One thing that self driving will "give back" is time to the occupant. And the thing that most human driven accidents are correlated to damage; speed.

rayiner 7y ago

It all makes sense if you realize that Uber’s tech fundamentally doesn’t work, and if you programmed it react immediately or set off alarms immediately the false positives would make it unusable.

dkonofalski 7y ago

>This is borderline criminal negligence. Thank you for flatly and plainly stating this. That is unacceptable and anyone involved with approving that decision should be prosecuted in the death.

trashface 7y ago
I'd also argue that US government should shut down Uber's self-driving research division. If they want self-driving cars they can license from Waymo, or someone else who is actually taking it seriously. Its pretty clear that Uber's techbros have some serious culture issues. They have shown they are incompetent and should be disqualified by regulators.
- uoaei 7y ago
  I think it'll be hard to come up with a legally defensible reason to demand that they shut down all research efforts without looking like it's a vendetta. Also, Uber's business plan is 100% to maintain share in the rideshare market until their cars drive themselves. That is the end game here. To shut down their self-driving research is to doom the company entirely. Not that I'd be against that. Would be great to see Uber die. It's already caused enough headache and pain.
  - _jal 7y ago
    > I think it'll be hard to come up with a legally defensible reason to demand that they shut down all research efforts without looking like it's a vendetta. They don't have to shut down their research. I don't care if they want to write software and play on private tracks. As I see it, Uber should be explaining exactly why their privilege to endanger the public by operating unsafe robots on public roads should not be revoked for criminal incompetence.
  - jacquesm 7y ago
    A fatality should do nicely. /s
  - michaelmrose 7y ago

mzs 7y ago

No it shouldn't have applied the brakes at that point. There are lots of vehicles around in most cases. If the car braked every time it spotted another vehicle it would almost never move. Only take corrective action if the paths look like they will intersect instead.

chungus_khan 7y ago
At five seconds from impact it absolutely should have. And if it couldn't tell that it was that close to the "vehicle" that's even more reason it shouldn't be anywhere near the road in this state. The system should be able to behave cautiously in situations where input is erratic. Good human drivers don't go full speed ahead when they can't see.
- mzs 7y ago
  It took over four seconds for the system to decide swerving was not sufficient based on path prediction. The improvements made after this report would have initiated braking seconds earlier. -5.6s 44mph Classification: Vehicle - by radar Path prediction: None; not on the path of the SUV - Radar makes the first detection of the pedestrian and estimates its speed. ... -1.5s 44mph Classification: Unknown - by lidar Path prediction: Static; partially on the path of the SUV - Lidar detects an unknown object; since this is a changed classification, and an unknown object, it lacks tracking history and is not assigned a goal. ADS predicts the object’s path as static. - Although the detected object is partially in the SUV’s lane of travel, the ADS generates a motion plan around the object (maneuver to the right of the object); this motion plan remains valid—avoiding the object—for the next two data points. -1.2s 43mph Classification: Bicycle - by lidar Path prediction: The travel lane of the SUV; fully on the path of the SUV - Lidar detects a bicycle; although this is a changed classification and without a tracking history, it was assigned a goal. ADS predicts the bicycle to be on the path of the SUV. - The ADS motion plan—generated 300 msec earlier—for steering around the bicycle was no longer possible; as such, this situation becomes hazardous. - Action suppression begins. * The vehicle started decelerating due to the approaching intersection, where the pre-planned route includes a right turn at Curry Road. The deceleration plan was generated 3.6 seconds before impact https://dms.ntsb.gov/public/62500-62999/62978/629713.pdf https://dms.ntsb.gov/public/62500-62999/62978/629713.pdf

SamuelAdams 7y ago

> If a 3500lb vehicle can't decide what it is about to impact, it needs to slow down (to a stop if necessary). Don't be so sure. What if it's a bird, and you're traveling at 70 mph on a busy freeway? Slowing down or stopping could result in getting rear-ended or worse. Clearly more testing is needed, and more work needs to be put into identifying objects early.

jacquesm 7y ago
If you can't tell a bird from a car or a bike you have zero business being in the self driving car arena.
- gugagore 7y ago
  It's not that I disagree with you, it's just by your standards, [almost] no one should be in the self-driving car arena.
  - jacquesm 7y ago
    Not on public roads.
  - bumby 7y ago
    ...yet? At least this doesn't seem to be that hard of a line to draw in terms of competency before allowing on public roads
  - patrickyeon 7y ago
    That's quite possibly a reasonable assessment of the situation.
- Johnny555 7y ago
  What do you do when your radar says it could be a person, but your vision sensor tells you it's probably a bird?
  - jacquesm 7y ago

shadowgovt 7y ago

> Why were there no alarms going off at 5.6 seconds when the vehicle was confused!!!?? I wonder if it's because their system is not reliable enough yet; a continuous alarm isn't useful.

rwbhn 7y ago
Then it can't be deployed on public roadways!
- jerf 7y ago
  You know, I don't usually like this quote when people deploy it, because it rarely applies. Frankly I find it questionable even in the original movie. But it applies here like gangbusters: "Your scientists were so preoccupied with whether or not they could that they didn't stop to think if they should." There are a lot of commenters here who seem to be operating on a subconscious presumption that being "the best we could do" is some sort of defense. It's not. If the best you can do is not good enough, don't deploy it.
Nition 7y ago
Considering the aforementioned one second "got a weird result, wait and see if it goes away" system, this seems highly likely. If everything's an alarm, nothing is.

theptip 7y ago

> This is borderline criminal negligence. Counterargument: the safety driver is supposed to be prepared to take control at any time, should the vehicle do something unsafe. It seems to me to be clearly criminal to use this code to run a fully automated environment (i.e. no safety driver). It's not clear to me what the expectations should be of the code when there is supposed to be an attentive safety driver in the vehicle. I think the safety driver is going to jail, because they were watching videos instead of watching the road at the time their vehicle killed a pedestrian.

johnday 7y ago
There is a clear subtext in everything put out by Tesla and Uber that their cars are self-driving. That is to say, they drive themselves. That is to say, nobody else needs to drive them in order for them to get from A to B. If they are truly trying to stick to the guns of "well the 'driver' should have been prepared to take over at zero notice", then their (potentially criminal) negligence is in how they present and market their product and not in the software. But it's one or the other.
- dabbledash 7y ago
  Not to mention, the idea that a human failsafe will be attentive enough to respond instantly is ridiculous to begin with. If the car drives itself 99% of the time, it better drive itself 100% of the time, because the human is checking out.
  - jacquesm 7y ago
    > because the human is checking out. In some cases in the most literal sense of that.
- close04 7y ago
  > There is a clear subtext in everything put out That's in marketing materials rather then the legal ones. Which is great to make regular people interpret it "look, it can drive itself, I'll just fiddle with my phone" but doesn't put any of the legal responsibility on the manufacturer's shoulders. I'm guessing regulators will eventually have to weigh in on this.

nathanvanfleet 7y ago

I think it's a bit vaguely described but I think the pedestrian wasn't in the path of the vehicle at 5.6 seconds since they were walking perpendicularly. The system couldn't classify the person or that they were walking into its path. Or at least that what I gathered from that. I guess if it classifies something as unknown it's basically would have no behavior? I am just speculating.

a_t48 7y ago

> The system should have started applying brake at this point. If a 3500lb vehicle can't decide what it is about to impact, it needs to slow down (to a stop if necessary). > Why were there no alarms going off at 5.6 seconds when the vehicle was confused!!!?? That sort of depends on the specifics of how their obstacle prediction and avoidance works - having a fuzzy view on what exactly something is at 5.6 seconds out is probably ok. The important bit is that it notices that the obstacle is moving out to the road and should stop it. Classification is not needed to avoid objects. The key words here are actually "at each change, the object's tracking history is unavailable" and "without a tracking history, ADS predicts the bicycle's path as static" which is a horrible oversight. >> That triggered what Uber called “action suppression,” in which the system held off braking for one second > This is borderline criminal negligence. Yeah, this is. Even though it was too late to swerve, there was still enough time to slow down and attempt to reduce the speed of impact. _This_ is probably where an alarm should have fired - the car is now in a really unsafe state and the safety driver should absolutely know about it. Disclaimer - Up until a few months ago I worked at a major (non Uber) self driving car company.

gambiting 7y ago
>>The important bit is that it notices that the obstacle is moving out to the road and should stop it The problem here is that if I'm doing 70mph+ on the motorway and I see a plastic bag flying in my path, the correct course of action isn't to brake hard or panically try to avoid it - but a computer cannot tell a difference between a soft plastic bag and something more rigid. If the only classifier is that "there's an object in my way, I need to stop" then that's also super dangerous. In fact, there are cases where unfortunately the correct course of action is to keep going straight and hit the object - any kind of small animal you shouldn't try to avoid, especially at higher speeds - yes they will wreck your car, but a panicked pull to the left or right can actually kill you. As a human you should do this instinctively - another human = avoid at all cost, small animal = brake hard but keep going straight, plastic bags = don't brake, don't move, keep going straight. How do you teach that to a machine if the machine can't reliably tell what it's looking at?
- simion314 7y ago

dragonwriter 7y ago

> This is borderline criminal negligence. Yes, but on the border between “criminal negligence” and “depraved indifference” [0] not the border between “criminal negligence” and “innocent conduct”. [0] https://en.m.wikipedia.org/wiki/Depraved-heart_murder https://en.m.wikipedia.org/wiki/Depraved-heart_murder

thaumasiotes 7y ago

> If a 3500lb vehicle can't decide what it is about to impact, it needs to slow down (to a stop if necessary). Well, also, any of those categories should trigger braking themselves. You can't hit vehicles or bicycles. You can't hit "other" either. If you plan to hit something, it needs to be a piece of paper or some such. As a_t48 points out, classification is not needed to avoid objects, so being confused about what the object is is totally irrelevant.

johannes1234321 7y ago
> Well, also, any of those categories should trigger braking themselves. You can't hit vehicles or bicycles. A bicycle on the side of the same lane will typically move in the same direction and stay on the side. Thus the car has to overtake. That is different from a person crossing the street. You always have to make assumptions, about elements in the path. And it has to have an option to handle the worst case.
smileysteve 7y ago
This, whether it's a speed bump, trash, tire tread, tow hitch, bicycle, animal, or person in the road, it does significantly more damage at 35mph than it does at the reasonable decrease to 25mph. Basically, absent vehicles behind you and to each side, a driver should stop until the object can be identified and determined if it will do damage or be avoided.

throwaway35784 7y ago

> This is borderline criminal negligence. Uber had told the world who they are: criminals. They have flaunted many laws since inception. Will they ever stop? Who will stop them?

perl4ever 7y ago
They have flaunted their criminality. They have flouted the law.

ska 7y ago

This is borderline criminal negligence. I don't think there is much border.