4 ms·

> The system should have started applying brake at this point. If a 3500lb vehicle can't decide what it is about to impact, it needs to slow down (to a stop if

by gdulli 7y ago

> The system should have started applying brake at this point. If a 3500lb vehicle can't decide what it is about to impact, it needs to slow down (to a stop if necessary). I suspect there's a big problem in the other direction, too. If the system starts to brake every time it thinks it might need to, it will happen all the time. It might see false positives like this all the time and that's why it doesn't act on them right away. If it's (appropriately) conservative about starting to brake it will be braking/slowing all the time. If it's not conservative, people or cars will occasionally get hit. The former could make people uncomfortable or carsick or create some subtler danger by stopping short needlessly. The latter might mostly work and mostly not kill people, except for when it does.

fmpwizard 7y ago
If we apply this to a human, let's say I'm getting old and my eyes aren't as good any more, I don't just keep driving like nothing changed, I need to find a way to see better, because the risk of getting into an accident is higher. If the car's system cannot tell what an object is, you don't just assume it's going to be ok, you need to either get better sensors or find some other real solution.
trhway 7y ago
>If the system starts to brake every time it thinks it might need to, it will happen all the time. It might see false positives like this all the time and that's why it doesn't act on them right away. if your car always sees empty space ahead as non-empty space, you probably shouldn't let the car on the road until you fix that. Once you fixed that, if the car sees that the space ahead is non-empty, even it can't classify it, it should slow down well before and warn the driver and continue with the braking if the driver is asleep/watching youtube. It is AZ, there is no rain nor snowflakes falling which would mislead the lidar. An object ahead - slow down and stop if you can't navigate it. Presenting it as the AI-hard issue of misclassification is just an attention misdirection from and a whitewhashing/laundering of the foundational issue of knowingly letting car on the road with missing basic safety level of "don't hit objects in front of the car". Similar to the Boeing blaming 737 MAX crashes on the failed sensor.
- dkonofalski 7y ago
  >It is AZ, there is no rain nor snowflakes falling This is so understated. This car was basically driving in ideal conditions at night for a self-driving car. If it can't avoid a hit under these conditions, it clearly wasn't ready to be on the road.
nolok 7y ago
Then your system is not ready to be on the road. We're taking about metal box weighting at least a ton, launched at dozen of miles an hour on roads where they can hit people. "They had to not react quickly because they keep mis-detecting things" is really not an acceptable justification. If that's the why (and it probably is) that car should never have been on the street.
- gdulli 7y ago
  Exactly my point. I'm not apologizing for the software. I have severe doubt that software is anywhere close to making safe and appropriate decisions in the context of real world driving with open ended conditions about the makeup of the road, full range of obstacles and edge case conditions, weather, lighting, etc. The scariest thing is that it's so fucking familiar, this idea of technologists solving a range of simple cases and assuming the solution extrapolates to success in the real world. We cannot get into these cars if that kind of software development feels anything like everyday software development, where random web site bugs can be tolerated.
  - rhizome 7y ago
    No, the scariest thing is that capital may eventually push through a launch of these crappy devices before they're ready. And they'll never be ready.
    - shadowgovt 7y ago
      Why do you think they'll never be ready?
      - rhizome 7y ago
        Because I don't think computer vision will ever be good enough to be safe for this purpose.
        shadowgovt 7y ago
rhizome 7y ago
>* suspect there's a big problem in the other direction, too. If the system starts to brake every time it thinks it might need to, it will happen all the time. It might see false positives like this all the time and that's why it doesn't act on them right away.* Oh no, it doesn't work and Travis & friends have been lying for years. Wake me when there are consequences for the follies of rich white guys.
smileysteve 7y ago
> If the system starts to brake every time it thinks it might need to, it will happen all the time. One thing that self driving will "give back" is time to the occupant. And the thing that most human driven accidents are correlated to damage; speed.

by gdulli 7y ago

fmpwizard 7y ago

If we apply this to a human, let's say I'm getting old and my eyes aren't as good any more, I don't just keep driving like nothing changed, I need to find a way to see better, because the risk of getting into an accident is higher. If the car's system cannot tell what an object is, you don't just assume it's going to be ok, you need to either get better sensors or find some other real solution.

trhway 7y ago

>If the system starts to brake every time it thinks it might need to, it will happen all the time. It might see false positives like this all the time and that's why it doesn't act on them right away. if your car always sees empty space ahead as non-empty space, you probably shouldn't let the car on the road until you fix that. Once you fixed that, if the car sees that the space ahead is non-empty, even it can't classify it, it should slow down well before and warn the driver and continue with the braking if the driver is asleep/watching youtube. It is AZ, there is no rain nor snowflakes falling which would mislead the lidar. An object ahead - slow down and stop if you can't navigate it. Presenting it as the AI-hard issue of misclassification is just an attention misdirection from and a whitewhashing/laundering of the foundational issue of knowingly letting car on the road with missing basic safety level of "don't hit objects in front of the car". Similar to the Boeing blaming 737 MAX crashes on the failed sensor.

dkonofalski 7y ago
>It is AZ, there is no rain nor snowflakes falling This is so understated. This car was basically driving in ideal conditions at night for a self-driving car. If it can't avoid a hit under these conditions, it clearly wasn't ready to be on the road.

nolok 7y ago

Then your system is not ready to be on the road. We're taking about metal box weighting at least a ton, launched at dozen of miles an hour on roads where they can hit people. "They had to not react quickly because they keep mis-detecting things" is really not an acceptable justification. If that's the why (and it probably is) that car should never have been on the street.

gdulli 7y ago
Exactly my point. I'm not apologizing for the software. I have severe doubt that software is anywhere close to making safe and appropriate decisions in the context of real world driving with open ended conditions about the makeup of the road, full range of obstacles and edge case conditions, weather, lighting, etc. The scariest thing is that it's so fucking familiar, this idea of technologists solving a range of simple cases and assuming the solution extrapolates to success in the real world. We cannot get into these cars if that kind of software development feels anything like everyday software development, where random web site bugs can be tolerated.
- rhizome 7y ago
  No, the scariest thing is that capital may eventually push through a launch of these crappy devices before they're ready. And they'll never be ready.
  - shadowgovt 7y ago
    Why do you think they'll never be ready?
    - rhizome 7y ago
      Because I don't think computer vision will ever be good enough to be safe for this purpose.
      - shadowgovt 7y ago

rhizome 7y ago

>* suspect there's a big problem in the other direction, too. If the system starts to brake every time it thinks it might need to, it will happen all the time. It might see false positives like this all the time and that's why it doesn't act on them right away.* Oh no, it doesn't work and Travis & friends have been lying for years. Wake me when there are consequences for the follies of rich white guys.

smileysteve 7y ago

> If the system starts to brake every time it thinks it might need to, it will happen all the time. One thing that self driving will "give back" is time to the occupant. And the thing that most human driven accidents are correlated to damage; speed.