8 ms·

It only highlights how incredibly dumb these autopilots are. They’re incredibly diligent but completely fail at the necessary skill of understanding. What is t

It only highlights how incredibly dumb these autopilots are. They’re incredibly diligent but completely fail at the necessary skill of understanding. What is the point of any of this if the car with all of its sensors knowingly plows into a stationary red 30-ton obstacle. It’s something that a 4 year old human would understand instantly as an object to be avoided at all cost. I really think folks don’t realize how critical all of our human understanding is to driving.

_jal 7y ago
...Also, that urban environments are the _last_ places these should be deployed. There is just too much everyday weirdness. Hell, I get confused sometimes in driving SF - flakey drivers, aggressively solipsistic homeless folks crossing (or just standing around in) the street randomly, bicycles, shopping carts, dogs, kids... I want to see the robot death boxes competently handle less fraught environments before allowing them around humans. And industry suggestions that the humans not in robot death boxes cede the streets to those who are? Ah, no. Cars own practically everywhere else - you can go play with your toy on the freeway or drive the damn car yourself around other humans.
NeedMoreTea 7y ago
Except humans crash into stationary vehicles in lane all the damn time. Most highway pile ups are a simple crash lots of folks drive into the back of. Why's it hard? A vehicle coming from a side road has a lot of movement within your field of vision. When driving on a highway we're well used to the fact and expectation that stuff isn't stopped. When the vehicle ahead emergency brakes or stops you have the growth of size of the target to gauge by. That's not much movement, until the last moment when it's far too late to evade. Some stationary debris is much easier - it's not meant to be there at all - you transition to evasion right from the get go. For a human in a car, the dangerous highway case is the stationary vehicle in the fast lane, maybe one or two vehicles ahead. When flying it's the other aircraft that's on a collision course. Neither will move within your field of vision. Neither is easy to detect as problem, until it's very late. FWIW 4 year old humans often run into things. :)
- akira2501 7y ago
  > Why's it hard? That humans get distracted and often make mistakes is no surprise, it should also be obvious that the amount of risky behavior is correlated to the amount of _perceived_ risk in any given situation, which is probably why so many are "a simple crash." It is, however, somewhat scary that a machine that has nothing else to do other than to drive the car as safely as possible still makes these "dumb" mistakes.
  - greglindahl 7y ago
    Adaptive cruise control makes dumb mistakes. Cruise control makes dumb mistakes. I know people who think automatic transmissions makes people worse drivers.
    - urban_strike 7y ago
      > I know people who think automatic transmissions makes people worse drivers. Well, it's certainly a bit trickier to to respond to texts or browse instagram when you need two hands to operate the vehicle.
      - Corrado 7y ago
        You would probably be surprised what a person with 30+ years behind the stick can do while operating a manual transmission vehicle.
    - whenchamenia 7y ago
      There are indeed satistics showing that standard transmissions actually correlate positively with driver attenttiveness. I will try to find a link...
    - whenchamenia 7y ago
      https://journals.sagepub.com/doi/abs/10.1177/1087054706288103 https://journals.sagepub.com/doi/abs/10.1177/108705470628810... Well, thats half of the problematic drivers, and i suspect texters as well. /s
mixmastamyk 7y ago
Sounds like dumb policy to me. Why wouldn’t it stop for a boulder or fallen tree?
- Retric 7y ago
  /devils advocate False positives are dangerous. Trying to detect ‘Anything’ may result in more deaths than hitting the tiny number of exceptions. Alternatively, the system would be overly flakey and hard break all the time.
  - mixmastamyk 7y ago
    Slowing down is a lot safer than plowing into a stationary object at high speed. This is so obvious there must be something missing to the conversation.
    - Retric 7y ago
      Self driving cars are going to be driven a lot, so something that’s a 1/10,000 risk that happens every week to 1 million cars is going to happen every day. Self driving cars need to have a threshold before hard breaking. Hard breaking is risky, it can cause serious accidents. Reducing that threshold thus increases some risks even if it lowers others. So, now we need to balance a risk of say ~1/10,000,000 per car per year vs what? I don’t know what changing those thresholds would result in, but I do know it’s not free and there would be false positives.
      - mixmastamyk 7y ago
        Everything you wrote is plausible or true, but the fact remains slowing to a stop is the best choice unless the obstruction is tiny. That’s what a human would do. There might be a possibility of changing lanes but that is next level.
- dahfizz 7y ago
  How often are bolders in the road? As software engineers, we can understand the hidden costs in complexity and bugs when implementing features that don't need to be implemented. I could imagine that false positives for bolder avoidance causing more harm than false negatives. Obviously it should be smart enough to detect any and every obstruction, but I'd rather the 99% use case get ironed out first.
Gustomaximus 7y ago
We need to look at driverless a different way and I suspect this is why it's going to be hard to accept driverless for longer than we need to. Computers look at the world in a different way. So they are going to have accidents that seem very obvious to humans making people go "look how bad driverless is because a human would never...". Many people will take that as a reason to avoid driverless tech. Yet driverless will (or should) avoid many more accidents humans would have made if they were in control. Because of this we need to look at overall statistics. If we only look at specific driverless accidents from a human lens it's going to be easy to non-lateral thinkers to get caught up in criticising but it doesn't actually understand the benifits or not.
- jmpman 7y ago
  And the lawyers will become very wealthy putting these companies into bankruptcy... as they should.
  - Gustomaximus 7y ago
    Why should they?
    - jmpman 7y ago
      Because self driving car companies are putting a dangerous product onto the road, citing “safer than human drivers” in a disingenuous manner. Musk likes to cite autopilot as safer than human drivers, based upon accidents per miles driven. However, where are autopilot miles primarily being driven? On the highway. I expect a sober human driver to have a much lower accident rate than Tesla’s autopilot when driving the same highway miles. And yet Musk pushes his agenda... which is possibly an overreach. When I see cars plowing into the rear of fire trucks, into freeway split barriers, into tow trucks or under turning semis, I have to ask myself if I would have been able to avoid those, and I can honestly say that I would have. Would a drunken high schooler? Probably not. I simply don’t believe the “safer than humans” narrative is being honest with the number of caveats. Until self driving car executives are willing to send their children into the street in a real life game of frogger with their cars being driven by a bunch of drunken frat boys in a blizzard, they shouldn’t expect the general public to blindly agree to be their guinea pigs either.
dahfizz 7y ago
This is completely ridiculous. Adults drive into stationary objects all the time. 4 year olds walk into stationary objects all the time. As far as I'm concerned, there is only one criterion that matters when evaluating self driving cars: is it better than human drivers? The fact that we are still discussing this _one_ accident that happened _a year ago_ indicates to me that the answer is a resounding yes.
- AndrewBissell 7y ago
  There are many Tesla Autopilot accidents under investigation, and their relative infrequency is due to the fact that a.) accidents themselves are rare, and b.) Autopilot (mercifully) has yet to see widespread deployment in the U.S. vehicle fleet.
- matt-attack 7y ago
  > Adults drive into stationary objects all the time I agree. But do they when alert and focused and when they clearly see the object? I highly doubt it. This Tesla was all of those things, and still crashed.
  - aidenn0 7y ago
    That's a non-sequitur unless the average driver the autopilot is replacing is alert and focused.
    - rohit2412 7y ago
      Or you could have only the automatic emergency braking in a vehicle, and not autosteer. Stop acting like it's all or nothing. The technology behind autopilot is being touted as driverless/self-driving, but it can do passive accident avoidance. That should be the benchmark to compare too. In this particular case, humans get distracted and get in an accident. Autopilot is dumb as shit and gets in an accident. But a human with an emergency braking system is smart and doesn't get in an accident on the account of driver distraction.
      - 7y ago
Animats 7y ago
While "human understanding" would be nice, detecting big solid obstacles is a solved problem. It just costs money. Until somebody orders enough LIDAR units to get the cost down. Tesla's "autopilot" collisions are all very similar. They're all "hit big stationary obstacle that wasn't the rear end of a car". So far, a fire truck, a street sweeper, a stalled car at a freeway median, a crossing semitrailer, and two freeway barriers. This is a well-defined defect.
- Shivetya 7y ago
  you do not need LIDAR to recognize a stopped vehicle, higher powered visual processing will suffice.
  - tehwhale 7y ago
    You sound like Elon. LIDAR is far superior to radar in a lot of cases. The best situation would be for Tesla's to use a mix of both radar and lidar.
StreamBright 7y ago
Exactly. This is what I always point out to autonomous car people, driving a car cannot be modeled by ML too well, a simple fire truck parked in the lane makes it obvious.

_jal 7y ago

...Also, that urban environments are the _last_ places these should be deployed. There is just too much everyday weirdness. Hell, I get confused sometimes in driving SF - flakey drivers, aggressively solipsistic homeless folks crossing (or just standing around in) the street randomly, bicycles, shopping carts, dogs, kids... I want to see the robot death boxes competently handle less fraught environments before allowing them around humans. And industry suggestions that the humans not in robot death boxes cede the streets to those who are? Ah, no. Cars own practically everywhere else - you can go play with your toy on the freeway or drive the damn car yourself around other humans.

NeedMoreTea 7y ago

Except humans crash into stationary vehicles in lane all the damn time. Most highway pile ups are a simple crash lots of folks drive into the back of. Why's it hard? A vehicle coming from a side road has a lot of movement within your field of vision. When driving on a highway we're well used to the fact and expectation that stuff isn't stopped. When the vehicle ahead emergency brakes or stops you have the growth of size of the target to gauge by. That's not much movement, until the last moment when it's far too late to evade. Some stationary debris is much easier - it's not meant to be there at all - you transition to evasion right from the get go. For a human in a car, the dangerous highway case is the stationary vehicle in the fast lane, maybe one or two vehicles ahead. When flying it's the other aircraft that's on a collision course. Neither will move within your field of vision. Neither is easy to detect as problem, until it's very late. FWIW 4 year old humans often run into things. :)

akira2501 7y ago
> Why's it hard? That humans get distracted and often make mistakes is no surprise, it should also be obvious that the amount of risky behavior is correlated to the amount of _perceived_ risk in any given situation, which is probably why so many are "a simple crash." It is, however, somewhat scary that a machine that has nothing else to do other than to drive the car as safely as possible still makes these "dumb" mistakes.
- greglindahl 7y ago
  Adaptive cruise control makes dumb mistakes. Cruise control makes dumb mistakes. I know people who think automatic transmissions makes people worse drivers.
  - urban_strike 7y ago
    > I know people who think automatic transmissions makes people worse drivers. Well, it's certainly a bit trickier to to respond to texts or browse instagram when you need two hands to operate the vehicle.
    - Corrado 7y ago
      You would probably be surprised what a person with 30+ years behind the stick can do while operating a manual transmission vehicle.
  - whenchamenia 7y ago
    There are indeed satistics showing that standard transmissions actually correlate positively with driver attenttiveness. I will try to find a link...
  - whenchamenia 7y ago
    https://journals.sagepub.com/doi/abs/10.1177/1087054706288103 https://journals.sagepub.com/doi/abs/10.1177/108705470628810... Well, thats half of the problematic drivers, and i suspect texters as well. /s

mixmastamyk 7y ago

Sounds like dumb policy to me. Why wouldn’t it stop for a boulder or fallen tree?

Retric 7y ago
/devils advocate False positives are dangerous. Trying to detect ‘Anything’ may result in more deaths than hitting the tiny number of exceptions. Alternatively, the system would be overly flakey and hard break all the time.
- mixmastamyk 7y ago
  Slowing down is a lot safer than plowing into a stationary object at high speed. This is so obvious there must be something missing to the conversation.
  - Retric 7y ago
    Self driving cars are going to be driven a lot, so something that’s a 1/10,000 risk that happens every week to 1 million cars is going to happen every day. Self driving cars need to have a threshold before hard breaking. Hard breaking is risky, it can cause serious accidents. Reducing that threshold thus increases some risks even if it lowers others. So, now we need to balance a risk of say ~1/10,000,000 per car per year vs what? I don’t know what changing those thresholds would result in, but I do know it’s not free and there would be false positives.
    - mixmastamyk 7y ago
      Everything you wrote is plausible or true, but the fact remains slowing to a stop is the best choice unless the obstruction is tiny. That’s what a human would do. There might be a possibility of changing lanes but that is next level.
dahfizz 7y ago
How often are bolders in the road? As software engineers, we can understand the hidden costs in complexity and bugs when implementing features that don't need to be implemented. I could imagine that false positives for bolder avoidance causing more harm than false negatives. Obviously it should be smart enough to detect any and every obstruction, but I'd rather the 99% use case get ironed out first.

Gustomaximus 7y ago

We need to look at driverless a different way and I suspect this is why it's going to be hard to accept driverless for longer than we need to. Computers look at the world in a different way. So they are going to have accidents that seem very obvious to humans making people go "look how bad driverless is because a human would never...". Many people will take that as a reason to avoid driverless tech. Yet driverless will (or should) avoid many more accidents humans would have made if they were in control. Because of this we need to look at overall statistics. If we only look at specific driverless accidents from a human lens it's going to be easy to non-lateral thinkers to get caught up in criticising but it doesn't actually understand the benifits or not.

jmpman 7y ago
And the lawyers will become very wealthy putting these companies into bankruptcy... as they should.
- Gustomaximus 7y ago
  Why should they?
  - jmpman 7y ago
    Because self driving car companies are putting a dangerous product onto the road, citing “safer than human drivers” in a disingenuous manner. Musk likes to cite autopilot as safer than human drivers, based upon accidents per miles driven. However, where are autopilot miles primarily being driven? On the highway. I expect a sober human driver to have a much lower accident rate than Tesla’s autopilot when driving the same highway miles. And yet Musk pushes his agenda... which is possibly an overreach. When I see cars plowing into the rear of fire trucks, into freeway split barriers, into tow trucks or under turning semis, I have to ask myself if I would have been able to avoid those, and I can honestly say that I would have. Would a drunken high schooler? Probably not. I simply don’t believe the “safer than humans” narrative is being honest with the number of caveats. Until self driving car executives are willing to send their children into the street in a real life game of frogger with their cars being driven by a bunch of drunken frat boys in a blizzard, they shouldn’t expect the general public to blindly agree to be their guinea pigs either.

dahfizz 7y ago

This is completely ridiculous. Adults drive into stationary objects all the time. 4 year olds walk into stationary objects all the time. As far as I'm concerned, there is only one criterion that matters when evaluating self driving cars: is it better than human drivers? The fact that we are still discussing this _one_ accident that happened _a year ago_ indicates to me that the answer is a resounding yes.

AndrewBissell 7y ago
There are many Tesla Autopilot accidents under investigation, and their relative infrequency is due to the fact that a.) accidents themselves are rare, and b.) Autopilot (mercifully) has yet to see widespread deployment in the U.S. vehicle fleet.
matt-attack 7y ago
> Adults drive into stationary objects all the time I agree. But do they when alert and focused and when they clearly see the object? I highly doubt it. This Tesla was all of those things, and still crashed.
- aidenn0 7y ago
  That's a non-sequitur unless the average driver the autopilot is replacing is alert and focused.
  - rohit2412 7y ago
    Or you could have only the automatic emergency braking in a vehicle, and not autosteer. Stop acting like it's all or nothing. The technology behind autopilot is being touted as driverless/self-driving, but it can do passive accident avoidance. That should be the benchmark to compare too. In this particular case, humans get distracted and get in an accident. Autopilot is dumb as shit and gets in an accident. But a human with an emergency braking system is smart and doesn't get in an accident on the account of driver distraction.
    - 7y ago

Animats 7y ago

While "human understanding" would be nice, detecting big solid obstacles is a solved problem. It just costs money. Until somebody orders enough LIDAR units to get the cost down. Tesla's "autopilot" collisions are all very similar. They're all "hit big stationary obstacle that wasn't the rear end of a car". So far, a fire truck, a street sweeper, a stalled car at a freeway median, a crossing semitrailer, and two freeway barriers. This is a well-defined defect.

Shivetya 7y ago
you do not need LIDAR to recognize a stopped vehicle, higher powered visual processing will suffice.
- tehwhale 7y ago
  You sound like Elon. LIDAR is far superior to radar in a lot of cases. The best situation would be for Tesla's to use a mix of both radar and lidar.

StreamBright 7y ago

Exactly. This is what I always point out to autonomous car people, driving a car cannot be modeled by ML too well, a simple fire truck parked in the lane makes it obvious.