4 ms·

Maybe I'm too optimistic about self driving but it seems that a fail-safe default of "upon encountering an active emergency situation, stop at a reasonable dist

Maybe I'm too optimistic about self driving but it seems that a fail-safe default of "upon encountering an active emergency situation, stop at a reasonable distance away" is good enough and better than what a significant minority of human drivers do.

ulizzle 4y ago
Who are the significant minority? This is the same claim always given by the A.I boosters. Who sees firefighters fighting a fire and decides to just plow through and make matters worse? Because by "significant minority" what you are saying is that 2 or 3 out of 10 human drivers do this.
- colechristensen 4y ago
  Exactly. The fact that some humans are drunks or psychopaths does not mean that AI mistakes are going to just be accepted. We send people to prison when they make those kinds of mistakes, what do we do when machine learning models kill while functioning entirely normally?
  - j16sdiz 4y ago
    Send a low level engineer to prison, of course.
    - ulizzle 4y ago
      That's actually not as wild as you think, because that's what happens when a building or bridge falls. The contractor that built the bridge, just like the company that designed it, the agency responsible for inspections and maintenance, etc, is also liable for damages, including prison if it's deemed so by the courts. Of course, software engineers get treated with kid's gloves, otherwise, everyone just freaks out and cries and cries, because their robotic utopia dreams are meaningless nonsense. Apparently, smart enough to code, but not smart enough to take responsibility for their actions.
  - hgsgm 4y ago
    > We send people to prison when they make those kinds of mistakes, what do we do when machine learning models kill while functioning entirely normally? Assess a massive fine. Money is lifeblood of a corpo.
steveBK123 4y ago
How do we know we can make the machines smart enough to detect the dangerous situations it is incapable of handling on its own? It's just a bit too perfect, no?
- hgsgm 4y ago
  How do we know we can make humans do it?
throwaway173738 4y ago
If the dangerous situation is a flash flood or landslide stopping may be entirely the wrong response. The entire problem with these rigid failsafes and the whole idea of ai-driven cars is this idea that a programmer can predict every situation the car will encounter by extrapolating from the climate and road conditions in places like san francisco. One project I worked on was a safety critical system that almost got sunk by an overly-simplistic fault response. The device would reboot if it detected any sensor readings out of range. One day someone put a bag over it while it was delivering concentrated oxygen from the air and it rebooted because the ambient o2 sensor was reading well below what anyone could reasonably expect were the machine not enclosed. The patient had to be bagged while the machine came back on. You simply can’t predict every emergency situation and you most certainly shouldn’t assume a simplistic single-response model is enough to get the passenger to safety. Humans are really good at inferring correct behavior socially and it would be much better for an AV to watch what other drivers are doing to figure out how to handle unexpected situations. To the degree an AI-driven system is unable to do this it is completely unsuited for use on the road.
- ElFitz 4y ago
  I think many of us aren’t used to working on anything actually critical to human lives. And I wouldn’t be surprised if quite a few of those who are have never actually fully understood what that means. I certainly don’t. But then, there’s a reason I stay away from life-critical systems.
sempron64 4y ago
What about a dangerous situation like a criminal trying to enter the vehicle where the right response is to avoid them and drive away?

ulizzle 4y ago

Who are the significant minority? This is the same claim always given by the A.I boosters. Who sees firefighters fighting a fire and decides to just plow through and make matters worse? Because by "significant minority" what you are saying is that 2 or 3 out of 10 human drivers do this.

colechristensen 4y ago
Exactly. The fact that some humans are drunks or psychopaths does not mean that AI mistakes are going to just be accepted. We send people to prison when they make those kinds of mistakes, what do we do when machine learning models kill while functioning entirely normally?
- j16sdiz 4y ago
  Send a low level engineer to prison, of course.
  - ulizzle 4y ago
    That's actually not as wild as you think, because that's what happens when a building or bridge falls. The contractor that built the bridge, just like the company that designed it, the agency responsible for inspections and maintenance, etc, is also liable for damages, including prison if it's deemed so by the courts. Of course, software engineers get treated with kid's gloves, otherwise, everyone just freaks out and cries and cries, because their robotic utopia dreams are meaningless nonsense. Apparently, smart enough to code, but not smart enough to take responsibility for their actions.
- hgsgm 4y ago
  > We send people to prison when they make those kinds of mistakes, what do we do when machine learning models kill while functioning entirely normally? Assess a massive fine. Money is lifeblood of a corpo.

steveBK123 4y ago

How do we know we can make the machines smart enough to detect the dangerous situations it is incapable of handling on its own? It's just a bit too perfect, no?

hgsgm 4y ago
How do we know we can make humans do it?

throwaway173738 4y ago

If the dangerous situation is a flash flood or landslide stopping may be entirely the wrong response. The entire problem with these rigid failsafes and the whole idea of ai-driven cars is this idea that a programmer can predict every situation the car will encounter by extrapolating from the climate and road conditions in places like san francisco. One project I worked on was a safety critical system that almost got sunk by an overly-simplistic fault response. The device would reboot if it detected any sensor readings out of range. One day someone put a bag over it while it was delivering concentrated oxygen from the air and it rebooted because the ambient o2 sensor was reading well below what anyone could reasonably expect were the machine not enclosed. The patient had to be bagged while the machine came back on. You simply can’t predict every emergency situation and you most certainly shouldn’t assume a simplistic single-response model is enough to get the passenger to safety. Humans are really good at inferring correct behavior socially and it would be much better for an AV to watch what other drivers are doing to figure out how to handle unexpected situations. To the degree an AI-driven system is unable to do this it is completely unsuited for use on the road.

ElFitz 4y ago
I think many of us aren’t used to working on anything actually critical to human lives. And I wouldn’t be surprised if quite a few of those who are have never actually fully understood what that means. I certainly don’t. But then, there’s a reason I stay away from life-critical systems.

sempron64 4y ago

What about a dangerous situation like a criminal trying to enter the vehicle where the right response is to avoid them and drive away?