4 ms·

It's the usual AI bug, crashing on a rare edge case (i.e. not well represented in the training data) that seems straightforward logically to a human, leading to

by gpm 6y ago

It's the usual AI bug, crashing on a rare edge case (i.e. not well represented in the training data) that seems straightforward logically to a human, leading to us being surprised/shocked/horrified that the AI missed the issue. The most interesting part of this, is that this is a case that humans screw up on a pretty regular basis too. However we understand the human reason for doing so (lack of concentration), and understand that AIs don't suffer from the same fault. So when an AI screws it up we are still surprised.

shagie 6y ago
The issue is that the AI will do the exact same thing each time it is presented with the same data (crash) whereas a human could go to remedial driver's education. And taking away the license of one inattentive driver gets them (theoretically) off the road... whereas if you have an AI crash into a fire truck and you said "stop it" - you still have all the other identical deployments out there with the same flaw. And "add some partial blockage by a vehicle in the lane, retrain the AI, and then redeploy it" isn't a proper answer... wouldn't that be a different model AI that would need to be validated again?
- deleted 6y ago
  [deleted]
- gpm 6y ago
  You can turn this around though. It's harder to teach the AI, sure. But once you've taught it, it stays taught. With humans you have to deal with the continually renewing stream of young people who are new to driving. > And "add some partial blockage by a vehicle in the lane, retrain the AI, and then redeploy it" isn't a proper answer... wouldn't that be a different model AI that would need to be validated again? You're the one adding unnecessary constraints here. If the validation process harms more than helps, you don't do it. In all likelyhood the correct validation process is something lightweight that can be done quickly for simple changes. This sort of training process is likely to be a continuous process, they're always ingesting new data, trying to learn how to drive better from it (and keep up with how patterns in the world change), and therefore continually deploying new models. Because that makes driving safer overall. Not a "this bad thing happened so let's figure out how to put out a bugfix for it".
- AshamedCaptain 6y ago
  > The issue is that the AI will do the exact same thing each time it is presented with the same data That is not true at all these days, not to mention that even a slight change in illumination will result in completely different input data, the way current AIs work.
- deleted 6y ago
  [deleted]
thaumasiotes 6y ago
> crashing on a rare edge case (i.e. not well represented in the training data) This isn't a case where it makes any sense to try to extrapolate from training data and call it a day. You don't need any training to avoid hitting stationary objects. Compare this comment from 2018: https://news.ycombinator.com/item?id=17015528 https://news.ycombinator.com/item?id=17015528 > The first job of automated driving is to drive only on flat surfaces. Doesn't matter why it's not flat. Then you worry about where to go. That's how the off-road DARPA Grand Challenge vehicles had to work. "Flat road" is a pure geometry thing. Not much AI needed. Why use "AI" to solve a problem you don't have in the first place? This is the wrong tool for the wrong job. (Compare also GPT-2 chess, in which a model trained on transcripts of chess games isn't able to stop itself from making illegal moves. The solution is straightforward - give your chess model a model of the rules of chess! And don't attempt to drive into spaces you cannot physically occupy.)
- gpm 6y ago
  > You don't need any training to avoid hitting stationary objects. You do, because you need to correctly conclude that there is going to be a stationary object there with high probability. As it turns out, that's difficult to do perfectly.
  - thaumasiotes 6y ago
    I'm having difficulty following your train of thought. It can be difficult to make conclusions about the future position of mobile objects. It can't be difficult to make conclusions about the current position of stationary objects.
    - gpm 6y ago
      > It can't be difficult to make conclusions about the current position of stationary objects. You need to know that there is a stationary object in the first place, of a kind you care about (not a plastic bag). I.e. that it's not actually a moving object, an optical illusion, etc. You also need to know that you are going to end up moving through it (it's not a parked car that you will avoid by following your lane). That's... not trivial. (going to be was probably poor phrasing on my part, sorry)
mike_d 6y ago
It isn't an AI bug, it is a lack of sensor input for the AI to work on. Elon's insistence that he can throw a couple extra cameras on a car and perform at a level of safety equal to what Google has figured out requires a dozen LIDAR sensors is criminal.
AshamedCaptain 6y ago
And this is for me a dealkiller issue. Because it means that there are going to be ways to e.g. obscure traffic signs in ways that no human will notice, leading to very nightmarish scenarios where "you expect vs the AI expects" and ... crash. In my opinion until this is somehow solved "full self driving" in presence of other human drivers sharing the road is just a dream.

by gpm 6y ago

shagie 6y ago

The issue is that the AI will do the exact same thing each time it is presented with the same data (crash) whereas a human could go to remedial driver's education. And taking away the license of one inattentive driver gets them (theoretically) off the road... whereas if you have an AI crash into a fire truck and you said "stop it" - you still have all the other identical deployments out there with the same flaw. And "add some partial blockage by a vehicle in the lane, retrain the AI, and then redeploy it" isn't a proper answer... wouldn't that be a different model AI that would need to be validated again?

deleted 6y ago
[deleted]
gpm 6y ago
You can turn this around though. It's harder to teach the AI, sure. But once you've taught it, it stays taught. With humans you have to deal with the continually renewing stream of young people who are new to driving. > And "add some partial blockage by a vehicle in the lane, retrain the AI, and then redeploy it" isn't a proper answer... wouldn't that be a different model AI that would need to be validated again? You're the one adding unnecessary constraints here. If the validation process harms more than helps, you don't do it. In all likelyhood the correct validation process is something lightweight that can be done quickly for simple changes. This sort of training process is likely to be a continuous process, they're always ingesting new data, trying to learn how to drive better from it (and keep up with how patterns in the world change), and therefore continually deploying new models. Because that makes driving safer overall. Not a "this bad thing happened so let's figure out how to put out a bugfix for it".
AshamedCaptain 6y ago
> The issue is that the AI will do the exact same thing each time it is presented with the same data That is not true at all these days, not to mention that even a slight change in illumination will result in completely different input data, the way current AIs work.
deleted 6y ago
[deleted]

thaumasiotes 6y ago

> crashing on a rare edge case (i.e. not well represented in the training data) This isn't a case where it makes any sense to try to extrapolate from training data and call it a day. You don't need any training to avoid hitting stationary objects. Compare this comment from 2018: https://news.ycombinator.com/item?id=17015528 https://news.ycombinator.com/item?id=17015528 > The first job of automated driving is to drive only on flat surfaces. Doesn't matter why it's not flat. Then you worry about where to go. That's how the off-road DARPA Grand Challenge vehicles had to work. "Flat road" is a pure geometry thing. Not much AI needed. Why use "AI" to solve a problem you don't have in the first place? This is the wrong tool for the wrong job. (Compare also GPT-2 chess, in which a model trained on transcripts of chess games isn't able to stop itself from making illegal moves. The solution is straightforward - give your chess model a model of the rules of chess! And don't attempt to drive into spaces you cannot physically occupy.)

gpm 6y ago
> You don't need any training to avoid hitting stationary objects. You do, because you need to correctly conclude that there is going to be a stationary object there with high probability. As it turns out, that's difficult to do perfectly.
- thaumasiotes 6y ago
  I'm having difficulty following your train of thought. It can be difficult to make conclusions about the future position of mobile objects. It can't be difficult to make conclusions about the current position of stationary objects.
  - gpm 6y ago
    > It can't be difficult to make conclusions about the current position of stationary objects. You need to know that there is a stationary object in the first place, of a kind you care about (not a plastic bag). I.e. that it's not actually a moving object, an optical illusion, etc. You also need to know that you are going to end up moving through it (it's not a parked car that you will avoid by following your lane). That's... not trivial. (going to be was probably poor phrasing on my part, sorry)

mike_d 6y ago

It isn't an AI bug, it is a lack of sensor input for the AI to work on. Elon's insistence that he can throw a couple extra cameras on a car and perform at a level of safety equal to what Google has figured out requires a dozen LIDAR sensors is criminal.

AshamedCaptain 6y ago

And this is for me a dealkiller issue. Because it means that there are going to be ways to e.g. obscure traffic signs in ways that no human will notice, leading to very nightmarish scenarios where "you expect vs the AI expects" and ... crash. In my opinion until this is somehow solved "full self driving" in presence of other human drivers sharing the road is just a dream.