8 ms·

> Did they pass a driver's license check? Thought experiment: imagine a particular self-driving algo were to do something that would warrant a human driver get

> Did they pass a driver's license check? Thought experiment: imagine a particular self-driving algo were to do something that would warrant a human driver getting their driving licence suspended, would that mean all the cars with that same algo would be simultanously suspended, or just that one car?

alisonkisk 5y ago
qayxc 5y ago
That is a very intriguing scenario indeed. Due to the non-deterministic nature of many self-driving systems, I'd reckon a broader investigation would have to follow. Should that investigation reveal a reproducible behaviour, a recall or general suspension of vehicles running that particular software version would seem like a reasonable response. Otherwise suspending just that one car could be an option. It's uncharted territory both in terms of the legal system and w.r.t accountability so I guess no one has an answer to that yet.
- pxmpxm 5y ago
  How is it non-deterministic? If you run the same trained model on the same input data, you should arrive at the exactly identical output.
  - qayxc 5y ago
    > How is it non-deterministic? It's non-deterministic in a practical sense; you cannot reproduce the exact circumstances that caused a certain behaviour. Maybe a bug flew by one of the cameras at a certain moment, or a pedestrian with certain type of clothing passed by. You get the idea.
    - eru 5y ago
      Depends. In principle you can log exactly what data the camera sends to the decision making unit. And you could replay that data. The camera itself might not be deterministic enough to reproduce the same data with the same beetle, but replaying the log should work. (Keep the log in a 'black box' on the car, and perhaps only keep the last x hours of data, if necessary.) I think Waymo already does something like these kinds of replays.
      - joshribakoff 5y ago
        what you’re referring to is just simulating, at least after the point your modified algorithm does something slightly different in the simulator than the real car did in real life.
        eru 5y ago
        I wondered about the last point, too. But Waymo apparently worked out a satisfactory solution to that.
  - dahfizz 5y ago
    I think 'chaotic' is the technical term. It is technically deterministic, but the system is incomprehensibly complex and sensitive, such that it may as well be non deterministic / pseudorandom. You could drive the same car down the same road at the same time tomorrow, and it may behave differently. It makes bugs / edges cases impossible to reproduce and fix. With a human you can just say "why did you do that? Don't do that." Little fine tuned tweaks to neural nets is not so easy.
hakuseki 5y ago
It does sound silly at first, but I think suspending the one car is actually a rather elegant solution on several counts. 1. Easy to implement 2. Doesn't punish fleet scale (compare: suspending the whole fleet means a fleet that's twice as large would be suspended twice as often) 3. Punishes the fleet owner in proportion to the frequency of the error 4. Is exactly correct when the cause is really a hardware problem specific to the car
- tiku 5y ago
  But the software "that did the crime" is all the same..
  - hakuseki 5y ago
    Yes, and that's why suspending the one car might seem counterintuitive. But that doesn't mean it's a bad solution.
    - AvocadoPanic 5y ago
      Can we have lawn darts back if we suspend the darts that maim or kill?
      - eru 5y ago
        You can't even have Kinder Surprise..
        AvocadoPanic 5y ago
        Sometimes the Indian supermarket has contraband imported Kinder Surprise, not the shitty domestic versions.
      - FabHK 5y ago
        Just sell guns without any controls whatsoever, and if a gun is used to shoot someone, impound that gun. Problem solved. Right?
paulcole 5y ago
Easy. All of the cars with that algo would be simultaneously suspended.
flohofwoe 5y ago
That's easy and obvious: The whole fleet is "grounded" (or at least the self-driving feature disabled) until the bug has been identified and fixed. If you ask me, only a single car misbehaving is actually much more alarming than if the bug manifests in all cars of that type.
- KptMarchewa 5y ago
  >If you ask me, only a single car misbehaving is actually much more alarming than if the bug manifests in all cars of that type. It's not if it's hardware failure.
  - bee_rider 5y ago
    A hardware failure that puts the vehicle in a semi-functioning, but non-safe condition needs to be investigated, it could be silently lurking in any number of cars.
    - eru 5y ago
      Investigated, yes. Grounding the whole fleet? Not necessarily. It's a cost-benefit trade-off. We don't ground the whole fleet of human drivers, when one of them crashes their car, because the benefit doesn't outweigh the costs. It's not a given that the answer will change that much for self-driving cars. Especially for extremely rare failure modes.
      - bee_rider 5y ago
        Humans all have different versions of the 'driving' software installed, there's no way of identifying which ones will misbehave. If Teslas are malfunctioning, we shouldn't ground Chevy Cruise. We exist on a spectrum of competence. If a type of car has a widespread issue that unexpectedly gets it into a state where a minimally competent non-impaired person can't operate it safely, that type of car should be pulled from the road. However, because humans are so sloppy (0 human drivers should be the ultimate goal), it is generally assumed that most accidents are caused by us, and so we require a significant accumulation of evidence to take a type of car off the road. Without a human in the loop, what's the unreliable part to point the finger at?
notatoad 5y ago
Waymo's marketing would imply they think the whole fleet should be suspended in this case. They call their system the driver, and seem to anthropomorphise it. But more practically: what could a self-driving car do to get it's licence suspended? Afaik That's a pretty rare punishment outside of two cases: drunk driving or street racing.
- toast0 5y ago
  Many (most?) states have a point based system for suspending licenses based on moving violations. You tend to collect points for things like speeding, failure to follow traffic signals, or illegal turns. Get enough points and your license is suspended for some period. Other states give you a number of points, and you lose points for infractions, get to zero and suspension.
- ufmace 5y ago
  It's a rare normal punishment for any single crime. But it's a pretty common escalation for repeated failure to cooperate with the system. A ticket for speeding or running a stop sign might be just a fine. But if you start no-showing your court dates, not making the payments, keep raking up more moving violations without properly resolving any of them, then you might find your license getting suspended.
deleted 5y ago
[deleted]

alisonkisk 5y ago

qayxc 5y ago

That is a very intriguing scenario indeed. Due to the non-deterministic nature of many self-driving systems, I'd reckon a broader investigation would have to follow. Should that investigation reveal a reproducible behaviour, a recall or general suspension of vehicles running that particular software version would seem like a reasonable response. Otherwise suspending just that one car could be an option. It's uncharted territory both in terms of the legal system and w.r.t accountability so I guess no one has an answer to that yet.

pxmpxm 5y ago
How is it non-deterministic? If you run the same trained model on the same input data, you should arrive at the exactly identical output.
- qayxc 5y ago
  > How is it non-deterministic? It's non-deterministic in a practical sense; you cannot reproduce the exact circumstances that caused a certain behaviour. Maybe a bug flew by one of the cameras at a certain moment, or a pedestrian with certain type of clothing passed by. You get the idea.
  - eru 5y ago
    Depends. In principle you can log exactly what data the camera sends to the decision making unit. And you could replay that data. The camera itself might not be deterministic enough to reproduce the same data with the same beetle, but replaying the log should work. (Keep the log in a 'black box' on the car, and perhaps only keep the last x hours of data, if necessary.) I think Waymo already does something like these kinds of replays.
    - joshribakoff 5y ago
      what you’re referring to is just simulating, at least after the point your modified algorithm does something slightly different in the simulator than the real car did in real life.
      - eru 5y ago
        I wondered about the last point, too. But Waymo apparently worked out a satisfactory solution to that.
- dahfizz 5y ago
  I think 'chaotic' is the technical term. It is technically deterministic, but the system is incomprehensibly complex and sensitive, such that it may as well be non deterministic / pseudorandom. You could drive the same car down the same road at the same time tomorrow, and it may behave differently. It makes bugs / edges cases impossible to reproduce and fix. With a human you can just say "why did you do that? Don't do that." Little fine tuned tweaks to neural nets is not so easy.

hakuseki 5y ago

It does sound silly at first, but I think suspending the one car is actually a rather elegant solution on several counts. 1. Easy to implement 2. Doesn't punish fleet scale (compare: suspending the whole fleet means a fleet that's twice as large would be suspended twice as often) 3. Punishes the fleet owner in proportion to the frequency of the error 4. Is exactly correct when the cause is really a hardware problem specific to the car

tiku 5y ago
But the software "that did the crime" is all the same..
- hakuseki 5y ago
  Yes, and that's why suspending the one car might seem counterintuitive. But that doesn't mean it's a bad solution.
  - AvocadoPanic 5y ago
    Can we have lawn darts back if we suspend the darts that maim or kill?
    - eru 5y ago
      You can't even have Kinder Surprise..
      - AvocadoPanic 5y ago
        Sometimes the Indian supermarket has contraband imported Kinder Surprise, not the shitty domestic versions.
    - FabHK 5y ago
      Just sell guns without any controls whatsoever, and if a gun is used to shoot someone, impound that gun. Problem solved. Right?

paulcole 5y ago

Easy. All of the cars with that algo would be simultaneously suspended.

flohofwoe 5y ago

That's easy and obvious: The whole fleet is "grounded" (or at least the self-driving feature disabled) until the bug has been identified and fixed. If you ask me, only a single car misbehaving is actually much more alarming than if the bug manifests in all cars of that type.

KptMarchewa 5y ago
>If you ask me, only a single car misbehaving is actually much more alarming than if the bug manifests in all cars of that type. It's not if it's hardware failure.
- bee_rider 5y ago
  A hardware failure that puts the vehicle in a semi-functioning, but non-safe condition needs to be investigated, it could be silently lurking in any number of cars.
  - eru 5y ago
    Investigated, yes. Grounding the whole fleet? Not necessarily. It's a cost-benefit trade-off. We don't ground the whole fleet of human drivers, when one of them crashes their car, because the benefit doesn't outweigh the costs. It's not a given that the answer will change that much for self-driving cars. Especially for extremely rare failure modes.
    - bee_rider 5y ago
      Humans all have different versions of the 'driving' software installed, there's no way of identifying which ones will misbehave. If Teslas are malfunctioning, we shouldn't ground Chevy Cruise. We exist on a spectrum of competence. If a type of car has a widespread issue that unexpectedly gets it into a state where a minimally competent non-impaired person can't operate it safely, that type of car should be pulled from the road. However, because humans are so sloppy (0 human drivers should be the ultimate goal), it is generally assumed that most accidents are caused by us, and so we require a significant accumulation of evidence to take a type of car off the road. Without a human in the loop, what's the unreliable part to point the finger at?

notatoad 5y ago

Waymo's marketing would imply they think the whole fleet should be suspended in this case. They call their system the driver, and seem to anthropomorphise it. But more practically: what could a self-driving car do to get it's licence suspended? Afaik That's a pretty rare punishment outside of two cases: drunk driving or street racing.

toast0 5y ago
Many (most?) states have a point based system for suspending licenses based on moving violations. You tend to collect points for things like speeding, failure to follow traffic signals, or illegal turns. Get enough points and your license is suspended for some period. Other states give you a number of points, and you lose points for infractions, get to zero and suspension.
ufmace 5y ago
It's a rare normal punishment for any single crime. But it's a pretty common escalation for repeated failure to cooperate with the system. A ticket for speeding or running a stop sign might be just a fine. But if you start no-showing your court dates, not making the payments, keep raking up more moving violations without properly resolving any of them, then you might find your license getting suspended.

deleted 5y ago

[deleted]