4 ms·

> A fully autonomous but non-AGI car could simply do what humans do: pull over, turn around, or call for help. Right or wrong, there is going to be (as you can

> A fully autonomous but non-AGI car could simply do what humans do: pull over, turn around, or call for help. Right or wrong, there is going to be (as you can already see on Tesla) disproportionately harsh response to the AI failing in a car, doubly so if it results in an accident. Humans mess up, and if someone is found to be at fault, they are punished. We also feel compassion to other human beings. E.g. killing another human, doubly so when by accident (ie not premeditated), is likely haunt anyone to the end of their own life. A machine is afforded no such sympathy, and if anything, the machine is looked at with vastly increased suspicion and scrutiny because it lacks empathy. We have to face reality here. If the current implementations do a hack job and just handwave problems away: "meh, humans make mistakes too", they run a very strong risk that there is going to be a big backlash, people in general will write self driving cars off as fundamentally unsafe and set progress back 50 years. A technology like this has to adapt to humans and our biases, the other way around is not happening.

leereeves 7y ago
How did you get from "pull over, turn around, or call for help" to "meh, humans make mistakes too"? A completely non-AI self-driving solution could be cautious and responsible, and sometimes the cautious, responsible thing to do is pull over. So while what you say is true, I don't think it's relevant to whether or not AGI is required for self-driving.
- short_sells_poo 7y ago
  How do you make the system decide that a situation is uncertain? It might have high conviction in an outcome only because it lacks general understanding. You cannot just handwave the problem away saying "oh just pull over when the system doesn't know". You are ignoring the most difficult class of errors, high conviction but incorrect predictions. This is not a question of reducing complexity, as even very simple statistical tools will sometimes give you high conviction (and yet incorrect) predictions because they are simplified models. The assumption that error conditions are easier to recognize and control than normal operating conditions is insidious. For example, an AI could misidentify a patch of ice with very high conviction that it is line marking. A human could too, but we are talking about an AI that has to recognize when it's unsafe and pull over (as you wrote). I'm sure you've seen the case where an NN taught to recognize wolves did so by picking up the presence of snow as the sole feature in a photo. Only because that happened to be the most distinguishing feature of images with a wolf on it! That AI would only recognize a wolf as long as the photo had some snow. It'd also of course misidentify mere snowy landscapes as a wolf. The mistake is silly, but issues like this are very difficult to diagnose once the inputs become more complex.

leereeves 7y ago

How did you get from "pull over, turn around, or call for help" to "meh, humans make mistakes too"? A completely non-AI self-driving solution could be cautious and responsible, and sometimes the cautious, responsible thing to do is pull over. So while what you say is true, I don't think it's relevant to whether or not AGI is required for self-driving.

short_sells_poo 7y ago
How do you make the system decide that a situation is uncertain? It might have high conviction in an outcome only because it lacks general understanding. You cannot just handwave the problem away saying "oh just pull over when the system doesn't know". You are ignoring the most difficult class of errors, high conviction but incorrect predictions. This is not a question of reducing complexity, as even very simple statistical tools will sometimes give you high conviction (and yet incorrect) predictions because they are simplified models. The assumption that error conditions are easier to recognize and control than normal operating conditions is insidious. For example, an AI could misidentify a patch of ice with very high conviction that it is line marking. A human could too, but we are talking about an AI that has to recognize when it's unsafe and pull over (as you wrote). I'm sure you've seen the case where an NN taught to recognize wolves did so by picking up the presence of snow as the sole feature in a photo. Only because that happened to be the most distinguishing feature of images with a wolf on it! That AI would only recognize a wolf as long as the photo had some snow. It'd also of course misidentify mere snowy landscapes as a wolf. The mistake is silly, but issues like this are very difficult to diagnose once the inputs become more complex.