3 ms·

While it's true it should already be considered a kind of failure to get into this situation in the first place, that shouldn't get in the way of "hope for the

While it's true it should already be considered a kind of failure to get into this situation in the first place, that shouldn't get in the way of "hope for the best, plan for the worst." It's never going to be possible to anticipate every kind of condition and have perfect information. Ideally we want these systems to engage in whatever harm and risk mitigation is believed to be possible based on the current reality, we don't want the system saying to itself, "The real solution is for me to go back in time and not have made some earlier mistake." That analysis is great afterwards for figuring out how to prevent in the system from getting into that situation again in the future, so it's still important too, but we need both.

danShumway 5y ago
Thought experiment: Let's assume that a burglar has broken into someone's house, but Amazon Alexa knows the burglar is unarmed and that the owner is armed. Alexa has IDed the burglar and knows that they're a college kid who is unlikely to cause injury or attack someone. Alexa looks up the kid in police databases and identifies that they do not have a criminal record. The owner of the house is trying to hunt down the burglar with a gun. The owner asks Alexa where the burglar is currently hiding. Alexa knows the average response time of police officers in that neighborhood, and knows that if they're called, they'll arrive too late to intervene in the situation. Should Amazon Alexa: A) remain neutral and refuse to answer, letting the situation play out as it may? B) lie to the owner so that the kid can escape? C) threaten the owner with revealing an embarrassing secret so that they'll let the kid go? D) tell the truth and let the owner find the kid? And does it bother you that Amazon doesn't have a public policy available about what Alexa should do in this situation? Does it bother you that Amazon hasn't prepared for the worst in that situation? That Amazon is selling Alexa devices even though we haven't had a public debate about it or put any public funding into answering this question? ---- The problem with the questions on this site is not that they're preparing for the worst, it's that they're preparing for the absurd. They're saying we should build capabilities to prepare for situations that devices shouldn't be in to start with, that devices shouldn't realistically be making decisions about, and that devices should respond to those situations based on information that shouldn't be available to them using capabilities that they don't have. A "prepare for the worst" policy for self-driving cars as they exist today is realistically "slam on the breaks, swerve the car without over-correcting the wheel and going into oncoming traffic." That is a significant improvement over human drivers today, none of which are capable of forming split-second philosophical decisions. Simple consistent rules like "brake, swerve but don't cross oncoming lanes" are good enough. Realistically, they're good enough forever, we never need to ask these philosophical questions. All of the energy spent debating them can be poured into decreasing car brake time. And by positing a scenario where a car is able to identify people's age and demographics in milliseconds, we're entering into a world where there really is zero excuse for why the car got into that situation to begin with. We're positing a scenario where a car somehow has godlike powers to make split-second philosophical decisions, but where they can't avoid accidentally speeding up through a cross-walk. I don't think that's realistic, and I think it's a waste of time to worry about. If it doesn't bother you that Alexa doesn't have a philosophical threshold around burglar age/danger where it will decide whether or not to sell out that someone's hiding under the stairs from an owner with a gun, then it also probably shouldn't bother you that your Tesla doesn't have a special mode where it decides whether or not to swerve into a pedestrian based on how old they are. Because both of those scenarios are equally ridiculous, and there are some worst-case scenarios that are so ridiculous that it's a waste of time to even think about them. A car shouldn't have the capabilities to make any of the decisions we're talking about. If engineers start building those capabilities then something has gone clearly wrong and we all need to stop driving the car that does socioeconomic profiling of the people around it. I guarantee you that any car that is designed to do that is also doing some horrifying surveillance crap that should be opposed at every turn. And if the capabilities ever actually do get built for some reason (which they shouldn't), and do work reliably (which they won't), and are capable of being used in these situations where split-second decisions matter, then engineers should be going to jail for negligence every time that car crashes, because any system that advanced could also avoid crashing in the first place.
- dataangel 5y ago
  It doesn't bother me that Amazon doesn't have a policy for this kind of scenario because it depends on several science fiction capabilities and several practicalities that don't currently exist. * Alexa has no idea what average police response time is. * Alexa does not have access to any police database. * Alexa has no idea that the burglar is a burglar or that they are a threat. * Alexa does not even know that you are you.

danShumway 5y ago

Thought experiment: Let's assume that a burglar has broken into someone's house, but Amazon Alexa knows the burglar is unarmed and that the owner is armed. Alexa has IDed the burglar and knows that they're a college kid who is unlikely to cause injury or attack someone. Alexa looks up the kid in police databases and identifies that they do not have a criminal record. The owner of the house is trying to hunt down the burglar with a gun. The owner asks Alexa where the burglar is currently hiding. Alexa knows the average response time of police officers in that neighborhood, and knows that if they're called, they'll arrive too late to intervene in the situation. Should Amazon Alexa: A) remain neutral and refuse to answer, letting the situation play out as it may? B) lie to the owner so that the kid can escape? C) threaten the owner with revealing an embarrassing secret so that they'll let the kid go? D) tell the truth and let the owner find the kid? And does it bother you that Amazon doesn't have a public policy available about what Alexa should do in this situation? Does it bother you that Amazon hasn't prepared for the worst in that situation? That Amazon is selling Alexa devices even though we haven't had a public debate about it or put any public funding into answering this question? ---- The problem with the questions on this site is not that they're preparing for the worst, it's that they're preparing for the absurd. They're saying we should build capabilities to prepare for situations that devices shouldn't be in to start with, that devices shouldn't realistically be making decisions about, and that devices should respond to those situations based on information that shouldn't be available to them using capabilities that they don't have. A "prepare for the worst" policy for self-driving cars as they exist today is realistically "slam on the breaks, swerve the car without over-correcting the wheel and going into oncoming traffic." That is a significant improvement over human drivers today, none of which are capable of forming split-second philosophical decisions. Simple consistent rules like "brake, swerve but don't cross oncoming lanes" are good enough. Realistically, they're good enough forever, we never need to ask these philosophical questions. All of the energy spent debating them can be poured into decreasing car brake time. And by positing a scenario where a car is able to identify people's age and demographics in milliseconds, we're entering into a world where there really is zero excuse for why the car got into that situation to begin with. We're positing a scenario where a car somehow has godlike powers to make split-second philosophical decisions, but where they can't avoid accidentally speeding up through a cross-walk. I don't think that's realistic, and I think it's a waste of time to worry about. If it doesn't bother you that Alexa doesn't have a philosophical threshold around burglar age/danger where it will decide whether or not to sell out that someone's hiding under the stairs from an owner with a gun, then it also probably shouldn't bother you that your Tesla doesn't have a special mode where it decides whether or not to swerve into a pedestrian based on how old they are. Because both of those scenarios are equally ridiculous, and there are some worst-case scenarios that are so ridiculous that it's a waste of time to even think about them. A car shouldn't have the capabilities to make any of the decisions we're talking about. If engineers start building those capabilities then something has gone clearly wrong and we all need to stop driving the car that does socioeconomic profiling of the people around it. I guarantee you that any car that is designed to do that is also doing some horrifying surveillance crap that should be opposed at every turn. And if the capabilities ever actually do get built for some reason (which they shouldn't), and do work reliably (which they won't), and are capable of being used in these situations where split-second decisions matter, then engineers should be going to jail for negligence every time that car crashes, because any system that advanced could also avoid crashing in the first place.

dataangel 5y ago
It doesn't bother me that Amazon doesn't have a policy for this kind of scenario because it depends on several science fiction capabilities and several practicalities that don't currently exist. * Alexa has no idea what average police response time is. * Alexa does not have access to any police database. * Alexa has no idea that the burglar is a burglar or that they are a threat. * Alexa does not even know that you are you.