3 ms·

I think self-awareness is still so far off that it is not the most pressing problem. The more serious problem is that once we make life-changing choices using

I think self-awareness is still so far off that it is not the most pressing problem. The more serious problem is that once we make life-changing choices using machine learning models, the models should be just and accountable. Obviously, these are vague terms, but in terms of machine learning models they can have concrete meanings: * Just: choices should be made on what we consider (morally) relevant inputs. * Accountable: choices should be traceable to inputs and parameters (or eventually training data). E.g. suppose that we find drones that stun people with a weapon acceptable. Such a drone would be just if it decides to stun a person because they have a weapon. Such a drone would be unjust if it decides to stun a person because they have a particular skin color [1]. Now, suppose that such a drone stuns a person robbing a supermarket with a fake weapon. Regardless of whether we consider this to be just, we probably want to know why the drone misrecognized the gun as real (the model should be accountable). [1] There may be a non-uniform prior distribution p(weapon_use|race). But I hope we all agree that using such prior distributions would be absolutely unfair towards the individual.

fixermark 10y ago
This gets down to the notion of EU "Right to Explanation." We can make all the anti-bigotry laws we want, it won't do any good if people can lock illegal profiling into machine-learning algorithms and excuse bad effects with "The machine said do it, so I did it."
blazespin 10y ago
Well, we hope it is. I don't think we can say that with much confidence though unless we understand what the computers are doing. Take some massive datacenter with petabyte sized deep networks doing neural networks. Think of the dimensionality of those matricies. Do you really know what's going on??
- danieldk 10y ago
  Think of the dimensionality of those matricies. Do you really know what's going on? As someone who uses deep networks for language processing, I completely agree. IMHO this implies that we should be careful making life-changing decisions using such models. Though, I am not sure whether legislation at this point is useful either. The lack of understanding of new technology in legislative branches typically leads to bad laws.
- jimbokun 10y ago
  "Do you really know what's going on??" No, so its probably a bad idea to trust such a system to make important, consequential decisions. Tagging faces in photos, recommending books or movies, probably fine. Making an independent decision to fire a weapon, probably very bad idea.
- hx87 10y ago
  To be fair, we don't know what going on in that bundle of several billion interconnect neurons between everyone's ears; what the OP refers to as "accountability" is actually process transparency, and we certainly don't have process transparency for individual humans the way we do for human institutions. We do have accountability though--the ability to recognize mistakes and learn from them (with incentives, i.e. rewards and punishments). Machine learning can also have this sort of accountability--if you fuck up, the fuckup is another data point to train on, perhaps weighted for the seriousness of the fuckup.

fixermark 10y ago

This gets down to the notion of EU "Right to Explanation." We can make all the anti-bigotry laws we want, it won't do any good if people can lock illegal profiling into machine-learning algorithms and excuse bad effects with "The machine said do it, so I did it."

blazespin 10y ago

Well, we hope it is. I don't think we can say that with much confidence though unless we understand what the computers are doing. Take some massive datacenter with petabyte sized deep networks doing neural networks. Think of the dimensionality of those matricies. Do you really know what's going on??

danieldk 10y ago
Think of the dimensionality of those matricies. Do you really know what's going on? As someone who uses deep networks for language processing, I completely agree. IMHO this implies that we should be careful making life-changing decisions using such models. Though, I am not sure whether legislation at this point is useful either. The lack of understanding of new technology in legislative branches typically leads to bad laws.
jimbokun 10y ago
"Do you really know what's going on??" No, so its probably a bad idea to trust such a system to make important, consequential decisions. Tagging faces in photos, recommending books or movies, probably fine. Making an independent decision to fire a weapon, probably very bad idea.
hx87 10y ago
To be fair, we don't know what going on in that bundle of several billion interconnect neurons between everyone's ears; what the OP refers to as "accountability" is actually process transparency, and we certainly don't have process transparency for individual humans the way we do for human institutions. We do have accountability though--the ability to recognize mistakes and learn from them (with incentives, i.e. rewards and punishments). Machine learning can also have this sort of accountability--if you fuck up, the fuckup is another data point to train on, perhaps weighted for the seriousness of the fuckup.