5 ms·

This sounds far fetch and sci fi, but intelligible machines are particularly of importance in avoiding machines becoming self aware unbeknownst. However, I thi

This sounds far fetch and sci fi, but intelligible machines are particularly of importance in avoiding machines becoming self aware unbeknownst. However, I think it's also possible that we may never be able to create machines that are self aware as people are unless we do allow them to build without our understanding of them. There could be a recursive relationship in that an intelligence is not capable of understanding itself.

flippyhead 10y ago
We could just wait until we completely understand how the human brain works, then replicate it directly. It would take longer but be (supposedly) safer than going for a novel but faster approach involving machine learning, genetic algorithms and the like.
- willismichael 10y ago
  We already have the technology to replicate the human brain, you and I and everybody else were born with it.
- calgoo 10y ago
  I think tech is helping us understand how some of the brain works, or at least helps us to model it. (my own idea, no "proof" or sources)
danieldk 10y ago
I think self-awareness is still so far off that it is not the most pressing problem. The more serious problem is that once we make life-changing choices using machine learning models, the models should be just and accountable. Obviously, these are vague terms, but in terms of machine learning models they can have concrete meanings: * Just: choices should be made on what we consider (morally) relevant inputs. * Accountable: choices should be traceable to inputs and parameters (or eventually training data). E.g. suppose that we find drones that stun people with a weapon acceptable. Such a drone would be just if it decides to stun a person because they have a weapon. Such a drone would be unjust if it decides to stun a person because they have a particular skin color [1]. Now, suppose that such a drone stuns a person robbing a supermarket with a fake weapon. Regardless of whether we consider this to be just, we probably want to know why the drone misrecognized the gun as real (the model should be accountable). [1] There may be a non-uniform prior distribution p(weapon_use|race). But I hope we all agree that using such prior distributions would be absolutely unfair towards the individual.
- fixermark 10y ago
  This gets down to the notion of EU "Right to Explanation." We can make all the anti-bigotry laws we want, it won't do any good if people can lock illegal profiling into machine-learning algorithms and excuse bad effects with "The machine said do it, so I did it."
- blazespin 10y ago
  Well, we hope it is. I don't think we can say that with much confidence though unless we understand what the computers are doing. Take some massive datacenter with petabyte sized deep networks doing neural networks. Think of the dimensionality of those matricies. Do you really know what's going on??
  - danieldk 10y ago
    Think of the dimensionality of those matricies. Do you really know what's going on? As someone who uses deep networks for language processing, I completely agree. IMHO this implies that we should be careful making life-changing decisions using such models. Though, I am not sure whether legislation at this point is useful either. The lack of understanding of new technology in legislative branches typically leads to bad laws.
  - jimbokun 10y ago
    "Do you really know what's going on??" No, so its probably a bad idea to trust such a system to make important, consequential decisions. Tagging faces in photos, recommending books or movies, probably fine. Making an independent decision to fire a weapon, probably very bad idea.
  - hx87 10y ago
    To be fair, we don't know what going on in that bundle of several billion interconnect neurons between everyone's ears; what the OP refers to as "accountability" is actually process transparency, and we certainly don't have process transparency for individual humans the way we do for human institutions. We do have accountability though--the ability to recognize mistakes and learn from them (with incentives, i.e. rewards and punishments). Machine learning can also have this sort of accountability--if you fuck up, the fuckup is another data point to train on, perhaps weighted for the seriousness of the fuckup.

flippyhead 10y ago

We could just wait until we completely understand how the human brain works, then replicate it directly. It would take longer but be (supposedly) safer than going for a novel but faster approach involving machine learning, genetic algorithms and the like.

willismichael 10y ago
We already have the technology to replicate the human brain, you and I and everybody else were born with it.
calgoo 10y ago
I think tech is helping us understand how some of the brain works, or at least helps us to model it. (my own idea, no "proof" or sources)

danieldk 10y ago

I think self-awareness is still so far off that it is not the most pressing problem. The more serious problem is that once we make life-changing choices using machine learning models, the models should be just and accountable. Obviously, these are vague terms, but in terms of machine learning models they can have concrete meanings: * Just: choices should be made on what we consider (morally) relevant inputs. * Accountable: choices should be traceable to inputs and parameters (or eventually training data). E.g. suppose that we find drones that stun people with a weapon acceptable. Such a drone would be just if it decides to stun a person because they have a weapon. Such a drone would be unjust if it decides to stun a person because they have a particular skin color [1]. Now, suppose that such a drone stuns a person robbing a supermarket with a fake weapon. Regardless of whether we consider this to be just, we probably want to know why the drone misrecognized the gun as real (the model should be accountable). [1] There may be a non-uniform prior distribution p(weapon_use|race). But I hope we all agree that using such prior distributions would be absolutely unfair towards the individual.

fixermark 10y ago
This gets down to the notion of EU "Right to Explanation." We can make all the anti-bigotry laws we want, it won't do any good if people can lock illegal profiling into machine-learning algorithms and excuse bad effects with "The machine said do it, so I did it."
blazespin 10y ago
Well, we hope it is. I don't think we can say that with much confidence though unless we understand what the computers are doing. Take some massive datacenter with petabyte sized deep networks doing neural networks. Think of the dimensionality of those matricies. Do you really know what's going on??
- danieldk 10y ago
  Think of the dimensionality of those matricies. Do you really know what's going on? As someone who uses deep networks for language processing, I completely agree. IMHO this implies that we should be careful making life-changing decisions using such models. Though, I am not sure whether legislation at this point is useful either. The lack of understanding of new technology in legislative branches typically leads to bad laws.
- jimbokun 10y ago
  "Do you really know what's going on??" No, so its probably a bad idea to trust such a system to make important, consequential decisions. Tagging faces in photos, recommending books or movies, probably fine. Making an independent decision to fire a weapon, probably very bad idea.
- hx87 10y ago
  To be fair, we don't know what going on in that bundle of several billion interconnect neurons between everyone's ears; what the OP refers to as "accountability" is actually process transparency, and we certainly don't have process transparency for individual humans the way we do for human institutions. We do have accountability though--the ability to recognize mistakes and learn from them (with incentives, i.e. rewards and punishments). Machine learning can also have this sort of accountability--if you fuck up, the fuckup is another data point to train on, perhaps weighted for the seriousness of the fuckup.