3 ms·
It's not a 'black box'. We know what goes in: matrix multiplication and simple math, in millions of similar units. We can also probe the net to know what a neur
by visarga 9y ago
It's not a 'black box'. We know what goes in: matrix multiplication and simple math, in millions of similar units. We can also probe the net to know what a neuron does, or what a specific configuration means - but it might be disappointing - each individual neuron or weight might not have much meaning on its own and even if it were removed, the neural net world work just as well (we can remove almost 95% of the neurons and still make it work).
We can also perturb the inputs to study how the outputs depend on them. There are many things we can do to probe a neural net. We might not be able to say for sure what situation will make it fail, but we can try it with millions of tests to see how good it is. That's how neural nets are rated, actually - by testing on loads of new data, they haven't seen during training.
On the other hand, human brains are really black boxes. We know much less about how they work or how they reach a conclusion. The fact that we can self-report is not the same as knowing how a human would behave in any reasonable situation. We don't actually know how people will behave. And yet, a few people do have access to incredible power to do harm to humanity. We have had to live with that fear since the invention of atomics and genetic viruses.
So I don't think the argument that "neural nets are black boxes" is so powerful. As opposed to what, I would say? Even our dear president is such a black box. We have no idea how it works, or what he will do next.