4 ms·
>This means that even the designer of a neural network cannot know, once that network has been trained, exactly how it is doing what it does. This is just fals
by rdlecler1 9y ago
>This means that even the designer of a neural network cannot know, once that network has been trained, exactly how it is doing what it does.
This is just false. The reason neural networks seem so mysterious is because in a fully connected neural network a large portion of the interactions (non zero w_ij in the weight matrix) are completely spurious. We look at the fully connected network topology and we throw our hands up. We can apply algorithms to trim out the spurious weights and what we’re left with is a logical circuit that we can analyze. Show an electrical engineer the circuit diagram of a 3 bit added and she’ll know exactly the function. Add a bunch of spurious circuits to that same diagram and of course it’s not going to make sense.
- kazinator 9y agoIf those elements of the graph are "spurious", why not remove them from the very beginning? Or, should I say, not have them there?
- throwaway2048 9y agoBecause we dont know which ones are spurious and the parent is pretty optimistic about the idea of being able to analyize which ones are/are not.
- rdlecler1 9y agoWe only know which ones are spurious if we run the trimming algorithms. I wrote a paper about 10 years ago showing how to use generic algorithms on the weight matrix to remove spurious interactions. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2538912/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2538912/
- deleted 9y ago[deleted]
- DonaldFisk 9y agoGenetic algorithms?
- rdlecler1 9y agoYes, sorry. Typo.
- kazinator 9y agoDo we know which ones are absolutely spurious? Or do we just know which ones are spurious with respect to a particular result that we got? Like for this image of Grandma, these cells are recognizing it, so the others are spurious. Oops, for this other image of Grandma, a different subset are recognizing it. If, say, the same 90% of the structure is not involved in recognizing any instances, then why wouldn't we have made the whole thing 10X smaller in the first place and trained that much smaller network? Or why don't we make it 10X smaller now? Trim away the dead weight and make it lighter and faster.
- rdlecler1 9y agoThat’s what the genetic algorithm is for. You essentially add a entropy function that trims connections. If the performance is neutral or improves then the leaner network topology is subject to positive selection in a population.
- hinkley 9y agoIf removing all of the spurious circuits is “cutting the bullshit” then what is left over is the truth. Distilling a problem to its essence. Isn’t that the hardest step in explaining it?