3 ms·
Understanding the role of individual units in a deep neural network
- lamename 6y agoPerturbing units and observing change is so much like the early days of neurology and neuroscience. Lesion studies in post-surgery patients or those with stroke or brain damage told us a lot, removing part of the circuit and observing what changed.
- signa11 6y agoyup. i remember reading about something called ‘optimal brain damage’ which aims at pruning non-essential weights from the network for faster training etc.
- moritzmeister 6y agoWe also think there should be more Ablation Studies in deep learning research. To make it easier, we are working on automating the process of disabling components of models with a framework called Maggy. Pass in your Keras Model and specify which layes/components to mask out and we will generate all models and train them in parallel. https://databricks.com/session_eu20/parallel-ablation-studies-for-machine-learning-with-maggy-on-apache-spark https://databricks.com/session_eu20/parallel-ablation-studie... https://github.com/logicalclocks/maggy https://github.com/logicalclocks/maggy
- mrfusion 6y agoThe most interesting use cases for this would be applying it to a game playing AI. We might be able to find novel strategies to games. Also interesting to find out how the network that solved protein folding works.
- dash2 6y agoThis is all super cool, but for those of you who tl:dr;ed, check out their application - creating a "concept painting" app. You can literally click "door" and spray on the picture, and it'll put a door there for you. http://gandissect.csail.mit.edu http://gandissect.csail.mit.edu http://gandissect.res.ibm.com/ganpaint.html?project=churchoutdoor&layer=layer4 http://gandissect.res.ibm.com/ganpaint.html?project=churchou...
- deleted 6y ago[deleted]