We can understand how and why neural networks work. The issue is that obtaining this understanding requires a lot of effort. OpenAI invested in trying to unde

We can understand how and why neural networks work. The issue is that obtaining this understanding requires a lot of effort. OpenAI invested in trying to understand how a simple CNN works. You can read their results here: https://distill.pub/2020/circuits/ https://distill.pub/2020/circuits/ > In the original narrative of deep learning, each neuron builds progressively more abstract, meaningful features by composing features in the preceding layer. In recent years, there’s been some skepticism of this view, but what happens if you take it really seriously? > InceptionV1 is a classic vision model with around 10,000 unique neurons — a large number, but still on a scale that a group effort could attack. What if you simply go through the model, neuron by neuron, trying to understand each one and the connections between them? The circuits collaboration aims to find out.