3 ms·
For people who want to dig deeper: The fancy ML term-of-art for the practice of cutting out a piece of a neural network and measuring the resulting effect on it
by Centigonal 1y ago
For people who want to dig deeper: The fancy ML term-of-art for the practice of cutting out a piece of a neural network and measuring the resulting effect on its performance is an ablation study.
Since around 2018, ablation has been an important tool to understand the structure and function of ML models, including LLMs. Searching for this term in papers about your favorite LLMs is a good way to learn more.
- magicalhippo 1y agoIt's my understanding that dropout[1] is also an important aspect of training modern neural nets. When using dropout you intentionally remove some random number of nodes ("neurons") from the network during a training step. By constantly changing which nodes are dropped during training, you effectively force delocalization and so it seems to me somewhat unsurprising that the resulting network is resilient to local perturbations. [1]: https://towardsdatascience.com/dropout-in-neural-networks-47a162d621d9/ https://towardsdatascience.com/dropout-in-neural-networks-47...