3 ms·
This doesn't seem right and most people recognize that 'neurons' encode for multiple activations. https://transformer-circuits.pub/2022/toy_model/index.html htt
by vonneumannstan 1y ago
This doesn't seem right and most people recognize that 'neurons' encode for multiple activations. https://transformer-circuits.pub/2022/toy_model/index.html https://transformer-circuits.pub/2022/toy_model/index.html
- Der_Einzige 1y agoThey’re 1000% right on the idea that most models are hilariously undertrained
- vonneumannstan 1y agoPretty sure that since the Chinchilla paper this probably isn't the case. https://arxiv.org/pdf/2203.15556 https://arxiv.org/pdf/2203.15556