3 ms·
The bitter-er lesson is that distillation from bigger models works pretty damn well. It’s great news for the GPU poor, not great for the guys training the model
by janalsncm 1y ago
The bitter-er lesson is that distillation from bigger models works pretty damn well. It’s great news for the GPU poor, not great for the guys training the models we distill from.
- rfv6723 1y agoDistillation is great for researchers and hobbyists. But nearly all frontier models have anti-distillation ToS, so distillation is out of question for western commercial companies like Apple.
- janalsncm 1y agoEven if Apple needs to train an LLM from scratch, they can distill it and deploy on edge devices. From that point, inference is free to them.