3 ms·
Not sure if scam, honestly depends on the data sometimes it might work.
by barrenko 2y ago
Not sure if scam, honestly depends on the data sometimes it might work.
- rlforllms 2y agoThe goal is distillation is to distill into smaller models like 7B, 1.5B. They didn't even change the model size, let alone try a different class of models. Getting expert model's trajectories is trivial if you have vLLM to do batched inference.