3 ms·
This was published here in HN last week: https://news.ycombinator.com/item?id=35810663 https://news.ycombinator.com/item?id=35810663 Don't know if there any pu
by mrbungie 3y ago
This was published here in HN last week: https://news.ycombinator.com/item?id=35810663 https://news.ycombinator.com/item?id=35810663
Don't know if there any public technical reports by any of the big AI companies about this, as its pretty new.
- sanxiyn 3y agoNo, distilling step-by-step https://arxiv.org/abs/2305.02301 https://arxiv.org/abs/2305.02301 distills LLM to task specific model. That works, and I know of multiple successes. But it doesn't relate to choice of optimizing training FLOP vs training and inference FLOP, since the resulting distilled model is not LLM.