4 ms·
You can’t build a frontier model with one single thing. This is an incremental improvement but it doesn’t explain the entire success of the model. The training
by Aurornis 3mo ago
You can’t build a frontier model with one single thing. This is an incremental improvement but it doesn’t explain the entire success of the model. The training set is immensely important, regardless of how you feel about distillation.
- EGreg 3mo agoReminds me of this btw: https://www.bbc.com/news/technology-12343597 https://www.bbc.com/news/technology-12343597 Microsoft replied that Bing uses “many different signals” —- including cribbing from Google :-)
- krlx 3mo agoI remember 15 years ago or so, one of my first student job was to evaluate Bing results compared to the same query on Google. Didn't know then that I was a distillation attacker.