3 ms·
MS and Google keep releasing small models like Phi and Gemma, I don't think they want to hide the fact these can be very capable.
by imjonse 2y ago
MS and Google keep releasing small models like Phi and Gemma, I don't think they want to hide the fact these can be very capable.
- bookaway 2y agoThe Gemma of Clement Farabet the French-American student of Meta's French-American LeCunn? I wouldn't be surprised if they share an ideological affinity. The big players allow some breathing space for the top brass so they don't walk. Look at the timing of Andrej Karpathy departure from OpenAI. I'm sure he might give a lot of different public answers, but it just so happened that it coincided when the oxygen started getting a little stale around there. [0] This is the forest for the trees situation. The correct analogy is Chrome and ad blockers. Google didn't tighten the screws until the bean counters started saying it was starting to bite. [0] https://xcancel.com/karpathy/status/1812983013481062761 https://xcancel.com/karpathy/status/1812983013481062761
- gmerc 2y agoThey need to. You can’t do experimental science with bigger models, training runs are getting too expensive. Everyone has a small model to do science now and most open source it because realistically otherwise everyone will just use the leader (gpt4 mini)