3 ms·
Frontier models aren't released, they are closed source. And the Chinese have been a huge source of innovation in the field.
by epolanski 10mo ago
Frontier models aren't released, they are closed source.
And the Chinese have been a huge source of innovation in the field.
- jdmoreira 10mo agodistilling does not require the models to be released. They simply use the apis. They have been a source of innovation but probably not in training them.
- NoOn3 10mo agoThey just lack of performant hardware. They have enough knowledge. And so they choose a more effective strategy without wasting resources on training from scratch.
- deleted 10mo ago[deleted]
- Tostino 10mo agoYou can do a rough distill through the APIs. You don't need the weights. It was much easier when companies had models on the /completion style APIs, because you could actually get the logits for each generation step, and use that as a dataset to fit your model to. That isn't to diminish the efforts of the Chinese developers though, they are great.