2 ms·
People have decent oss solutions for running inference. There are currently no high quality oss code for training and the best checkpoints are still unavailable
by ivalm 4y ago
People have decent oss solutions for running inference. There are currently no high quality oss code for training and the best checkpoints are still unavailable (gpt3.5+4/palm/claude)…
There are only a few hundred folks who have actual experience training at this scale.
llama is both not oss and not as good as da-vinci-003/etc. Prob best actual oss model is like flan-t5-xxl.
- recuter 4y agoSo we can't lay eggs yet, at least we got one. Who knows what might hatch.
- ivalm 4y agoI guess I just felt an objection about “[oss] running circles around OpenAI.” The reality is that the core technology is still very unapproachable (making these models). OSS solved like 1% of the total problem/difficulty. My biggest hope is that meta/google/anthropic actually decide to release training code + data + checkpoint. Meta is most likely to do it but still unlikely Without that it would take eutherai/carperai to really exceed expectations for OSS to be competitive.