4 ms·
All that to end with “no meaningful improvement over the salesforce codegen model” is a bit disappointing. Negative results are interesting in their own right.
by moconnor 4y ago
All that to end with “no meaningful improvement over the salesforce codegen model” is a bit disappointing.
Negative results are interesting in their own right. I’d rather read about why this isn’t better at the 6B parameter level than e see a hand wave that, well, the samples are more diverse and look the 350M model is better.
- youssefabdelm 4y agoYeah I felt the same way. Although perhaps at a higher scale the fine-tuning can make a bigger difference? The results go against this hypothesis but at least OpenAI states that GPT-3 only needs 200 examples, so who knows. In fact I wonder how well GPT-3 would do against this when fine-tuned on just 200 examples.