3 ms·

BLOOM and OPT are both GPT-3 sized models that are open source and similarily sized (175B parameters) but a) text generation is not as good as GPT-3 for whateve

BLOOM and OPT are both GPT-3 sized models that are open source and similarily sized (175B parameters) but a) text generation is not as good as GPT-3 for whatever reason and b) getting them to run in a cost-effective manner is not easy and with this new price decrease, it may be more economical to use GPT-3 for deployment despite that.

edgyquant 4y ago
Is this due to entirely to economies of scale?
- tehsauce 4y ago
  Economies of scale and years of optimization, some of which likely have not been publicly shared.
- minimaxir 4y ago
  Not entirely. There are many ways to optimize models for deployment both at the software and hardware levels: some are public such as quantization/fusing, others are not.
tehsauce 4y ago
The reason the similarly sized open source models aren’t as good is because gpt-3 has had a 2 year head start of additional development and improvement. The gpt-3 that is available today is Much better than what was initially produced in 2020.