3 ms·

Any GPT-3-level fully-opensource models yet?

• minimaxir 4y ago
  BLOOM and OPT are both GPT-3 sized models that are open source and similarily sized (175B parameters) but a) text generation is not as good as GPT-3 for whatever reason and b) getting them to run in a cost-effective manner is not easy and with this new price decrease, it may be more economical to use GPT-3 for deployment despite that.
  • edgyquant 4y ago
    Is this due to entirely to economies of scale?
    • tehsauce 4y ago
      Economies of scale and years of optimization, some of which likely have not been publicly shared.
    • minimaxir 4y ago
      Not entirely. There are many ways to optimize models for deployment both at the software and hardware levels: some are public such as quantization/fusing, others are not.
  • tehsauce 4y ago
    The reason the similarly sized open source models aren’t as good is because gpt-3 has had a 2 year head start of additional development and improvement. The gpt-3 that is available today is Much better than what was initially produced in 2020.