3 ms·

Right now the closed models are incredibly higher quality than the open models. They're useful as a stopgap for 1-2 years in hopes/expectation of open models re

Right now the closed models are incredibly higher quality than the open models. They're useful as a stopgap for 1-2 years in hopes/expectation of open models reaching a point where they can be swapped in. It burns cash now, but in exchange you can grab more market share sooner while you're stuck using the expensive but high quality OpenAI models. It's not cost-effective, but it may be part of a valid business plan.

ronsor 3y ago
If you're finetuning your own model, the closed models being "incredibly higher quality" is probably less relevant.
- claytonjy 3y ago
  That's how we all want it to work, but the reality today is that GPT-4 is better at almost anything than a fine-tuned version of any other model. It's somewhat rare to have a task and good enough dataset that you can finetune something else to be close enough in quality to GPT-4 for your task.
  - zirgs 3y ago
    GPT-4 is still heavily censored and will simply refuse to talk about many "problematic" things. How is that better than a completely uncensored model?
    - Veen 3y ago
      Depends what you’re using it for. For many use cases, the censorship is irrelevant.
- wongarsu 3y ago
  Finetuning a better model still yields better results than finetuning a worse model.
fnordpiglet 3y ago
That should be a wake up call to every corporation pinning their business on OAI models. My experience thus far is no one is seeing a need to plan an exit from OAI, and the perception is “AI is magic and we aren’t magicians.” There needs to be a concerted effort to finance and tune high quality freely available models and tool chains asap. That said I think efficiencies will dramatically improve over the next few years and over investing now probably captures very little value beyond building internal competency - which doesn’t grow with anything but time and practice. The longer you depend on OAI, the longer you will depend on OAI past your point of profound regret.
- reaperman 3y ago
  > There needs to be a concerted effort to finance and tune high quality freely available models and tool chains asap. Absolutely. A large consortium of companies could each contribute 0.2-2% of the total cost and fund something much larger than OpenAI.

ronsor 3y ago

If you're finetuning your own model, the closed models being "incredibly higher quality" is probably less relevant.

claytonjy 3y ago
That's how we all want it to work, but the reality today is that GPT-4 is better at almost anything than a fine-tuned version of any other model. It's somewhat rare to have a task and good enough dataset that you can finetune something else to be close enough in quality to GPT-4 for your task.
- zirgs 3y ago
  GPT-4 is still heavily censored and will simply refuse to talk about many "problematic" things. How is that better than a completely uncensored model?
  - Veen 3y ago
    Depends what you’re using it for. For many use cases, the censorship is irrelevant.
wongarsu 3y ago
Finetuning a better model still yields better results than finetuning a worse model.

fnordpiglet 3y ago

That should be a wake up call to every corporation pinning their business on OAI models. My experience thus far is no one is seeing a need to plan an exit from OAI, and the perception is “AI is magic and we aren’t magicians.” There needs to be a concerted effort to finance and tune high quality freely available models and tool chains asap. That said I think efficiencies will dramatically improve over the next few years and over investing now probably captures very little value beyond building internal competency - which doesn’t grow with anything but time and practice. The longer you depend on OAI, the longer you will depend on OAI past your point of profound regret.

reaperman 3y ago
> There needs to be a concerted effort to finance and tune high quality freely available models and tool chains asap. Absolutely. A large consortium of companies could each contribute 0.2-2% of the total cost and fund something much larger than OpenAI.