4 ms·
Is there a non-obvious reason that models keep getting compared to GPT-3.5 instead of 4?
by jstummbillig 3y ago
Is there a non-obvious reason that models keep getting compared to GPT-3.5 instead of 4?
- seydor 3y agonobody can run gpt4 on their machine
- jstummbillig 3y agoI am not sure what that means. How can they run GPT 3.5 any more or less?
- deleted 3y ago[deleted]
- simonvc 3y agoGPT-3.5 (unfinetuned) has been matched by many OW (open weights) models now, and with fine tuning to a specific task (coding, customer care, health advice etc) can exceed it. It's still useful as a well known model to compare with, since it's the model the most people have experience with.
- stsquad 3y agoI think they mean the assumed parameter size of GPT-4 is so large you couldn't run it on commodity hardware even if you could get hold of the model.
- karmasimida 3y agoThe rumored GPT4 model size is 1.8 trillion parameters
- baq 3y agonon-obvious? don't think so. the obvious is GPT-4 blows them all out of the water in quality and is completely trounced on quality/inference cost.
- deleted 3y ago[deleted]
- opdahl 3y agoMaybe because GPT-3.5 is a free model so it's basically comparing free models with each other.
- skerit 3y agoMost people I know that actually use ChatGPT, use the free (GPT3.5) version. Not a lot of people are willing to pay the extra 20 euros per month.
- deleted 3y ago[deleted]
- viraptor 3y agoYet, if you want to go cheaper, you totally can by paying for the API access. Gpt4 is accessible there and you get to use your own app. $20 will last you way longer then a month if you're not a heavy user.
- skerit 3y agoTrue. Though I myself still use ChatGPT a lot more as I can quickly reach the $20 threshold via the API
- rrrrrrrrrrrryan 3y agoIt really depends on usage. If you need to have long conversations, or are sending huge texts, the all-you-can-eat for $20 plan will almost certainly be cheaper than API access. If you're doing lots of smaller one shot stuff without a lot of back and forth, the API will be cheaper.
- criddell 3y agoDo you think it's getting close to the point where it adds a euro of value per workday (on average) for them?
- skerit 3y ago
- deleted 3y ago[deleted]
- dcastm 3y agoGPT-3.5 is probably the most popular model used in applications (due to price point vs. GPT-4, and quality of results vs. open-weight models). So I guess they're trying to say now it's a no-brainer to switch to open-weight models.
- tyfon 3y agoI second this. For some type of applications, the 4 model can quickly ramp up costs, especially with large contexts and 3.5 often does the job just fine. So for many applications it's the real competitor.
- anonylizard 3y agoToo bad GPT3.5 Turbo is dirt cheap. Open source models are substantially more expensive when you factor in operating costs. There is no mature ecosystem where you can just plug in a model and spin up a robust infrastructure to run a local LLM at scale, aka you need infrastructure/ML engineers to do it, aka extremely expensive unless you are using LLMs at extremely large scales.
- dcastm 3y agoI think we'll start seeing a lot more services like https://www.together.ai https://www.together.ai soon. Having open-weight models better than gpt-3.5 will drive a lot of competition on the LLM infra.
- happycube 3y agoMistral's endpoint for mistral-small is slightly cheaper.
- Der_Einzige 3y agoThe additional control/features (support for grammars, constraints, fine-tuning, etc) far o/w the cost savings.
- anonyfox 3y agonot only price but also speed and API limits. I always ask myself the following pseudo-question: "for this geneneration/classification task, do I need to be more intelligent than an average highschool student?" Almost always in business tasks, the answer is a no. Therefore I go with GPT3.5. Its much quicker and good enough to accomplish the task usually. And then I need to run this task thousands of times, so the API limits are the most limiting factor, which are much higher in GPT3.5 variants, whereas when using GPT4 I have to be more careful with limiting/queueing requests. I patiently wait for a efficient enough model that only needs to be on a GPT3.5 level I can self-host alongside my applications with reasonably low server requirements. No need for GPT-5 for now, for business automations the lower end of "intelligence" is more than enough, but efficiency/scaling is the real deal.
- 0x008 3y agoGPT3.5 is the most popular MoE model probably.
- orbital-decay 3y agoBecause GPT-4 is multimodal, which puts it into the entirely different class. GPT-3.5 was trained on text, same as this model.