2 ms·
The problem is that GPT-3.5 is so cheap, and the results that you get out of it are still quite a bit better than LLaMA/Alpaca. There just doesn't seem to be an
by int_19h 4y ago
The problem is that GPT-3.5 is so cheap, and the results that you get out of it are still quite a bit better than LLaMA/Alpaca. There just doesn't seem to be any solid economic reason to run it locally except to keep inputs and/or outputs private.
- brucethemoose2 4y agoThat is not sustainable though. OpenAI is in the "burning money to gain market share" phase.
- int_19h 4y agoIs there any concrete evidence for that? Like, did anyone try to calculate how much power it actually costs to run the ChatGPT API at its present scale? My understanding is that the upfront costs on hardware are really expensive, but powering it + amortized wear and tear to replace it eventually is orders of magnitude smaller. So, what is the actual estimated cost of one ChatGPT token, and how much money is getting burned? But also, even if they double the price at some point, it'd still be cheap enough for this to apply.