3 ms·
The high compute costs for training and legacy software foothold seems that it may encourage rent seeking in the big tech companies. I hope the huge costs and c
by oidar 4y ago
The high compute costs for training and legacy software foothold seems that it may encourage rent seeking in the big tech companies. I hope the huge costs and compute demands won't only enrich the big tech companies that can afford to run the models on heavy duty hardware. If it does,
it will potentially lead to the rich companies, individuals, and persons greatly outpacing their peers.
- brucethemoose2 4y agoThese models can be run locally, but I see far too many businesses and bloggers resign themselves to renting OpenAI's cloud instead of tinkering with LLaMA/Alpaca, the community Stable Diffusion finetunes and such. Adobe does have a legal edge here, which is interesting and perhaps actually worth a subscription if needed.
- int_19h 4y agoThe problem is that GPT-3.5 is so cheap, and the results that you get out of it are still quite a bit better than LLaMA/Alpaca. There just doesn't seem to be any solid economic reason to run it locally except to keep inputs and/or outputs private.
- brucethemoose2 4y agoThat is not sustainable though. OpenAI is in the "burning money to gain market share" phase.
- int_19h 4y agoIs there any concrete evidence for that? Like, did anyone try to calculate how much power it actually costs to run the ChatGPT API at its present scale? My understanding is that the upfront costs on hardware are really expensive, but powering it + amortized wear and tear to replace it eventually is orders of magnitude smaller. So, what is the actual estimated cost of one ChatGPT token, and how much money is getting burned? But also, even if they double the price at some point, it'd still be cheap enough for this to apply.