4 ms·
The part about compute cost ruining the economics of AI companies is somewhat hilarious and eye rolling to me for the following simple reason: users have comput
by api 4y ago
The part about compute cost ruining the economics of AI companies is somewhat hilarious and eye rolling to me for the following simple reason: users have computers right in front of them.
Some models are small enough to run locally. For the rest is it conceivable to ask the customer to contribute cycles? Especially for anything cheap or free?
There might be major technical challenges here but that’s not my point. My point is that I get the distinct impression this has barely occurred to most of these people as a possibility to even explore.
Are we so far into peak SaaS now that the industry has forgotten about local compute the same way we forgot about data centers during peak PC?
- beepbooptheory 4y ago> Are we so far into peak SaaS now that the industry has forgotten about local compute the same way we forgot about data centers during peak PC? I mean, yes? You can host your own email server at home, but Google has still made billions selling email. Regardless, paying a company to allow them to run cycles of their proprietary model on my hardware doesn't feel like a great deal to me. Why would I choose that?
- api 4y agoIf it cost less? Or if you bought the model and ran it locally instead? The reason it’s hard to DIY a mail server is spam and all the attendant black lists and white lists. It’s not a resource problem.
- HPsquared 4y agoIt's not just a question of cycles; training these large models requires a lot of RAM and I/O, far beyond what's on a personal machine. Training these kinds of large models is not conducive to being parcelled-out in small chunks.
- api 4y agoTraining maybe but most of a model’s use is evaluation.