I'm running https://text-generator.io https://text-generator.io from my house, two 3090s right now powering it. It allows the service to undercut OpenAI around

I'm running https://text-generator.io https://text-generator.io from my house, two 3090s right now powering it. It allows the service to undercut OpenAI around 10x on Text/code generation and Google over 8x on speech to text. A Cloudflare tunnel is pointing to it running locally. It makes development very fast too. Its a bit tricky to keep purchasing new hardware to spin up new instances but that's getting easier with practice and autoscaling cloud providers doesn't necessarily work that well either. I think co's should seriously consider this or at least adding everyones development machines to the prod cluster during when they sleep, which is what we did to render movies when i was at Weta Digital. 1000's of developer machines are pretty valuable if put to good use.