This is not really an answer, but I really like the "AI horde" API setup. https://aihorde.net/ https://aihorde.net/ Namely, you have a pool of gen AI voluntee

This is not really an answer, but I really like the "AI horde" API setup. https://aihorde.net/ https://aihorde.net/ Namely, you have a pool of gen AI volunteer workers constantly chewing through API requests. Host requests are prioritized, but the "downtime" of your AI worker instance is spent fulfilling other requests, which earns karma that can be used to prioritize requests from other hosts should you need more throughput. Economically, this is way more efficient than self hosting or "renting" an instance that will be idle between requests, and you aren't paying a massive tax for a proprietary API.