3 ms·
How much is it costing you to run the LLM? Is it OpenAI?
by yosito 3y ago
How much is it costing you to run the LLM? Is it OpenAI?
- rexec 3y agoMixtral on Anyscale Endpoints: https://docs.endpoints.anyscale.com/pricing https://docs.endpoints.anyscale.com/pricing
- yosito 3y agoThanks! I'm not experienced enough in this domain to have an idea of what that means for the bottom line. I assume that at the scale of a niche app, the cost is close to negligible and you can pay it out of pocket, but I imagine that if Amazon would pre-install something like this on every Kindle, the costs would go through the roof. Speaking in broad terms, how much scale do you think an app like this could handle, while still being something an individual might consider worthwhile to pay for out of pocket?