2 ms·
>> I suppose the question is where are they most commercially viable. Fair point, though I'm not aiming to start a competing LLM SaaS service, rather i'm evalu
by TuringNYC 3y ago
>> I suppose the question is where are they most commercially viable.
Fair point, though I'm not aiming to start a competing LLM SaaS service, rather i'm evaluating swapping out the TCO of Azure Cognitive Service OpenAI for the TCO of dedicated cloud compute running my own LLM -- to serve my own LLM calls currently being sent to a metered service (Azure Cognitive Service OpenAI)
Evaluation points would be: output quality; meter vs fixed breakeven points; latency; cost of human labor to maintain/upgrade
in most cases, i'd outsource and not think about it. BUT we're currently in some strange economics where the costs are off the charts for some services