5 ms·
How to Migrate from OpenAI to Cerebrium for Cost-Predictable AI Inference
- amelius 1y agoHow to move from one service that is out of your control to another service that is out of your control.
- anonymousDan 1y agoI don't understand - what do they mean when they say you can run things on your own infrastructure then?
- amelius 1y agoThey say "serverless infrastructure", which is something else.
- dist-epoch 1y agoYour own infrastructure in the same sense as your own AWS EC2 machines.
- klabb3 1y ago> your own AWS EC2 machines Not disagreeing, but this is quite an expression.
- Incipient 1y agoHaving the ABILITY to move seamlessly and without significant cost is absolutely critical. It gives you flexibility if the provider isn't keeping pace with the market and it prevents the provider from jacking prices relative to its competitors. Vendor lockin is awful. Hypothetically, imagine how stuffed you'd be if your core virtualisation provider jacked prices 500%! You'd be really hurting. ...ohwait.
- kristianc 1y agoYou're not really locked in in any meaningful way currently, you just switch the API you're using. Rather like is being demonstrated here.
- deleted 1y ago[deleted]
- apt-apt-apt-apt 1y ago[dead]
- iamlintaoz 1y agoWhy? Honestly, there are already tons of Model-as-a-Service (MaaS) platforms out there—big names like AWS Bedrock and Azure AI Foundry, plus a bunch of startups like Groq and fireflies.ai. I’m just not seeing what makes Cerebrium stand out from the crowd.
- deleted 1y ago[deleted]
- benterix 1y agoWell, they are announcing their $8.5m seed round and hope to attract the maximum number of users by giving away $30 in credits.
- tomschwiha 1y agoThe "not optimized" self hosted deployment is 3x slower and costs 34x the price using the cheapest GPU / a weak model. I don't see the point in self hosting unless you deploy a gpu in your own datacenter where you really have control. But that costs usually more for most use cases.
- Incipient 1y agoIs there actually some scale magic that allows the 34x cost saving (over 100x when you include performance), or is it just insane investment allowing these companies to heavily subsidise cost to gain market share?
- tomschwiha 1y agoCalculating without energy costs: The A10 Gpu itself costs 3200$. With a 3 year usage that is 0,002$ per minute. From the blog post the cost per minute is charged at 0,02$, so a premium of 10x. So with energy if you can load the GPU at minimum 15-20% self hosted becomes cheaper. But you need to take care of your own infrastructure. With larger purchases the GPU prices also drop so that is the scaling logic.
- ToucanLoucan 1y ago> I don't see the point in self hosting unless you deploy a gpu in your own datacenter where you really have control. But that costs usually more for most use cases. Not wanting to send tons of private data to a company who's foundation is exploiting data it didn't have permission to use?
- dabedee 1y agoThis isn't really about cost savings, it's about control. Self-hosting makes sense when you need data privacy, custom fine-tuning, specialized models, or predictable costs at scale. For most use cases requiring GPT-4o-mini quality, you'll pay more for self-hosting until you reach significant volume.
- ivape 1y agoI’m trying to figure out the cost predictability angle here. It seems like they still have a cost per input/output tokens, so how is it any different? Also, do I have to assume one gpu instance will scale automatically as traffic goes up? LLM pricing is pretty intense if you’re using anything beyond a 8b model, at least that’s what I’m noticing on OpenRouter. 3-4 calls can approach eating up a $1 with bigger models, and certainly on frontier ones.
- jameswhitford 1y agoServerless setups (like Cerebrium) charge per second the model is running, its not token based.
- ivape 1y agoAh you’re right, misread the OpenAI/cerbrium pricing config variables.
- BoorishBears 1y agoYou're still paying more than the GPU typically costs on an hourly basis to take advantage of their per-second billing... and if you don't have enough utilization to saturate an hourly rental then your users are going to be constantly running into cold starts which tend to be brutal for larger models. Their A100 80GB is going more than what I pay to rent H100s: if you really want to save money, getting the cheapest hourly rentals possible is the only way you have any hope of saving money vs major providers. I think people vastly underestimate how much companies like OpenAI can do with inference efficiency between large nodes, large batch sizes, and hyper optimized inference stacks.
- ivape 1y agoI'll echo one of my original concerns, which is how is this supposed to scale? Am I responsible for that?
- BoorishBears 1y ago
- benterix 1y agoTo people from Cerebrium: why should I use your services when Runpod is cheaper? I mean, why did you decide to set your prices higher than an established company with significant user base?
- Sanzig 1y ago(Not affiliated with Cerebrium, just did a bit of looking into this a little while back). Runpod outsources much of their infrastructure to small players that own GPUs. They have recently added some requirements on security and reliability (eg: some level of security audit such as SOC 2, has to be hosted in a real DC, has to be in a locked rack), but fundamentally they are leaning on small shops that slap some GPUs in a server at a colocation facility. This personally would make me nervous about any sensitive workloads. My impression is that Cerebrium either owns their own GPU servers or they're outsourcing to one of the big players. They certainly don't have the "partner program" advertised on their site like Runpod does.
- za_mike157 1y agoHey! Founder of Cerebrium here. - Runpod is one of the cheapest but it comes at the price of reliability (critical for businesses) - We have more performant cold start performance with something special launching soon here - Iterating on your application using CPUs/GPUs in the cloud takes just 2–10 seconds, compared to several minutes with Runpod due to Docker push/pull. - Allow you to deploy in multiple regions globally for lower latency and data residency compliance - We provide a lot of software abstractions (fire and forget jobs, websockets, batching, etc) where as Runpod just deploys your docker image. - SOC 2 and GDPR compliant With that all being said - we are working on optimisations to bring down pricing
- benterix 1y agoThanks, makes sense.
- Incipient 1y agoIs this article just saying openai is orders of magnitude cheaper than cerebrium?
- jameswhitford 1y agoIt's a demo project using the free tier hardware from Cerebrum, demonstrating how to migrate with a few lines of code from OpenAI. The cost is never going to beat OpenAI on an A10, there are more powerful options available.
- gordianlabs 1y agoDo you forecast costs or just provide more visibility?
- deleted 1y ago[deleted]