3 ms·
The other difference is that reliability for the gemini api is garbage, whereas for vertex ai it is fantastic.
by fooster 1y ago
The other difference is that reliability for the gemini api is garbage, whereas for vertex ai it is fantastic.
- nikcub 1y agoThe key to running LLM services in prod is setting up Gemini in Vertex, Anthropic models on AWS Bedrock and OpenAI models on Azure. It's a completely different world in terms of uptime, latency and output performance.
- shpat 1y agoHave you had any luck getting your Claude quota bumped on Bedrock? I tried working through AWS support but got nowhere. Gave up and used Vertex + Gemini
- com2kid 1y agoDoes OpenAI on azure still have that insane latency for content filtering? Last time I checked it added a huge # to time to first token, making azure hosting for real time scenarios impractical.
- shakna 1y agoYes. Unless you convince MS to let you at the "Provisioned Throughput" model. Which also requires being big enough for sales to listen to you.