3 ms·

It took me a while but I think the difference between Vertex and Gemini APIs is that Vertex is meant for existing GCP users and Gemini API for everyone else. If

It took me a while but I think the difference between Vertex and Gemini APIs is that Vertex is meant for existing GCP users and Gemini API for everyone else. If you are already using GCP then Vertex API works like everything else there. If you are not, then Gemini API is much easier. But they really should spell it out, currently it's really confusing. Also they should make it clearer which SDKs, documents, pricing, SLAs etc apply to each. I still get confused when I google up some detail and end up reading the wrong document.

fooster 1y ago
The other difference is that reliability for the gemini api is garbage, whereas for vertex ai it is fantastic.
- nikcub 1y ago
  The key to running LLM services in prod is setting up Gemini in Vertex, Anthropic models on AWS Bedrock and OpenAI models on Azure. It's a completely different world in terms of uptime, latency and output performance.
  - shpat 1y ago
    Have you had any luck getting your Claude quota bumped on Bedrock? I tried working through AWS support but got nowhere. Gave up and used Vertex + Gemini
  - com2kid 1y ago
    Does OpenAI on azure still have that insane latency for content filtering? Last time I checked it added a huge # to time to first token, making azure hosting for real time scenarios impractical.
    - shakna 1y ago
      Yes. Unless you convince MS to let you at the "Provisioned Throughput" model. Which also requires being big enough for sales to listen to you.
nprateem 1y ago
Which would all be fine except some models like Imagen 4 only work on vertex.
happyopossum 1y ago
> I think the difference between Vertex and Gemini APIs is that Vertex is meant for existing GCP users and Gemini API for everyone else Nahh, not really - Vertex has a HUGE feature surface, and can run a ton of models and frameworks. Gemini happens to be one of them, but you could also run non-google LLMs, non LLM stuff, run notebooks against your dataset, manage data flow and storage, and and and… Gemini is “just” an LLM.
throwaway1550 1y ago
Ex-googler here. Google shipped their org hierarchy here. Vertex API is managed by Vertex team in Google Cloud. This is a production ready infrastructure that is SRE managed but usually one or two steps from the bleeding edge. Gemini API, Jules etc are built by Google Labs. This is close to the bleeding edge but not as production ready.

fooster 1y ago

The other difference is that reliability for the gemini api is garbage, whereas for vertex ai it is fantastic.

nikcub 1y ago
The key to running LLM services in prod is setting up Gemini in Vertex, Anthropic models on AWS Bedrock and OpenAI models on Azure. It's a completely different world in terms of uptime, latency and output performance.
- shpat 1y ago
  Have you had any luck getting your Claude quota bumped on Bedrock? I tried working through AWS support but got nowhere. Gave up and used Vertex + Gemini
- com2kid 1y ago
  Does OpenAI on azure still have that insane latency for content filtering? Last time I checked it added a huge # to time to first token, making azure hosting for real time scenarios impractical.
  - shakna 1y ago
    Yes. Unless you convince MS to let you at the "Provisioned Throughput" model. Which also requires being big enough for sales to listen to you.

nprateem 1y ago

Which would all be fine except some models like Imagen 4 only work on vertex.

happyopossum 1y ago

> I think the difference between Vertex and Gemini APIs is that Vertex is meant for existing GCP users and Gemini API for everyone else Nahh, not really - Vertex has a HUGE feature surface, and can run a ton of models and frameworks. Gemini happens to be one of them, but you could also run non-google LLMs, non LLM stuff, run notebooks against your dataset, manage data flow and storage, and and and… Gemini is “just” an LLM.

throwaway1550 1y ago

Ex-googler here. Google shipped their org hierarchy here. Vertex API is managed by Vertex team in Google Cloud. This is a production ready infrastructure that is SRE managed but usually one or two steps from the bleeding edge. Gemini API, Jules etc are built by Google Labs. This is close to the bleeding edge but not as production ready.