If you are referencing Nvidia, note that Vertex, Groq, Cerebras, etc. all do not rely on GPUs for much or all of their inference and are all the better for it.

If you are referencing Nvidia, note that Vertex, Groq, Cerebras, etc. all do not rely on GPUs for much or all of their inference and are all the better for it. Kind of why Nvidia see themselves forced to make such deals, to lock labs in before they loose another to objectively superior inference. Staying purely with US labs, they just lost Meta and both Anthropic and Gemini models have been available on Vertex without relying on GPUs for a while. OpenAI equally is turning towards Cerebras so yeah, those deals will mean little for inference. Those circular deals aren’t a thing for the inference providers I tend to go for and they don’t change the fact that using a full GPU purely for inference is like using a pickup truck for a track day. It can be done and the flexibility has advantages, but once you focus on one task, there are better tools for the job.