2 ms·
Seriously? Does Netflix or Spotify cost the same everywhere around the world? They earn less and their buying power is less.
by segmondy 9mo ago
Seriously? Does Netflix or Spotify cost the same everywhere around the world? They earn less and their buying power is less.
- epolanski 9mo agoSure so do professional tools like Microsoft teams or compute in different places of the world.
- vineyardmike 9mo agoThe costs of Netflix and Spotify are licensing. Offering the subscription at half price to additional users is non-cannibalizing and a way to get more revenue from the same content. The cost of LLMs are the infrastructure. Unless someone can buy/power/run compute cheaper (Google w/ TPUs, locales with cheap electricity, etc), there won't be a meaningful difference in costs.
- storystarling 9mo agoThat assumes inference efficiency is static, which isn't really the case. Between aggressive quantization, speculative decoding, and better batching strategies, the cost per token can vary wildly on the exact same hardware. I suspect the margins right now come from architecture choices as much as raw power costs.