3 ms·
These astronomical AI data centers will be used for high-value inference with smarter models that really are too large for running locally. The investments will
by zozbot234 6mo ago
These astronomical AI data centers will be used for high-value inference with smarter models that really are too large for running locally. The investments will be fine once they pivot to that use. Currently available open models are not in that range.
- teiferer 6mo agoI don't buy that that will be a useful distinction. First of all, no AI model will say "I'm too smart for this question, I suggest you use a cheaper one so I don't make unnecessary money for my owner" or "I'm too dumb, so instead of hallucinating I'll suggest you go to the cloud and ask my smarter sibling". Second, there is no incentive in the market for tooling to evolve that way. There will be the illusion that some models will do that, similar to today (or maybe some harnesses rather) but nobody will willinglylet money sit on the table. These data centers are not being built to solve world hunger. They are built to ultimately hook you on more realistic fake bs youtube videos so you feel good while getting even more ads injected into your life.
- jononor 6mo agoDynamic routing is the usual name for the piece that orchestrates which LLM will be used, based on query complexity. There in an open source implementation as part of the vLLM project (and probably others), it is a field of active research in several universities and labs. It is also suspected that the frontier LLM providers might already be doing something like it behind the covers.