4 ms·
this is called progress
by anthonypasq 8mo ago
this is called progress
- metaltyphoon 8mo agoOr, we can bleed out cash for a very long time.
- worldsavior 8mo agoI'm asking technically how progress works. What is actually being improved here
- anthonypasq 8mo agomostly cost of hardware going down. as models scale, nvidia produces a new hardware generation that outputs more tokens per watt, but those speed gains get eaten by the fact that the model is bigger ie. more expensive to serve. Also we have no clue whether Anthropics inference margin is compressing or not and they just want to maintain the price.