2 ms·
What costs are 1/100th?
by boesboes 3mo ago
What costs are 1/100th?
- fooker 3mo agoOf serving a (approximately) gpt4 sized model.
- alangibson 3mo agoSo the number is irrelevant. No one wants yesterdays newspaper. The only relevant number is the price to serve a frontier or near-frontier model.
- barumrho 3mo agoIs this true? Hardware costs have only gone up during this time. Are you referring to electricity cost to serve these models? (i.e. compute got more efficient?)
- usrusr 3mo agoWhat made the cost go down? Can't be cheaper used H100, can't be cheaper RAM. A revolutionary breakthrough in hardware use per query?
- underlipton 3mo agoDoes that include the capital costs of spinning up to the current models/scale or is it just running costs? Also, lost revenue from other services being degraded by shifting resources to supporting training/serving models (Google Search...)?