4 ms·
Why would it reduce ten fold? I think performance of the chips gets a little better but not that much better. The cost of the infra is probably going to go up i
by ehnto 6mo ago
Why would it reduce ten fold? I think performance of the chips gets a little better but not that much better. The cost of the infra is probably going to go up if energy costs keep going up and presuming the US can keep getting cheap chips.
- ianm218 6mo agoThey have been exponentially decreasing so far [1] and the Vera Rubin generations chips will be going live that are 35X more efficient in terms of inference/ megawatt [2]. Even with rising prices 10x is possibly conservative. Maybe if demand is truly crazy the labs will take more margin 1. epoch.ai/data-insights/llm-inference-price-trends 2. http://hashrateindex.com/blog/nvidia-vera-rubin-nvl72-specs-breakdown/ http://hashrateindex.com/blog/nvidia-vera-rubin-nvl72-specs-...