3 ms·
They have been exponentially decreasing so far [1] and the Vera Rubin generations chips will be going live that are 35X more efficient in terms of inference/ me
by ianm218 6mo ago
They have been exponentially decreasing so far [1] and the Vera Rubin generations chips will be going live that are 35X more efficient in terms of inference/ megawatt [2]. Even with rising prices 10x is possibly conservative.
Maybe if demand is truly crazy the labs will take more margin
1. epoch.ai/data-insights/llm-inference-price-trends
2. http://hashrateindex.com/blog/nvidia-vera-rubin-nvl72-specs-breakdown/ http://hashrateindex.com/blog/nvidia-vera-rubin-nvl72-specs-...