3 ms·
While running locally will no doubt get cheaper over time (and hence become much more viable), cloud compute cost will also drop significantly as better hardwar
by celeritascelery 1y ago
While running locally will no doubt get cheaper over time (and hence become much more viable), cloud compute cost will also drop significantly as better hardware and more specialized models are created. We have seen this process already where the cost per million tokens has been falling rapidly.