3 ms·
Even if companies decided to move away from expensive models from the major labs, it probably much more economical to pay a cloud provider to host some open wei
by ssivark 4mo ago
Even if companies decided to move away from expensive models from the major labs, it probably much more economical to pay a cloud provider to host some open weights model which could then be amortized across all (internal) users and do inference at a substantial batch size, rather than giving everyone their own hardware -- which means the company would need to provision for peak usage and inference at batch size of one.