4 ms·

For 15k you could run the best open source models (that would require 200k in hardware to run) 24/7 at 50 tps for like 5 years straight. Im really not sure wh

For 15k you could run the best open source models (that would require 200k in hardware to run) 24/7 at 50 tps for like 5 years straight. Im really not sure why anyone would spend 15k to run local llms. The models you'll be able to run (70b) param models will be incredibly underwhelming.

choo-t 4mo ago
> For 15k you could run the best open source models (that would require 200k in hardware to run) 24/7 at 50 tps for like 5 years straight. Is this assuming no price increase and no throttling ?