2 ms·
You can run Kimi K2 (SOTA on metrics) for about $10k/mo on full speed for many parallel sessions and no quantization on rented hardware. About $20k to buy Mac S
by adastra22 1y ago
You can run Kimi K2 (SOTA on metrics) for about $10k/mo on full speed for many parallel sessions and no quantization on rented hardware. About $20k to buy Mac Studio hardware for running locally, at a slower but usable speed. Maybe $4k or so for an Epyc box that gets you a snail's pace output, but still usable if you are very patient.
3rd party dependence is out of convenience, not necessity.
- zelphirkalt 1y agoI am not getting your point. Are you saying I should be spending 2y of savings (20k) to get rid of the 3rd party dependency, and if I don't, it is merely convenience? And are you saying I should buy another lock-in system (an Apple device) in exchange?
- adastra22 1y agoCost of doing business. A contractor will need to buy a truck. A programmer would need to rent hardware. The cost of hardware would go down over time, of course. The point is with open weight models that are SOTA, this isn’t techno feudalism as some people here seem to claim.