4 ms·
Higher performance, I agree, but cheaper? I didn’t check how much this costs, but if you use AI locally a lot, it’s going to be amortised pretty quickly. Burni
by illiac786 5mo ago
Higher performance, I agree, but cheaper?
I didn’t check how much this costs, but if you use AI locally a lot, it’s going to be amortised pretty quickly. Burning 100$ a month on tokens has become insanely easy. I remember when it was unimaginable for me…
- chis 4mo agoSpeculation is it’ll cost at least $6k for 128GB
- girvo 4mo agoYou top out at 20 tokens per second on hardware with memory bandwidth this low for any local model actually worth using. Doing the maths, it’s not financially worth it. Only worth it for privacy and control reasons. I do love my GB10 Asus Spark-like though still!
- illiac786 4mo agoI don’t understand your calculation, can you elaborate? At 25USD/Mtk output, assuming your 20tk/s, I generated/saved (minus power costs) ~15k$ in a year. Granted, it won’t run 24/7, but over a couple of years, this is definitely cheaper.
- chis 4mo agoThis can’t run any models that cost $25/mtok lol. I think the fastest model it’ll reasonably run will be GPT-OSS 120B which costs $.05/mtok. This is a laptop for CUDA devs and AI larpers.
- chis 4mo agoThis won’t be able to run any of the cutting edge models. And the models it can run can be served from cloud providers for very cheap - like <$1 per million tokens for the latest deepseek. It’d take many years to break even on your $6000 investment, meanwhile better and better models will come out that the DGX can’t run.