4 ms·
> If you're inferencing, you're not getting much more out of this than you would a ~$2K Framework desktop. I was with you up till here. Come on! CPU inferenci
by NitpickLawyer 1y ago
> If you're inferencing, you're not getting much more out of this than you would a ~$2K Framework desktop.
I was with you up till here. Come on! CPU inferencing is not it, even macs struggle with bigger models, longer contexts (esp. visible when agentic stuff gets > 32k tokens).
The PRO6000 is the first gpu that actually makes sense to own from their "workstation" series.
- refulgentis 1y agoEr, CPU inferencing? :) I didn't think I mentioned that! The Framework Desktop thing is that has unified memory with the GPU, so much like an M-series, you can inference disproportionately large models.