2 ms·

I ran full version of this model without any swapping on cluster of 2x $3000 laptops (strix halo zbook 128GB) at about 20 tokens per second. I would say it is

I ran full version of this model without any swapping on cluster of 2x $3000 laptops (strix halo zbook 128GB) at about 20 tokens per second. I would say it is in reach for normal person. If anything buying it was great investment, it is work tool, I will probably sell it for more than what I bought it for :)

prmoustache 7mo ago
> I would say it is in reach for normal person. That is a very wealthy country centric thing to say.
- mycall 7mo ago
  ..and to think $2=3k is basically a minimum to really use local LLMs effectively, with prices definitely rising globally. Most people will just stick with online services for LLMs.