3 ms·
I read that as it runs in data centers (H100 GPUs) or high-end desktops/laptops (Strix Halo?).
by organsnyder 1y ago
I read that as it runs in data centers (H100 GPUs) or high-end desktops/laptops (Strix Halo?).
- xyc 1y agoI'm running it with ROG Flow Z13 128GB Strix Halo and getting 50 tok/s for 20B model and 12 tok/s for 120B model. I'd say it's pretty usable.
- organsnyder 1y agoExcellent! I have a Framework Desktop with 128GB on preorder—really looking forward to getting it.