2 ms·

Well, I don't have hardware to run a 141B parameters model, even if only 39B are active during inference.

• navbaker 2y ago
  It will be quantized in a matter of days and runnable on most laptops.
  • azinman2 2y ago
    8 bit is 149G. 4 bit is 80G. I wouldn’t call this runnable on most laptops.