3 ms·

It's open source, you can just download and run it for free on your own hardware.

• tinyhouse 3y ago
  Well, I don't have hardware to run a 141B parameters model, even if only 39B are active during inference.
  • navbaker 3y ago
    It will be quantized in a matter of days and runnable on most laptops.
    • azinman2 3y ago
      8 bit is 149G. 4 bit is 80G. I wouldn’t call this runnable on most laptops.
• astrodust 3y ago
  "Who among us doesn't have 8 H100 cards?"
  • MacsHeadroom 3y ago
    Four V100s will do. They're about $1k each on ebay.
    • astrodust 3y ago
      $1500 each, plus the server they go in, plus plus plus plus.
      • MacsHeadroom 3y ago
        Sure, but it's still a lot less than 8 h100s. ~$8k for an LLM server with 128GB of VRAM vs like $250k+ for 8 H100s.
        • astrodust 3y ago
          $8K sounds like a lot more than "free".