3 ms·

Are you running the full 65GB model on a MacBook Pro? What tokens per second do you get? What specs? M5?

• iAMkenough 11mo ago
  If they're running 120B on a M5 (32GB max of memory today), I'd like to know how.
  • thaw13579 11mo ago
    Probably an M4 which has up to 128GB currently
• jonaustin 11mo ago
  On an m4 pro 128gb: 75 t/s. Caveat: That's just for the first prompt.
• spullara 11mo ago
  I am running the full model on an 128GB M3 Max.