4 ms·

0.35 words/s on my 11th gen i5 with 7B model (framework laptop) not so bad !

• fguerraz 4y ago
  How long did you have to wait for it to load? On my machine it's been running for 15mins, I'm still waiting for a prompt...
  • kristianp 4y ago
    How much RAM do you both have?
    • popol12 4y ago
      32gb of ram and 64gb on NVME swap
  • popol12 4y ago
    You get the full answer after completion, so it’s normal if you don’t get an output immediately I computed the speed by doing speed=number of words/ total run time