4 ms·

You can get 1t/s on a raspberry pi. https://youtu.be/o1sN1lB76EA?si=i8ecEBjLdV0zewFQ https://youtu.be/o1sN1lB76EA?si=i8ecEBjLdV0zewFQ

• klohto 2y ago
  this has nothing to do with the full 671B and the ollama models are distilled qwen2.5
  • thomquaid 2y ago
    I appreciate both of these comments, thank you both.