3 ms·

Does this run on CPU or GPU? Would it be advantageous to divide the load and run on both?

• ekianjo 3y ago
  Obviously cpu
  • endorphine 3y ago
    Not obvious to me. I thought these AI things required vast amount of GPU power. Or is that for the training part?
    • ekianjo 3y ago
      Inference works fine on CPUs while a bit slow but fine for a chatbot
  • vunderba 3y ago
    No. Ollama takes advantage of Metal on Mac M1.