3 ms·

It should fly on your machine

• steinvakt2 1y ago
  Yeah, was super quick and easy to set up using Ollama. I had to kill some processes first to avoid memory swap though (even with 128gb memory). So a slightly more quantized version is maybe ideal, for me at least. Edit: I'm talking about the 120B model of course