4 ms·
Koboldcpp (a nice frontend for llama.cpp) is The Way. You really want to run OSX though, as its not very fast without Metal (or Vulkan). Also, you need a relat
by brucethemoose2 3y ago
Koboldcpp (a nice frontend for llama.cpp) is The Way.
You really want to run OSX though, as its not very fast without Metal (or Vulkan). Also, you need a relatively high memory M1 model to run the better llama variants.
- Ms-J 3y agoI'll take a look into Koboldcpp, a frontend is always nice, thanks! I do have the max specs on this M1.