4 ms·

You misspelled llama.cpp

• kennywinker 5mo ago
  I’ve read all the stuff about how llama.cpp is much faster and better than ollama, and i believe it - but good god llama.cpp isn’t user friendly. You’d think in an era where “code is free” there would be an easier story around running local ai than compiling llama.cpp by hand and then spending hours researching flags - only for it to crash from an oom error every ten prompts or so.
  • greenavocado 5mo ago
    You're supposed to use a cheap ChatGPT subscription to run optimization loops over llama.cpp flags with a self-contained reproducible benchmark script and just let it burn for hours/days until it is fully optimized ))))
  • numberwan9 5mo ago
    Please take a look at https://llama.app https://llama.app, all you need is 'llama serve'
    • trash_cat 5mo ago
      There is more to it than typing "llama serve".