3 ms·

LLaVA C++ server (based on llama.cpp)

• trzy 3y ago
  Simple API server for LLaVA based on llama.cpp. The llama.cpp project will add server and interactive support for LLaVA but I just couldn't wait. Runs great on my M1 Pro Macbook w/ 32gb RAM. Enjoy! :)
  • v3ss0n 3y ago
    We already have ollama,LocalAI.
    • trzy 3y ago
      Neither of which run LLaVA as far as I can tell. Use this until that happens. Also, this is way easier to set up. No dependencies. Easy.
      • v3ss0n 3y ago
        why not? both uses llamacpp as backend
        • trzy 3y ago
          llama.cpp does not have interactive mode support for LLaVA nor the web server yet. So if they're just invoking the app, there is a lot of overhead for each call until they add that in. This server doesn't have that issue.
• juliangoldsmith 3y ago
  Glad to see you're still working on something interesting. I saw the game console you developed many years ago. That paper demystified HDLs and the hardware side of the Z80 for me.