4 ms·

Simple API server for LLaVA based on llama.cpp. The llama.cpp project will add server and interactive support for LLaVA but I just couldn't wait. Runs great on

by trzy 3y ago

Simple API server for LLaVA based on llama.cpp. The llama.cpp project will add server and interactive support for LLaVA but I just couldn't wait. Runs great on my M1 Pro Macbook w/ 32gb RAM. Enjoy! :)

v3ss0n 3y ago
We already have ollama,LocalAI.
- trzy 3y ago
  Neither of which run LLaVA as far as I can tell. Use this until that happens. Also, this is way easier to set up. No dependencies. Easy.
  - v3ss0n 3y ago
    why not? both uses llamacpp as backend
    - trzy 3y ago
      llama.cpp does not have interactive mode support for LLaVA nor the web server yet. So if they're just invoking the app, there is a lot of overhead for each call until they add that in. This server doesn't have that issue.