4 ms·
LLAVA is pretty great
by nikolayasdf123 2y ago
LLAVA is pretty great
- simonw 2y agoDo you know of any good paid API providers offering LLAVA? I want to experiment with it out a bunch more without having to host it locally myself.
- nikolayasdf123 2y agonope. I am self-hosting. support is pretty good actually. llama.cpp supports it (v1.6 too; and in openai API server as well). ollama supports it. open-web-ui chat too. using it now on desktop (I am in China, so no OpenAI here) and in cloud cluster on project.
- xyc 2y agoCloudflare has it https://developers.cloudflare.com/workers-ai/models/llava-1.5-7b-hf/ https://developers.cloudflare.com/workers-ai/models/llava-1.... Locally it's actually quite easy to setup. I've made an app https://recurse.chat/ https://recurse.chat/ which supports Llava 1.6. It takes a zero-config approach so you can just start chatting and the app downloads the model for you.
- xyc 2y agoJust realized I read your blog about Llava llamafile which got me interested in local AI and made the app :) What's your reservation about running it locally?
- bfirsh 2y agohttps://replicate.com/yorickvp/llava-13b https://replicate.com/yorickvp/llava-13b :)
- nuz 2y agoWhat are you using it for? Curious if there's any interesting purposes I haven't thought of
- josefresco 2y agoIt is! Just downloaded it the other day and while far from perfect it's pretty neat. I uploaded a Gene Wilder/Charlie in the Chocolate Factory meme and it incorrectly told me that it was Johnny Depp. Close I guess! I run LLAVA and llama (among other models) using https://ollama.com https://ollama.com As a "web builder" I do think these tools will be very useful for accessibility (eventually), specifically generating descriptive alt tags for images.