I encourage you to fork this repo and build it. Might be worth checking out Ollama and bakllava. https://ollama.ai/library/bakllava https://ollama.ai/library/b

I encourage you to fork this repo and build it. Might be worth checking out Ollama and bakllava. https://ollama.ai/library/bakllava https://ollama.ai/library/bakllava Maybe the model is a bit too slow, but I'm sure smaller ones will come out soon. You can likely fine tune to do exactly what you need.