3 ms·
It's difficult because we actually ditched a lot of the c++ code with this change and rewrote it in golang. Specifically server.cpp has been excised (which was
by Patrick_Devine 2y ago
It's difficult because we actually ditched a lot of the c++ code with this change and rewrote it in golang. Specifically server.cpp has been excised (which was deprecated by llama.cpp anyway), and the image processing routines are all written in go as well. We also bypassed clip.cpp and wrote our own routines for the image encoder/cross attention (using GGML).
The hope is to be able to get more multimodal models out soon. I'd like to see if we can get Pixtral and Qwen2.5-vl in relatively soon.
- qrios 2y ago> Specifically server.cpp has been excised (which was deprecated by llama.cpp anyway) Is there any more specific info available about who (llama.cpp or Ollama) removed what, where? As far as I can see, the server is still part of llama.cpp. And more generally: Is this the moment when Ollama and Llama part ways?
- exe34 2y agothat's cool thank you! no grounding then? I don't get the impression it's actually part of llama 3.2v but I thought it's worth checking with somebody who might have the experience!
- Patrick_Devine 2y agoI haven't looked at cogvlm, but if you mean doing bounding boxes w/ classification, I'd love to support models like that (like detectron2) in the future.
- exe34 2y agoI'm not sure what you mean by classification, but something like it, yes: "what are the coordinates of the bounding box for the rubber duck in the image [img]" >>> "[10,50,200,300]"