4 ms·
Side question: what are the current top goto open models for image captioning and building image embeddings dbs, with somewhat reasonable hardware requirements?
by the_duke 11mo ago
Side question: what are the current top goto open models for image captioning and building image embeddings dbs, with somewhat reasonable hardware requirements?
- NitpickLawyer 11mo agoTry any of the qwen3-vl models. They have 8, 4 and 2B models in this family.
- Glemkloksdjf 11mo agoI would suggest YOLO. Depending on your domain, you might also finetune these models. Its relativly easy as they are not big LLMs but either image classification or bounding boxes. I would recommend bounding boxes.
- smallerize 11mo agoWhich YOLO?
- Glemkloksdjf 11mo agoAny current one. they are easy to use and you can just benchmark them yourself. I'm using small and medum. Also the code for using it is very short and easy to use. You can also use ChatGPT to generate small exepriments to see what fits your case better
- throwaway314155 11mo agoThere aren’t any YOLO models for captioning and the other models aren’t robust enough to make for good embedding models.
- Glemkloksdjf 11mo agoYou can get labels out of the classifier and bounding box models. They are super fast. Its just an alternative i'm mentioning. I would assume a person knowing a little bit of that domain. Otherwise the first option would be CLIP i assume. llm-vl is just super slow and compute intensive.
- jabron 11mo agoWhat do you mean "bounding boxes"? They were talking about captions and embeddings, so a vision language model is required.
- Glemkloksdjf 11mo agoI suggested YOLO and non llm-vl as a lot faster alternative. Of course CLIP would be otherwise the other option than a big llm-vl one.
- daemonologist 11mo agoFor pure image embedding, I find DINOv3 to be quite good. For multimodal embedding, maybe RzenEmbed. For captioning I would use a regular multimodal LLM, Qwen 3 or Gemma 3 or something, if your compute budget allows.