32 ms·
This is the most useful documentation I've found so far to help understand how this works: https://github.com/ggml-org/llama.cpp/tree/master/tools/mtmd#multimod
by simonw 2y ago
This is the most useful documentation I've found so far to help understand how this works: https://github.com/ggml-org/llama.cpp/tree/master/tools/mtmd#multimodal-support-in-llamacpp https://github.com/ggml-org/llama.cpp/tree/master/tools/mtmd...
- scribu 2y agoIt’s interesting that they decided to move all of the architecture-specific image-to-embedding preprocessing into a separate library. Similar to how we ended up with the huggingface/tokenizers library for text-only Tranformers.