3 ms·

why not just as you suggested i.e. https://qwen.readthedocs.io/en/latest/run_locally/llama.cpp.html#llama-server https://qwen.readthedocs.io/en/latest/run_local

• mdp2021 6d ago
  I am not sure that llama.cpp also supports image generation models.
  • utopiah 6d ago
    it's multimodal, see https://github.com/ggml-org/llama.cpp/blob/master/docs/multimodal.md https://github.com/ggml-org/llama.cpp/blob/master/docs/multi...
    • exe34 6d ago
      Multimodal doesn't guarantee input and output. > Currently, we support image, audio and video input.
      • utopiah 6d ago
        Seems I'm missing something. Does this model support other inputs? Image outputs are supported, videos I'm not sure but I don't think that's an output, just a preview of the equirectangular example, so, same question here, what does this model outputs that isn't supported?
        • deleted 6d ago
          [deleted]
        • exe34 6d ago
          It's a diffusion model, completely different from autoregressive attention models.
          • utopiah 6d ago
            like https://github.com/ggml-org/llama.cpp/tree/master/examples/diffusion https://github.com/ggml-org/llama.cpp/tree/master/examples/d... ? At the risk of stating the obvious llama.cpp isn't just about LLaMa as https://github.com/ggml-org/llama.cpp/blob/master/src/llama-arch.h#L13 https://github.com/ggml-org/llama.cpp/blob/master/src/llama-... someone else pointed out.
            • exe34 6d ago
              Aha I was wrong. Thanks for sharing that!
              • utopiah 6d ago
                no worries, I was wrong too, it is multimodal but only for inputs apparently, so for now there seem to only be text as output but no image as output
        • mdp2021 6d ago
          Not all architectures are supported by llama.cpp . The GGUF format encodes the NN in a standardized way, but then you need code that can use that NN structure. I understand that llama.cpp could only output text, last time I checked (I do not know how to find a good source for that though). See https://github.com/ggml-org/llama.cpp/blob/master/src/llama-arch.h https://github.com/ggml-org/llama.cpp/blob/master/src/llama-... , the enum llm_arch { ...
          • utopiah 6d ago
            I haven't used llama.cpp for image generation either but I recall an issue about it. Unfortunately I can't pinpoint it now and there is the older closed issue https://github.com/ggml-org/llama.cpp/issues/4408 https://github.com/ggml-org/llama.cpp/issues/4408 so unless mtmd supports also multimodal outputs out of the box safe to assume output is still limited to text.