3 ms·

GGML is another neat ML abstraction layer, but I don't think much work has been dedicated to the Windows port.

• tom_0 9mo ago
  GGML still runs on llama.cpp, and that still requires CUDA to be installed, unfortunately. I saw a PR for DirectML, but I'm not really holding my breath.
  • lostmsu 9mo ago
    You don't have to install the whole CUDA. They have a redistributable.
    • tom_0 9mo ago
      Oh, I can't believe I missed that! That makes whisper.cpp and llama.cpp valid options if the user has Nvidia, thanks.
      • lostmsu 9mo ago
        Whisper.cpp and llama.cpp also work with Vulkan.
        • tom_0 9mo ago
          Yeah, I researched this and I absolutely missed this whole part. To my defense I looked into this in 2023 which is ages ago :) Looks like local models are getting much more mature.