30 ms·

Why would llama with --mmap crash?

• zozbot234 7mo ago
  This doesn't surprise me all that much, mmap support gets little attention in general and interacts poorly with GPU-side inference. (And that's with it being default, you don't even really need to specify it as a CLI option.) OP has raised a discussion with the llama.cpp folks https://github.com/ggml-org/llama.cpp/discussions/20852 https://github.com/ggml-org/llama.cpp/discussions/20852 but little interest so far
  • lostmsu 7mo ago
    But if mmap already works why would there be any interest? Besides, discussions are for users. He didn't open PRs or issues.