3 ms·
They say no llama.cpp support yet, so no ollama yet (which uses llama.cpp)
by scosman 2y ago
They say no llama.cpp support yet, so no ollama yet (which uses llama.cpp)
- deleted 2y ago[deleted]
- sadeshmukh 2y agoOllama is supported: https://docs.continue.dev/setup/select-provider https://docs.continue.dev/setup/select-provider
- trsohmers 2y agoThey meant that there is no support for Codestral Mamba for llama.cpp yet.
- deleted 2y ago[deleted]
- HanClinto 2y agoCorrect. The only back-end that Ollama uses is llama.cpp, and llama.cpp does not yet have Mamba2 support. The issues to track Mamba2 and Codestral Mamba support are here: https://github.com/ggerganov/llama.cpp/issues/8519 https://github.com/ggerganov/llama.cpp/issues/8519 https://github.com/ggerganov/llama.cpp/issues/7727 https://github.com/ggerganov/llama.cpp/issues/7727 Mamba support was added in March of this year: https://github.com/ggerganov/llama.cpp/pull/5328 https://github.com/ggerganov/llama.cpp/pull/5328 I have not yet seen a PR to address Mamba2.