4 ms·
You’re a mad man - thank you! Do I understand correctly that Ollama doesnt do that, and that’s why responses hang forever on a M3 running the same model throug
by cyanregiment 3mo ago
You’re a mad man - thank you!
Do I understand correctly that Ollama doesnt do that, and that’s why responses hang forever on a M3 running the same model through Ollama?
- trollbridge 3mo agoPlease don’t use Ollama. https://sleepingrobots.com/dreams/stop-using-ollama/ https://sleepingrobots.com/dreams/stop-using-ollama/
- lemonlimesoda 3mo ago[dead]
- reddguard 3mo agoDoesn't Ollama use llama.cpp so their point stands even if they used it directly?
- cwillu 3mo agoIt wouldn't be the first time ollama's llama.cpp fork reintroduced bugs and was missing important optimizations.
- trollbridge 2mo agoNo, because Ollama is buggy. The first step to answering GPC’s question is to try using an up to date llama.cpp.
- gitpusher42 3mo agoThank you! afaik ollama relies on llama.cpp and mmap. mmap loads pages on demand and doesn't use the same explicit cache or parallel reads like my engine. Most likely ollama/llama.cpp will be way slower in this case