4 ms·
That’s useless without describing WHY you chose those flags, and how you did the optimisation…
by themanualstates 3mo ago
That’s useless without describing WHY you chose those flags, and how you did the optimisation…
- halJordan 3mo agoThe switches are all in the -h of llama.cpp (although the maintainers have a tendency to use the word in its definition). The actual values are essentially just what alibaba recommends. So you just need their model card. I would not call it highly optimized, more appropriately tuned.
- greenavocado 3mo agoI found every possible flag and its description including CUDA related environment variables and went back and iterated with Claude Opus 4.8 High until every single flag mattered above the temp one.