3 ms·llama.cpp builds a prefix cache so the only latency is on the first generation :)by LoganDark 3y agollama.cpp builds a prefix cache so the only latency is on the first generation :)