4 ms·
Totally fair. On a normal laptop you also need headroom to do your actual job, and KV cache + context length can eat that quickly.
by briansun 1y ago
Totally fair. On a normal laptop you also need headroom to do your actual job, and KV cache + context length can eat that quickly.