4 ms·

So use a block based cache and tune the block size to maximize the hit rate? This isn’t rocket science.

• wonnage 7mo ago
  This seems misguided, you have to cache a prefix due to attention.