2 ms·Yes but you can quantise the KV cache too just like you can the weights.by wkat4242 1y agoYes but you can quantise the KV cache too just like you can the weights.