3 ms·The cache hitrate is effectively 0 for LLMs since the datasets are so huge.by ryao 2y agoThe cache hitrate is effectively 0 for LLMs since the datasets are so huge.