3 ms·There is caching today.by pklausler 2y agoThere is caching today.ryao 2y agoThe cache hitrate is effectively 0 for LLMs since the datasets are so huge.