3 ms·Prompt caching but for RL – 7.5x speedup on long-prompt/short-response workloads4 points by kumama 5mo ago