6 ms·
This is one reason why price of SSDs also doubled, not just of RAM. > LMCache extends the KV Cache from the NVIDIA GPU's fast HBM (Tier 1) to larger, more cost
by dist-epoch 5mo ago
This is one reason why price of SSDs also doubled, not just of RAM.
> LMCache extends the KV Cache from the NVIDIA GPU's fast HBM (Tier 1) to larger, more cost-effective tiers like CPU RAM and local SSDs.
https://cloud.google.com/blog/topics/developers-practitioners/boosting-llm-performance-with-tiered-kv-cache-on-google-kubernetes-engine/ https://cloud.google.com/blog/topics/developers-practitioner...