4 ms·
Actual title: "Solve the GPU Cost Crisis with kvcached: A library to enable virtualized, elastic KV cache for LLM serving on shared GPUs"
by CharlesW 1y ago
Actual title: "Solve the GPU Cost Crisis with kvcached: A library to enable virtualized, elastic KV cache for LLM serving on shared GPUs"
- dang 1y agoYes, we've put that in the title above (shortened to fit HN's 80 char limit). Submitted title was "Time to build a GPU OS? Here is the first step".