2 ms·

This has nothing to do with the cost of storage. Surprisingly, you are not better informed than Anthropic on the subject of serving AI inference models. A sibl

This has nothing to do with the cost of storage. Surprisingly, you are not better informed than Anthropic on the subject of serving AI inference models. A sibling comment explains: https://news.ycombinator.com/item?id=47886200 https://news.ycombinator.com/item?id=47886200

uoaei 6mo ago
They don't cache model state to disk. I am proposing they do.
- jeremyjh 6mo ago
  I’m proposing that you should educate yourself on the subject of LLM KV context caching.