3 ms·
Oh, that's not a problem. Just cache the retrieval lookups too.
by chowells 1y ago
Oh, that's not a problem. Just cache the retrieval lookups too.
- michaelhoney 1y agoit's pointers all the way down
- drob518 1y agoJust add one more level of indirection, I always say.
- EGreg 1y agoBut seriously… the solution is often to cache / shard to a halfway point — the LLM model weights for instance — and then store that to give you a nice approximation of the real problem space! That’s basically what many AI algorithms do, including MCTS and LLMs etc.