3 ms·
The stronger need for LLM apps is for persistent response caching and reuse. Once this is available at scale with well-defined cache expiration policies, printi
by OutOfHere 2y ago
The stronger need for LLM apps is for persistent response caching and reuse. Once this is available at scale with well-defined cache expiration policies, printing logs selectively is easy. The solution ought to also support individual key invalidation as needed.
- idomi 2y agoI think it really depends on your need, if you're working on your local and trying to get something to work, caching, scale etc, might be an overkill.