3 ms·

Ask HN: Can token caching be a business idea?

• minimaxir 1mo ago
  No, because the cache is local to the GPUs and there would be no cost/compute reason to have long-lived caches outside of the 1 hour cache already done by LLM APIs.
  • sourav_biswas 1mo ago
    What if the real business isn't caching generic prompts/but caching outputs for a specific vertical where one can know two requests are truly equivalent n not just embedding similar?
• Omgawdmadeit 1mo ago
  [dead]
• Omgawdmadeit 1mo ago
  [dead]