3 ms·

I got some feedback that GPT-3.5 was letting people down so improved my caching strategy and defaulted everything to GPT-4o!

• sebastiennight 2y ago
  Great to hear! Would you be willing to share a bit on what changes you made to your caching strategy to optimize costs?
  • jnnnthnn 2y ago
    I made the caching window substantially longer (2 days instead of a few minutes) upon noticing most queries weren't about current events, and fixed a bug in the cache resolution logic that'd lead to a substantial number of cache misses.
    • sebastiennight 2y ago
      Cool, thanks!