4 ms·

I really want this (in OpenAI API). Does google not care about the reputation it's getting with this shit: > We'll be launching context caching soon, along wit

• nextworddev 2y ago
  They have adopted the strategy of mainly releasing to & iterating with the large enterprise customers, because they (rightly) realized that GA'ing to every developer makes no monetary sense, unless they are trying to learn something from that launch / need to do so for competitive reasons.
  • mritchie712 2y ago
    fair point
  • refulgentis 2y ago
    Presumably, this would be a competitive reason, no? It would further the cost savings they GA'd with Gemini Flash, and it's a differentiator from every other provider.
• aiauthoritydev 2y ago
  This actually is smarter strategy IMO. Throwing the hat in the ring forces your competitors think about it and make them work too. It also gives you first mover advantage making people think you did it first. A lot of AI tools make more sense for their enterprise customers rather than people like us. People like us are good only for hype and not making money.
  • flipbrad 2y ago
    Might help show prior art to defeat software patents too.
• hidelooktropic 2y ago
  Isn't this what the Assistants API is meant for? Honestly not sure, I haven't used it before but their documentation seems to suggest you can set up the assistant to already have this context, then just send API commands to it without said context.
  • swyx 2y ago
    but they charge u for the full context every time, at least for now. latency would suggest that no caching is happening