3 ms·
That would totally destroy the user experience. Users change their query so they can get a refined result, not so they get the same tired result.
by OutOfHere 2y ago
That would totally destroy the user experience. Users change their query so they can get a refined result, not so they get the same tired result.
- elawler24 2y agoDepends on the scenario. In a threaded query, or multiple queries from the same user - you’d want different outputs. If 20 different users are looking for the same result - a cache would return the right answer immediately for no marginal cost.
- OutOfHere 2y agoThat's not the use case of the parent comment: > for queries that are sufficiently similar
- pedrosorio 2y agoEven across users it’s a terrible idea. Even in the simplest of applications where all you’re doing is passing “last user query” + “retrieved articles” into openAI (and nothing else that is different between users, like previous queries or user data that may be necessary to answer), this will be a bad experience in many cases. Queries A and B may have similar embeddings (similar topic) and it may be correct to retrieve the same articles for context (which you could cache), but they can still be different questions with different correct answers.