3 ms·

Do you want to know how many identical questions ChatGPT gets asked every week, recomputing its answer each time because people stopped sharing results? I don't

by ndarray 18d ago

Do you want to know how many identical questions ChatGPT gets asked every week, recomputing its answer each time because people stopped sharing results? I don't, but I'm sure it far exceeds 3 billion. "How to center a div", "how to install this and that package", "what is this pop culture reference", "who is this famous person",...

busymom0 18d ago
Imagine if they used the stackoverflow model of "closed as dupe"
UltraSane 18d ago
OpenAI does cache the most frequent questions like this.
- gpt5 18d ago
  There is no way to cache the answer to questions that differ by even one token reliably without introducing potential problems.
  - serf 18d ago
    there absolutely are ways to cache common-routes to fields of commonly requested paths. (hint : it's not all regex and checksums.)
    - dimitri-vs 17d ago
      Ok I'll bite, how do you reuse the answer to "what color is the sky?" while also layering in memory, custom system instructions and custom response styles?
      - zamadatix 17d ago
        It's probably only memory that'd need an answer for some form of caching to be worthwhile (though I'd be curious what that answer is) since memory is on by default and rarely identical between users. The rest are nice to haves, but not everyone customizes settings just because they are there so there will be some large pool of users with the defaults who could hit cache without them.
- dimitri-vs 17d ago
  Doubtful. Considering features like memory, response style, custom system instructions, etc. can heavy alter a users output.
TiredOfLife 17d ago
I ask a question to ChatGPT and i get an answer. I ask a question to google and i spend 10-100 times longer visiting multiple websites making multiple servers generate webpages wasting electricity and time reading them finding my answer, trying different solutions
- ndarray 17d ago
  Sounds like a complex question. On average you'll visit like 1.5 sites. Those SQL queries are peanuts compared to an LLM burning through tokens. Besides, for hard questions, the LLM itself might download 30-60 websites to find the info on your behalf, e.g. checking the state of RAM prices.

by ndarray 18d ago

busymom0 18d ago

Imagine if they used the stackoverflow model of "closed as dupe"

UltraSane 18d ago

OpenAI does cache the most frequent questions like this.

gpt5 18d ago
There is no way to cache the answer to questions that differ by even one token reliably without introducing potential problems.
- serf 18d ago
  there absolutely are ways to cache common-routes to fields of commonly requested paths. (hint : it's not all regex and checksums.)
  - dimitri-vs 17d ago
    Ok I'll bite, how do you reuse the answer to "what color is the sky?" while also layering in memory, custom system instructions and custom response styles?
    - zamadatix 17d ago
      It's probably only memory that'd need an answer for some form of caching to be worthwhile (though I'd be curious what that answer is) since memory is on by default and rarely identical between users. The rest are nice to haves, but not everyone customizes settings just because they are there so there will be some large pool of users with the defaults who could hit cache without them.
dimitri-vs 17d ago
Doubtful. Considering features like memory, response style, custom system instructions, etc. can heavy alter a users output.

TiredOfLife 17d ago

I ask a question to ChatGPT and i get an answer. I ask a question to google and i spend 10-100 times longer visiting multiple websites making multiple servers generate webpages wasting electricity and time reading them finding my answer, trying different solutions

ndarray 17d ago
Sounds like a complex question. On average you'll visit like 1.5 sites. Those SQL queries are peanuts compared to an LLM burning through tokens. Besides, for hard questions, the LLM itself might download 30-60 websites to find the info on your behalf, e.g. checking the state of RAM prices.