3 ms·

> thought and heuristic patterns I’m not sure these are well-defined enough terms to make this claim.

• smoldesu 4y ago
  Heuristics is the ability to recognize multiple options and reason which one is correct. ChatGPT (and all GPT models for that matter) have a single context that doesn't get compared to other possible generations. You can back-propegate and see which tokens influenced the output most, but there is no evidence that AI can conceive multiple hypothetical responses and select the most-preferred one. At least not ChatGPT.
  • visarga 4y ago
    Have you heard of "beam search"? The model keeps a number of optional lines of text without being greedy and picking only the most probable token every time. You can also sample with T>0 multiple times to get an ensemble of answers that can be used as input for another AI operation. You just need to do multiple interactions with the model.