3 ms·
I asked Bing Chat the task (altered from your example), without telling it its name. What did it do? It searched the Web for "Wason selection task"(!) and then
by cubefox 4y ago
I asked Bing Chat the task (altered from your example), without telling it its name. What did it do? It searched the Web for "Wason selection task"(!) and then proceeded to give the wrong solution, based on Internet references.
Apparently it got confused because the web examples differed from mine, and what was correct in the web examples was incorrect in mine. Sigh. I guess GPT-4 or 5 will handle it?
- nl 4y agoInteresting! As I noted elsewhere in the thread chatGPT seemed to handle it correctly.
- cubefox 4y agoIt seems then Bing Chat's ability to search the Internet worked against it. Normally it is smarter than ChatGPT/GPT-3.5.
- civilized 4y agoWe should remember that, thanks to RLHF, ChatGPT has received a ton of feedback on the correct way to solve common benchmark problems. I have seen multiple instances in which it got a problem laughably wrong, this got publicized, then days or hours later it was always giving the right answer.