11 ms·
With a fairly naive ReAct implementation with access to, e.g., Wikipedia, GPT-3.5-turbo will decide it should confirm things that it already thinks it knows (on
by dragonwriter 4y ago
With a fairly naive ReAct implementation with access to, e.g., Wikipedia, GPT-3.5-turbo will decide it should confirm things that it already thinks it knows (one test I remember it doing this on was “What is the largest land animal?” where the “Thought” portion of its response was approximately [wasn’t recording the runs] “I believe its the elephant, but I should confirm”.) So, it doesn’t have to recognize it is wrong to decide that it should use available resources to confirm.
- jonplackett 4y agoCool thanks - so you mean it can detect if it’s ‘unsure’ even if it doesn’t know if it’s perfectly right/wrong? That’s very helpful if so! I wonder if with the right prompting you could get it to return a 0-1 score of accurateness like you get with speech recognition. GPT-4 is great at returning JSON so you can tack on extra attributes with the actual text response.