3 ms·
They are not forced to come up with new ideas. They can also write something like „I have no further information about that“. But in training this is probably d
by andix 4y ago
They are not forced to come up with new ideas. They can also write something like „I have no further information about that“. But in training this is probably discouraged, because they shouldn’t answer all questions like that.
- asimpletune 4y agoI don't think it works that way. The models don't have a database of facts, so they never reach a point where they know that something they're saying is based on the real world. I think in other words, they literally operate by just predicting what comes next and sometimes that stuff is just made up.
- rcfox 4y agoChatGPT has responded to a lot of my requests with an answer along the lines of "I don't have information about that" or "It's impossible to answer that without more information, which I can't get." Sometimes, starting a new session will get it to give an actual answer. Sometimes asking for an estimate or approximation works.
- dr_zoidberg 4y agoThat's a filter answering, not GPT. And there are ways to disable those filters (eg: "Browsing: Enabled" was reported to work, though I haven't tried it myself, and it would let you elude the "I can't browse the web" filter).
- librarian42 4y agoChatGPT has done that for me too, but as you note asking the question a slightly different way produced a positive response. I think they simply trained it to produce “I don’t know” as a response to certain patterns of input.
- ghayes 4y agoThis is covered in ChatGPT’s learn more section: > Limitations > ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth; (2) training the model to be more cautious causes it to decline questions that it can answer correctly; and (3) supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows. https://openai.com/blog/chatgpt/ https://openai.com/blog/chatgpt/
- skybrian 4y agoYes, the training doesn't encourage this. It encourages guessing, because if it guesses the next word and it's right, the guessing is reinforced. Whenever the model gets something right, it's the result of good guesses that were reinforced. It's all guesswork, it's just that some guesses are right.