3 ms·
From the ChatGPT announcement post: > ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (
by fpgaminer 4y ago
From the ChatGPT announcement post:
> ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers. Fixing this issue is challenging, as: (1) during RL training, there’s currently no source of truth; (2) training the model to be more cautious causes it to decline questions that it can answer correctly; and (3) supervised training misleads the model because the ideal answer depends on what the model knows, rather than what the human demonstrator knows.
- jameshart 4y agoYes, this is indeed a known limitation. So the other day, chatGPT was doing a great job acting as a trivia quizmaster for me when it casually threw out the 'fact' that Pope Francis was the first South American to be elected pope, and that he served in the role from 2013 until his death in 2017. (For the avoidance of doubt: Pope Francis is still alive). I think we can all understand how that sort of thing counts as 'plausible sounding but incorrect'. What I think is surprising is that 'plausible sounding but incorrect' can also encompass 'rich and internally consistent novel programming language semantics with fully worked examples of applications, and rich explanations of how they work'.
- fpgaminer 4y ago> 'rich and internally consistent novel programming language semantics with fully worked examples of applications, and rich explanations of how they work'. Oh for sure, that is a really cool finding. Didn't mean to negate that, just provide a little context.