2 ms·
Regarding your last question, ChatGPT isn’t deterministic. It can certainly give a correct answer one time and an incorrect answer another time, for the same p
by layer8 3y ago
Regarding your last question, ChatGPT isn’t deterministic. It can certainly give a correct answer one time and an incorrect answer another time, for the same prompt.
- croes 3y agoFor the same prompt and same version? That would be deal breaker if the same prompt gives different results. Would also make the whole prompt engineering thing pretty useless.
- claytonjy 3y agoYes, same prompt and same version. GPT-4 is especially stochastic, even with temperature=0.
- croes 3y agoSo all these comments claiming that an article about the GPT's responses is wrong are useless. There are also such comments on this article. https://news.ycombinator.com/item?id=37361521 https://news.ycombinator.com/item?id=37361521
- rytill 3y agoI wonder why that is. Maybe they’re A/B testing assistant prompts, or there’s noise built into the architecture somewhere.
- nomel 3y agoThere was a post about it a while back. From what I recall, it’s an efficiency/cost reduction side effect, but it could be made deterministic, if they wanted to eat the cost.
- whimsicalism 3y agoI firmly disagree with your latter two statements, but yes it is non deterministic
- capableweb 3y ago> Regarding your last question, ChatGPT isn’t deterministic. It can certainly give a correct answer one time and an incorrect answer another time, for the same prompt. It's supposed to be deterministic if you set temperature to 0.0, but seems that doesn't work well in GPT4 compared to earlier versions...
- Yiin 3y agoIt is not deterministic since text-davinci-001. https://152334h.github.io/blog/non-determinism-in-gpt-4/ https://152334h.github.io/blog/non-determinism-in-gpt-4/
- croes 3y agoGood to know, I thought it was deterministic, especially because of all the comments on other articles where people wrote that the author was crazy for getting a different answer to the same question.