Answers aren't deterministic.* Especially in ChatGPT, it's impossible to compare. Quick pass: - questions are different. - answer = function of several varia

Answers aren't deterministic.* Especially in ChatGPT, it's impossible to compare. Quick pass: - questions are different. - answer = function of several variables, most importantly, entire chat. then temperature, top-p... - models haven't changed. - entirely possible chatgpt's temperature, top-p etc. changed but model didn't see images of gpt3.5 at 0 temp and 0.3 on this tweet, 0.3 makes up an answer like february, 0 gets "sorry, as an AI..." https://twitter.com/jpohhhh/status/1665007972114743297?s=20 https://twitter.com/jpohhhh/status/1665007972114743297?s=20 * unless temp = 0 and all previous messages in the chat are the same. On ChatGPT, you can't even see all the messages in the chat System messages aren't visible