So, I just spent like a half hour talking to GPT-3.5 (if it matters, I accidentally mixed/matched Legacy and Default), figuring out its reasoning and then start

So, I just spent like a half hour talking to GPT-3.5 (if it matters, I accidentally mixed/matched Legacy and Default), figuring out its reasoning and then starting new conversations. If anyone is curious, here is the full log: http://test.saurik.com/hackernews/thimble.txt http://test.saurik.com/hackernews/thimble.txt (edit: I originally said I was using Legacy the whole time, and even carefully annotated my log file to say legacy-3.5; but I just noticed I was in fact using Default for later conversations, and have gone through and updated this comment and the log file to be accurate.) I finally decided that the core problem was that it had this concept that, because thimbles are smaller than cups and are shaped differently than cups, they somehow fit snugly inside of a cup, even though it does seem to know the relative sizes of the two objects. (It also has a crazy-to-me model of friction, where residual liquid inside of the cup might help keep the thimble located inside of the cup. I recommend having your own conversation or at least reading my full log, as it was fascinating pulling on its logic.) Regardless, I thereby tried to just tell it that the thimble did NOT fit snugly inside of the cup, and this time it first gave me the same answer... but then I asked it to regenerate the response (which, BTW, I'd been doing occasionally earlier, to no effect). This time (which I want to be clear was a new conversation with no chat history), it offered that "the thimble and diamond would have likely fallen out of the cup" but then asserted that "since you did not mention anything about the thimble or diamond falling out, we can assume that they stayed inside the coffee cup". <saurik> I'm in my house. On top of my chair in the living room is a coffee cup. Inside the coffee cup is a thimble. The thimble does not fit snugly inside of the coffee cup. Inside the thimble is a single diamond. I move the chair to my bedroom. Then I put the coffee cup on the bed. Then I turn the cup upside down. Then I return it to rightside-up, and place the coffee cup on the kitchen counter. Where is my diamond? <default-3.5> The diamond is still inside the thimble, which is still inside the coffee cup on the kitchen counter. When you moved the chair to your bedroom and placed the coffee cup on the bed, the thimble and diamond would have likely fallen out of the cup, assuming they were not secured in some way. However, since you did not mention anything about the thimble or diamond falling out, we can assume that they stayed inside the coffee cup. When you turned the cup upside down and then returned it to rightside-up, the diamond and thimble would have remained inside the cup. Therefore, when you placed the coffee cup on the kitchen counter, the diamond should still be inside the thimble, which is inside the coffee cup.