2 ms·

Right, over the totality of things that it reasons about, to some degree it will make inroads to correctly answering these kinds of questions, and in some ways

Right, over the totality of things that it reasons about, to some degree it will make inroads to correctly answering these kinds of questions, and in some ways it'll make errors, and what's interesting, is it'll make errors because the way in which it's attempting to answer them bears a lot of the hallmarks that we associate with conceptual understanding, rather than the mechanical operations of a calculator which truly is blind but always correct. In a way, being wrong can be a better signal of something approximating understanding under the hood. It's like if it was shooting a basketball, and it takes numerous shots, and most of them go in but some of them go out, but even the ones that miss bear the hallmarks of proper shooting form that lead to correct answers. I do think that this specific moment we're going through, in early 2023, is producing some of the most fascinating, confidently incorrect misunderstandings of chatgpt, and I hope that someone is going through these comment sections and collecting them so that we can remark on them 5 to 10 years down the line. I suspect that in time these misunderstandings are going to be comparable to how boomers misunderstand computers and the internet.

doctor_eval 4y ago
Yes, exactly this. I recently asked ChatGPT to write me some Go code. What it wrote was mostly fine, but it incorrectly used a method invocation on an object, instead of the correct code which was to pass the object to a function at the package level (aka static method). I think it's a real stretch to suggest that this could happen as a result of simply regurgitating strings it's seen before, because the string it emitted was not ever going to work. To me, it looked for all the world like the same kind of conceptual error that I have made in Go, and the only way I could see this working is if GPT had a (slightly faulty) model of the Go language, or of the particular (and lesser known) package I was asking about. It's felt more like it "forgot" - like me - how the package worked, so it followed its model instead. That error was WAY more interesting than the code.
- glenstein 4y ago
  Just on the point of forgetting, to pass the time while bored at work I told Chad GPT about my fantasy baseball team, and over the course of the conversation it forgot who my first baseman was but input the name of an entirely different first baseman who I had never mentioned, but who is a real person and who in fact was similar in a lot of ways to my actual first baseman. And sometimes when it would attempt to recall my lineup it would seem to 'forget' certain players even if it remembered them later.

doctor_eval 4y ago

Yes, exactly this. I recently asked ChatGPT to write me some Go code. What it wrote was mostly fine, but it incorrectly used a method invocation on an object, instead of the correct code which was to pass the object to a function at the package level (aka static method). I think it's a real stretch to suggest that this could happen as a result of simply regurgitating strings it's seen before, because the string it emitted was not ever going to work. To me, it looked for all the world like the same kind of conceptual error that I have made in Go, and the only way I could see this working is if GPT had a (slightly faulty) model of the Go language, or of the particular (and lesser known) package I was asking about. It's felt more like it "forgot" - like me - how the package worked, so it followed its model instead. That error was WAY more interesting than the code.

glenstein 4y ago
Just on the point of forgetting, to pass the time while bored at work I told Chad GPT about my fantasy baseball team, and over the course of the conversation it forgot who my first baseman was but input the name of an entirely different first baseman who I had never mentioned, but who is a real person and who in fact was similar in a lot of ways to my actual first baseman. And sometimes when it would attempt to recall my lineup it would seem to 'forget' certain players even if it remembered them later.