3 ms·
My take is: does a human programmer in this case would have to consciously talk to themselves to not make a mistake? "Remember, remember, 'el eye en' prints, 'p
by TeMPOraL 3y ago
My take is: does a human programmer in this case would have to consciously talk to themselves to not make a mistake? "Remember, remember, 'el eye en' prints, 'pee ar aye en tee' computes length". Would they have to double-check their output? If yes, then don't expect LLMs to do better in general case, unless you make them do those steps.
This is what I mean when I keep saying that a good analogy for current LLMs, GPT-4 in particular, is the "inner voice" - that bit in your head that spits out streams of thought using language. If you look at LLM output as you'd look at the thoughts popping into your head when you read the prompt, before you consciously fix/filter/"return to sender" them, I think you'd find the two very similar.
My objection to the article's test is thus that it's intentionally set up to confuse one's gut feel. "len" and "print" are not opaque tokens, they are words with whole lot of associations, and those associations are precisely why they were chosen to name those specific functions. Even with a clear problem statement, at intuitive level, your intuition would still cling to those associations (and heaps of Python code you may have read and wrote in your life) - you need to override that at a conscious level.
Since I claim that LLM output is equivalent to the thoughts produced by intuition, and there is no "conscious level" equivalent (unless you supply it yourself by a back-and-forth), I'd expect LLMs to have trouble with this test - and I see it disputing nothing but a bad mental model of the authors.
BTW. there is a way to test my claim, which is to construct similar tests explicitly designed to avoid going against training data and word-level associations (i.e. everything else the word used as function name, like "print" here, evokes). If GPT-4 fails that, where a typical person's "gut feel" wouldn't, then I'd see this as significant, and would be very interested in the exact details of the failure.
In fact, time allowing today/tomorrow, I'll try to do some tests like these myself.
- mrtranscendence 3y ago> Even with a clear problem statement, at intuitive level, your intuition would still cling to those associations (and heaps of Python code you may have read and wrote in your life) - you need to override that at a conscious level. For the simple problem posed here, I dispute this. I do not need to rely on consciously overriding my intuition in order to replace "print" with "len" in this simple of an example. It's immediately obvious and, frankly, intuitive.
- TeMPOraL 3y agoFWIW, I copy-pasted the example to GPT-4 and played with it a bit, and doing so I noticed that I do need to keep consciously overriding my intuition at all times when reading the output - the moment I stop, I get confused, and have to re-read the last 1-3 lines with even more concentration.
- YeGoblynQueenne 3y ago>> My take is: does a human programmer in this case would have to consciously talk to themselves to not make a mistake? "Remember, remember, 'el eye en' prints, 'pee ar aye en tee' computes length". Would they have to double-check their output? If yes, then don't expect LLMs to do better in general case, unless you make them do those steps. I'm pretty sure most people would not need to do any of that to answer the question in the motivating example correctly. Remember: it's three lines and a question. Pretty much the Country Music of experiments, eh?
- TeMPOraL 3y ago> Remember: it's three lines and a question. Pretty much the Country Music of experiments, eh? IDK, three lines of math is more than enough for me to make fatal mistakes in. For example, in my secondary and high school times, it took me years to stop carelessly forgetting to carry a minus sign during calculations - even as simple as requiring three lines. Another analogy - you may argue this is goal-post shifting, but I'm really not inventing new analogies here, only bringing up the other one of the two I always use - GPT-4 being like a 4 year old kid in terms of focus. Anyway: the few tests I run tended to show GPT-4 first understanding the problem correctly, and then confusing itself mid-way through its output. This is quite similar to what my 4 y.o. often does. She'll say stuff like "no, X isn't like Y, X is more like Z, it's not like Y", and then suddenly continue as if X was like Y, as if the idea of X being like Y overpowered her earlier assertion that it's not.
- YeGoblynQueenne 3y agoI think you're really good at convincing yourself of whatever you want to be convinced :P
- TeMPOraL 3y agoQuite possibly :). Not going to dispute that. But FWIW, it seems to be mostly working well for me, so there must be at least some correlation with objective reality here. Or at least that's what I want to believe :).