3 ms·
GPT-4 doesn't pay close attention to detail in some cases
- armchairhacker 4y ago> - Arthur outputs the sum of last two digits that he had seen. > Here is the sequence in which the actors act: MAMA. I didn't even get this on the first reading. It's like that brain teaser "count the number of Fs in 'Finished files are the result of years of scientific study combined with the experience of years'" (https://www.researchgate.net/figure/How-many-Fs-do-you-see-in-the-above-box-try-it-Most-people-see-3-or-4-some-5-but_fig1_262774924 https://www.researchgate.net/figure/How-many-Fs-do-you-see-i...). It shows that GPT4 understands the instructions like a human does, and makes the same mistakes.
- mcaledonensis 4y agoThe trick with the F is a good one. Yes, tokenization is very likely one of the contributing factors. The point of the eval is an attempt to exercise: attention to detail, tokenization, ability to stop, ability to follow instruction and ignore irrelevant instructions, biases, laziness, computational limits, etc.