4 ms·

We can tell some of what's in the training set. One of the answers for the inductive reasoning test begins "begin from the rightmost digit". Look that phrase up

We can tell some of what's in the training set. One of the answers for the inductive reasoning test begins "begin from the rightmost digit". Look that phrase up in Google. It shows up in Chegg, Course Hero, and Brainly content for elementary arithmetic. If you bash on those how-to articles, available for bases 2 and 10, you can probably generate the pattern for base 8. This looks like an LLM doing the usual LLM thing - finding relevant items and combining them to fit. This doesn't require the level of abstraction and induction the authors impute to the LLM. Ordinary LLM behavior explains this, once you've found the relevant training data. People often solve problems that way too, of course.

Terr_ 2y ago
That reminds me of an old paper about "Benny's Rules", a case-study focused on a kid who seemed to be doing better than average in math tests when it came to final answers... but for all the wrong reasons, using an inferred set of arbitrary text manipulation rules. The intent was to point out that the educational approach was flawed, but I think there are interesting parallels to token processing in LLMs, which--unlike a human child--are built in such a way that crazy partial-fit rules are likely their only option. > Benny believed that the fraction 5/10 = 1.5 and 400/400 = 8.00 because he believed the rule was to add the numerator and denominator and then divide by the number represented by the highest place value. https://blog.mathed.net/2011/07/rysk-erlwangers-bennys-conception-of.html?m=1 https://blog.mathed.net/2011/07/rysk-erlwangers-bennys-conce...
- Animats 2y ago
  This is a problem with some tests. The students may detect a pattern in the test answers which reflects the work of those generating the answers, not the content. See this article on SAT test prep.[1] The requirement that only one answer can be right means that wrong answers have easily identifiable properties. [1] https://blog.prepscholar.com/the-critical-fundamental-strategy-of-sat-reading https://blog.prepscholar.com/the-critical-fundamental-strate...

Terr_ 2y ago

That reminds me of an old paper about "Benny's Rules", a case-study focused on a kid who seemed to be doing better than average in math tests when it came to final answers... but for all the wrong reasons, using an inferred set of arbitrary text manipulation rules. The intent was to point out that the educational approach was flawed, but I think there are interesting parallels to token processing in LLMs, which--unlike a human child--are built in such a way that crazy partial-fit rules are likely their only option. > Benny believed that the fraction 5/10 = 1.5 and 400/400 = 8.00 because he believed the rule was to add the numerator and denominator and then divide by the number represented by the highest place value. https://blog.mathed.net/2011/07/rysk-erlwangers-bennys-conception-of.html?m=1 https://blog.mathed.net/2011/07/rysk-erlwangers-bennys-conce...

Animats 2y ago
This is a problem with some tests. The students may detect a pattern in the test answers which reflects the work of those generating the answers, not the content. See this article on SAT test prep.[1] The requirement that only one answer can be right means that wrong answers have easily identifiable properties. [1] https://blog.prepscholar.com/the-critical-fundamental-strategy-of-sat-reading https://blog.prepscholar.com/the-critical-fundamental-strate...