3 ms·
That reminds me of an old paper about "Benny's Rules", a case-study focused on a kid who seemed to be doing better than average in math tests when it came to fi
by Terr_ 2y ago
That reminds me of an old paper about "Benny's Rules", a case-study focused on a kid who seemed to be doing better than average in math tests when it came to final answers... but for all the wrong reasons, using an inferred set of arbitrary text manipulation rules.
The intent was to point out that the educational approach was flawed, but I think there are interesting parallels to token processing in LLMs, which--unlike a human child--are built in such a way that crazy partial-fit rules are likely their only option.
> Benny believed that the fraction 5/10 = 1.5 and 400/400 = 8.00 because he believed the rule was to add the numerator and denominator and then divide by the number represented by the highest place value.
https://blog.mathed.net/2011/07/rysk-erlwangers-bennys-conception-of.html?m=1 https://blog.mathed.net/2011/07/rysk-erlwangers-bennys-conce...
- Animats 2y agoThis is a problem with some tests. The students may detect a pattern in the test answers which reflects the work of those generating the answers, not the content. See this article on SAT test prep.[1] The requirement that only one answer can be right means that wrong answers have easily identifiable properties. [1] https://blog.prepscholar.com/the-critical-fundamental-strategy-of-sat-reading https://blog.prepscholar.com/the-critical-fundamental-strate...