I suppose every single ML method, ever, is effectively memorisation. The crux is the feature representation is builds to aid the memorisation. So take OLS, or

I suppose every single ML method, ever, is effectively memorisation. The crux is the feature representation is builds to aid the memorisation. So take OLS, or a decision tree / forest. When you evaluate them, they look at some feature space, then compare to some fitted parameters (i.e. memorises result) and produce an output. Methods such as SVM or Lasso are also memorisation, except with a clever feature mapping. Instead of memorising outputs for every sample, the memories them by some learned feature sunset or transformation. Perhaps the same thing happens for LLMs or NNs (I'm still reading the paper) but if so, this is just the name of the game in ML, it seems. Arguably that's what we humans do too. We are capable of creative thinking, whatever that means, but 90% of our thought process seems to be cache recall, and intelligence seems to correspond well to having a large and diverse cache. Many disciplines that require creative thinking to understand, like maths or playing instruments, seem to improve on repetitive practice that makes us memorize patterns.