>> This connects to a suspicion I have that a lot of the most impressive results, [...]a are due to memorization. I completely disagree with this. Although you

>> This connects to a suspicion I have that a lot of the most impressive results, [...]a are due to memorization. I completely disagree with this. Although you state your case very well. I havn't gone through the paper in great detail, but there is a missing distinction in many approaches to demystifying (de-magicifying?) models. Often there is confusion between three levels of algorithm. (1) The low level "blind" training algorithm: Gradient descent, or similar. (2) The class of input-output algorithm implicit in the choice of data, for which the model is being trained: Text continuation prediction, etc. (3) The actual algorithm learned by (1) in order to do (2). I.e. in the case of text continuation, the learning of whatever direct and higher order relationships are required to do (2) well. "Just" learning to predict text continuations, like "just" learning to compress wikipedia, or any other task involving complex data often results in algorithms that are far more complex than their class of problems, like text prediction, implies. Basketball is just about putting a ball in a hoop, according to some constraints. But that is only the "class" of algorithm. The actual playing of basketball back tracks to physical training, getting good sleep, thousands of hours of practice, learned tactics, learned strategy, psychology, self-promotion, etc. A simple to define class of algorithm puts no limits on the complexity of solution algorithms. Point being, the models trained to predict text don't "just" predict text. That's just the category of algorithms they learn. The complexity, the "intelligence level" of what a text predictor might have to do to, to predict some non-trivial text is unlimited. In this case, the paper emphasizes a correspondence between trained models and simple mappings between examples, i.e. a support vector/kernel equivalent interpretation. That may be the case. Assuming that as true (and if so, it is a great insight!), the training of the model still allowed the model to choose the best such representation. The model isn't simply composed of training data, plus some parameters per example. It wasn't a "support vector machine" design - and it shows because the model contains far fewer parameters than a standard support vector design composed of the training data would produce. Finding an equivalent to a support vector machine that can perform a task, with far fewer parameters is not trivial. It requires some way to sort and sieve through all that data. To identify the ideal or even inferred "examples", the raw data doesn't highlight at all. Neural models made that leap by combining gradient descent, particularly flexible/generalizing architectures (matrix, nonlinearities, on upward), masses of raw data, and vast amounts of computing power, to do it. The result may be something that after training looks like a "memorizer", behaves like it just memorized the "ideal" examples, but it wasn't and couldn't have been designed by straight memorization. The model had to choose what to virtually "memorize" from the data. The same is no doubt true of our brain. Simple algorithms learning complex things. Complex problems reduced to simple solutions. Neither is just simply "predicting" or "memorizing".