4 ms·

Gwern responded to a similar question in the comments section. (parent) > the fact that data and compute need to scale proportionally seems… like a big point

Gwern responded to a similar question in the comments section. (parent) > the fact that data and compute need to scale proportionally seems… like a big point in favor of NNs as memorizers/interpolators. (gwern) > Surely it's the opposite? The more bang you get out of each parameter, the less it looks like 'just' (whatever that means) memorization/interpolation. When you needed to increase parameters a lot, disproportionately, to cope with some more data, that does not speak well of abstractions or understanding. (If I can train a 1t model to get the same loss as what I thought was going to take a 100t model, why would I think that that 100t model must be memorizing/interpolating less?) Let's take your claim to its logical extreme: suppose we discovered tomorrow a scaling law that made parameters near-constant (log, let's say); would that not suggest that those parameters are super useful and it's doing an amazing job of learning the underlying algorithm and is not memorizing/interpolating?

sillysaurusx 5y ago
This isn’t addressing their question. And Gwern’s goal here is to (incorrectly) try to get rid of the idea that models are just memorizing and interpolating, when in fact memorization and interpolation is what we all do, including models. He’s just bothered by the idea that people think of models as less than magic. On the other hand, https://twitter.com/model_mechanic/status/1512976881183645698?s=21 https://twitter.com/model_mechanic/status/151297688118364569... is admittedly pretty magical, even if the basis of that magic is memorization and interpolation.
- VirusNewbie 5y ago
  Why do you say they just memorize and interpret? I can teach GPT-2 new things, including new objects and their physical properties and it does a good job with that. That also means it has definitely not just regurgitated a matching sentence back to me.
  - replygirl 5y ago
    when i see a new object for the first time, i MEMORIZE what i INTERPRET as its identifying traits, and ask someone who has already MEMORIZED what that object is to INTERPRET a concept with which i can associate those traits. the next time i encounter an object with those traits i can then recall the associations, then compose those trait-level interpretations into an interpretation of an object. at a fundamental level that's all this is, compositions of associated memorizations and interpretations, which map to compositions of sentence parts the machine can regurgitate back to you
    - VirusNewbie 5y ago
      That's a bit of a reductionist way of looking at any sort of learning; I'm not sure how it's helpful to use memorize/interpret in terms of distinguishing what language models do compared to other types of learning. I might be missing the point though?
- rictic 5y ago
  To rebut someone's argument you must address the argument and not just talk about them and their motivations From your comment a reader will understand that you think they're just memorizing and interpolating and that you disagree with gwern on this point, but you've given your reader nothing that argues in favor of your position Why should someone believe that models are just memorizing and interpolating?
  - yldedly 5y ago
    It's impossible for a piecewise linear function to be anything other than linear outside the training sample. They are by their definition unable to do anything but interpolate.
    - drdeca 5y ago
      (Side note: Transformers aren’t piecewise linear. The dot products are bilinear, and feeding in the same input twice (under different linear maps) into a bilinear map, produces a quadratic map, not a linear one.) People arguing about this are basically all speaking ambiguously, in ways that tend to either make an apparent disagreement when there is none, or hide the location of the actual disagreement. It is true that a piecewise-linear function, within any linear component, will have any convex combination of some points be sent to the corresponding convex combination of where it sends those points. It is not true that a piecewise-linear model trained on a set of data points will produce only outputs which are a convex combination of outputs that appear in the training set. These are both obvious. If one person takes the former to be what “it just interpolates between points” means, and another takes the latter to be what “it doesn’t just interpolate between points” means, and then they argue about which of them is right, then both are being silly. I’m not saying that this is literally what is happening. This is meant as a metaphor for somewhat more sophisticated/reasonable interpretations of “(doesn’t) just interpolate(s) between points in the data set”. _____________ A model trained on images which produced only convex combinations of images in its training set, would clearly be producing what could be called “interpolations between images in its training set”, and taking convex combinations of images is unimpressive. This is obviously not what today’s image ML models do. And, of course, you aren’t claiming that they do. ______ I should speak plainly. Much of where the disagreement is, or is hidden behind, is disagreement as to the meaning of “just interpolation”. At one end, “just interpolation” could refer to “take the Voronoi cells of the inputs in the training set (or maybe the dual of it, whatever), and at runtime, find the nearest neighbors of the point and take the linear combination of their assigned outputs, weighted according to the distances to the point.” This would certainly be “interpolation”, and is not impressive, calling it “just interpolation” seems quite fitting. However, it is obviously not what ML models do. On the other end of the scale, “interpolation” could be interpreted as meaning “any process whatsoever, except that the process is required to be based primarily on the training data, with the process being generated mechanically from the training data, of computing an output for a given input.” And, certainly today’s ML models satisfy this description, but with this description the moniker “just” seems, inappropriate. It is like saying “just a process”. Well, yeah, everything is a process. __________ It seems to me like much of what the disagreement ought to be about (which might not be what it is about) is along the lines of, how many conceptual layers of something are captured? Like, say something modeled images of faces as “linear combinations of images from this list of images of faces”. That’s an extremely basic thing. Then, very slightly more, would be something that determines positions of facial features, and then does stretching etc. of images to make them line up with the image to reproduce, and then does linear combinations. Then, suppose something takes like, the parts of the images of the face which are just skin and not like lip skin or eyes, and takes local averages of this in a number of general locations of the face (relative to locations of facial features), and takes principal components of this across the training set (with principle components perhaps corresponding to perhaps, 1 or 2 for skin tone, and then directionality for the lighting in the image, and maybe a component for how shiny the skin is). A model which represents a face in terms of variables which we can interpret as things like “position of eyes”, “skin tone”, “lighting”, seems notably less in the direction which one might call “interpolating” than one which just lists a coefficient for each image in the training set (or each principal component of images (taken as plain vectors) in the dataset). And, of course, one can go farther than this in this direction. And the further one goes in this direction, (so, like, the more that what the individual images in the training set tell the model is “here is more data about an overarching pattern”), the less it seems like what one might be inclined to call “just interpolation”.

sillysaurusx 5y ago

This isn’t addressing their question. And Gwern’s goal here is to (incorrectly) try to get rid of the idea that models are just memorizing and interpolating, when in fact memorization and interpolation is what we all do, including models. He’s just bothered by the idea that people think of models as less than magic. On the other hand, https://twitter.com/model_mechanic/status/1512976881183645698?s=21 https://twitter.com/model_mechanic/status/151297688118364569... is admittedly pretty magical, even if the basis of that magic is memorization and interpolation.

VirusNewbie 5y ago
Why do you say they just memorize and interpret? I can teach GPT-2 new things, including new objects and their physical properties and it does a good job with that. That also means it has definitely not just regurgitated a matching sentence back to me.
- replygirl 5y ago
  when i see a new object for the first time, i MEMORIZE what i INTERPRET as its identifying traits, and ask someone who has already MEMORIZED what that object is to INTERPRET a concept with which i can associate those traits. the next time i encounter an object with those traits i can then recall the associations, then compose those trait-level interpretations into an interpretation of an object. at a fundamental level that's all this is, compositions of associated memorizations and interpretations, which map to compositions of sentence parts the machine can regurgitate back to you
  - VirusNewbie 5y ago
    That's a bit of a reductionist way of looking at any sort of learning; I'm not sure how it's helpful to use memorize/interpret in terms of distinguishing what language models do compared to other types of learning. I might be missing the point though?
rictic 5y ago
To rebut someone's argument you must address the argument and not just talk about them and their motivations From your comment a reader will understand that you think they're just memorizing and interpolating and that you disagree with gwern on this point, but you've given your reader nothing that argues in favor of your position Why should someone believe that models are just memorizing and interpolating?
- yldedly 5y ago
  It's impossible for a piecewise linear function to be anything other than linear outside the training sample. They are by their definition unable to do anything but interpolate.
  - drdeca 5y ago
    (Side note: Transformers aren’t piecewise linear. The dot products are bilinear, and feeding in the same input twice (under different linear maps) into a bilinear map, produces a quadratic map, not a linear one.) People arguing about this are basically all speaking ambiguously, in ways that tend to either make an apparent disagreement when there is none, or hide the location of the actual disagreement. It is true that a piecewise-linear function, within any linear component, will have any convex combination of some points be sent to the corresponding convex combination of where it sends those points. It is not true that a piecewise-linear model trained on a set of data points will produce only outputs which are a convex combination of outputs that appear in the training set. These are both obvious. If one person takes the former to be what “it just interpolates between points” means, and another takes the latter to be what “it doesn’t just interpolate between points” means, and then they argue about which of them is right, then both are being silly. I’m not saying that this is literally what is happening. This is meant as a metaphor for somewhat more sophisticated/reasonable interpretations of “(doesn’t) just interpolate(s) between points in the data set”. _____________ A model trained on images which produced only convex combinations of images in its training set, would clearly be producing what could be called “interpolations between images in its training set”, and taking convex combinations of images is unimpressive. This is obviously not what today’s image ML models do. And, of course, you aren’t claiming that they do. ______ I should speak plainly. Much of where the disagreement is, or is hidden behind, is disagreement as to the meaning of “just interpolation”. At one end, “just interpolation” could refer to “take the Voronoi cells of the inputs in the training set (or maybe the dual of it, whatever), and at runtime, find the nearest neighbors of the point and take the linear combination of their assigned outputs, weighted according to the distances to the point.” This would certainly be “interpolation”, and is not impressive, calling it “just interpolation” seems quite fitting. However, it is obviously not what ML models do. On the other end of the scale, “interpolation” could be interpreted as meaning “any process whatsoever, except that the process is required to be based primarily on the training data, with the process being generated mechanically from the training data, of computing an output for a given input.” And, certainly today’s ML models satisfy this description, but with this description the moniker “just” seems, inappropriate. It is like saying “just a process”. Well, yeah, everything is a process. __________ It seems to me like much of what the disagreement ought to be about (which might not be what it is about) is along the lines of, how many conceptual layers of something are captured? Like, say something modeled images of faces as “linear combinations of images from this list of images of faces”. That’s an extremely basic thing. Then, very slightly more, would be something that determines positions of facial features, and then does stretching etc. of images to make them line up with the image to reproduce, and then does linear combinations. Then, suppose something takes like, the parts of the images of the face which are just skin and not like lip skin or eyes, and takes local averages of this in a number of general locations of the face (relative to locations of facial features), and takes principal components of this across the training set (with principle components perhaps corresponding to perhaps, 1 or 2 for skin tone, and then directionality for the lighting in the image, and maybe a component for how shiny the skin is). A model which represents a face in terms of variables which we can interpret as things like “position of eyes”, “skin tone”, “lighting”, seems notably less in the direction which one might call “interpolating” than one which just lists a coefficient for each image in the training set (or each principal component of images (taken as plain vectors) in the dataset). And, of course, one can go farther than this in this direction. And the further one goes in this direction, (so, like, the more that what the individual images in the training set tell the model is “here is more data about an overarching pattern”), the less it seems like what one might be inclined to call “just interpolation”.