4 ms·

This comment reminds me of a time, many years ago, when someone wanted an the internal design team to make a big poster out of a low res image. The design team

This comment reminds me of a time, many years ago, when someone wanted an the internal design team to make a big poster out of a low res image. The design team explained that they couldn't blow the image up to the size of a poster because it would look too grainy as it was extremely low resolution. The individual demanding the poster replied with a 'brilliant' solution: "Why don't you just take a picture of the low-res image, and then use that hi-resolution picture to blow up the image!" The same applies here. You need more information to learn the signal better, by definition there is no new information in the model.

bheadmaster 3y ago
> The same applies here. Does it, really? Analogies are great for explaining ideas, but aren't that great as logical arguments. > You need more information to learn the signal better You need more information than there is in the training set - which a well-generalizing model is supposed to be able to generate. Yes, there may be some bias if model doesn't generalize well, but that's why I put it in the assumption...
- version_five 3y ago
  > Does it, really? Yes. It's information theory. You can add information, by making rules like the class doesn't change if the thing is a different color or size or position, or on a different background. But you can't automatically create new images from a distribution that has been learned and expect them to add information.
  - flangola7 3y ago
    Then how do humans do it? Our knowledge relies heavily on sending the output of humans to the input of humans (verbal communication, books, self-thought, etc). At one point total human knowledge was little more than "some berries are bad" but through this recursive process arrived at the rich garden of information we have today.
    - version_five 3y ago
      Humans have nothing in common with neural networks (other than if you choose to misunderstand the literal definition of those two words), there's no direct analogy to be made.
      - flangola7 3y ago
        I didn't say anything about neural networks, I asked how humans do it.
        tempusalaria 3y ago
        To start with, humans have access to far more data than just what is generated by other humans. We can look at nature, other animals, mathematical laws and many other things. Humans also have far more ability than current model architectures to use memory (particularly in training), flexible context (Model context always directly follows the input distribution in training whereas we can flexibly combine different parts of our input data as we like ) and logic. We can also do live learning in ways current models can’t.
    - cubefox 3y ago
      Humans don't just try to imitate text from some group of humans, like language models. But if we wanted to imitate the text of, say, 18th century novelists, then only those 18th century novels seem to matter. If we also started to imitate our own imitations of 18th century novels (synthetic data) the result would probably get worse, since our imitations are imperfect.
      - cguess 3y ago
        To consider your analogy a bit further: Imagine you had access to only... five 18th century novels, that's it, no other knowledge about the period, the people or anything. Then you try to write your own. You can make up your own stories that are a merging of various parts of those five novels, but they'll always only contain what's in those, or some combination. If all the books deal with France then Germans or English will never make an appearance because it's literally impossible to guess that they exist since they're not in the original books (the books here are training data if that's not obvious).
  - kadoban 3y ago
    How does this train of thought apply to something like AlphaZero? There, computation/time seems to be spent creating data. Why can't the same thing in concept apply here?
    - version_five 3y ago
      Good question. I'm not an RL expert, but those kind of models are sampling the possible space of game states, based on the rules of the game, to learn a winning policy. Complex games can have effectively infinite different states. What we're talking about here is approximating a distribution from a finite and comparatively small set of images. There's no "game" you can play to get new images, the training data is fixed. The analogy might be say sampling images from a computer animation, which shows the object your interested in in different plausible poses and backgrounds, lighting, angles. There's a question of making sure this generalizes to "real" images, but that can help. But it relies on the animators knowledge of what the object can look like in different poses. That's how information gets added.
  - bheadmaster 3y ago
    > You can add information, by making rules like the class doesn't change if the thing is a different color or size or position, or on a different background. But you can't automatically create new images from a distribution that has been learned and expect them to add information. I am not quite sure I follow. Why would any of those things be true for all models?
  - sebzim4500 3y ago
    Here's a scenario where training on the output of a model could improve its own performance: Suppose you want to optimize the ability for an LLM to do arithmetic. You prompt the LLM to generate a bunch of arithmetic questions, prompting it to show its working. Then you take that output, remove the intermediate steps, and train on the results. I think you would improve some benchmarks by doing this, possibly at the expense of others. I'm sure you could do similar things with other types of model, but I think thissimple example shows the point adaquately. In any case, information theory is not especially relevant here. It puts an upper bound on model performance but we are so incredibly far away from that bound it isn't worth worrying about.
    - time_to_smile 3y ago
      > You prompt the LLM to generate a bunch of arithmetic questions, prompting it to show its working. Then you take that output, remove the intermediate steps, and train on the results. Removing or editing the output of the model is providing new information to the model, what's improving the performance of the model in this scenario is that you are explicitly adding new information and fine tuning it on these new cases. > information theory is not especially relevant here It's extremely relevant because people seem to be arguing with about mathematical facts as though they were somehow opinions. You cannot improve the performance of a model without adding new information to that model.
      - sebzim4500 3y ago
        What information is being provided? I'm not suggesting that you would do any kind of check to see if the new training data is correct. Just take the output and remove every line between the first and last. Deterministically removing data is not adding information, unless you are defining information in an extremely unusual way.
        deleted 3y ago
        [deleted]
        YeGoblynQueenne 3y ago
- time_to_smile 3y ago
  > Analogies are great for explaining ideas, but aren't that great as logical arguments. In this case it is hardly an analogy. With respect to Information Theory, compression and machine learning are essentially the same thing [0]. If you can understand why taking a hi-res photograph of a lo-res image does not create more information (and therefore cannot be used to make a hi-resolution poster) then it's essentially the exact same logic explaining why outputs of a model cannot be used to improve that same model. More to this point, which other comments have already pointed out, what's interesting about this post is it asks whether or not very, very large data sets can be reasonably compressed by models such that they can simulate training another model while requiring less data. From an information theoretic perspective this latter case is both completely sound and of potentially big practical importance. I do have to say, I'm a little surprised that basic information theory is no longer common knowledge among the HN community (more-so surprised at a resistance to it), especially among people interested in Machine Learning. Out of curiosity, do you have a background in ML and/or Comp Sci? 0. https://en.wikipedia.org/wiki/Data_compression#Machine_learning https://en.wikipedia.org/wiki/Data_compression#Machine_learn...

bheadmaster 3y ago

> The same applies here. Does it, really? Analogies are great for explaining ideas, but aren't that great as logical arguments. > You need more information to learn the signal better You need more information than there is in the training set - which a well-generalizing model is supposed to be able to generate. Yes, there may be some bias if model doesn't generalize well, but that's why I put it in the assumption...

version_five 3y ago
> Does it, really? Yes. It's information theory. You can add information, by making rules like the class doesn't change if the thing is a different color or size or position, or on a different background. But you can't automatically create new images from a distribution that has been learned and expect them to add information.
- flangola7 3y ago
  Then how do humans do it? Our knowledge relies heavily on sending the output of humans to the input of humans (verbal communication, books, self-thought, etc). At one point total human knowledge was little more than "some berries are bad" but through this recursive process arrived at the rich garden of information we have today.
  - version_five 3y ago
    Humans have nothing in common with neural networks (other than if you choose to misunderstand the literal definition of those two words), there's no direct analogy to be made.
    - flangola7 3y ago
      I didn't say anything about neural networks, I asked how humans do it.
      - tempusalaria 3y ago
        To start with, humans have access to far more data than just what is generated by other humans. We can look at nature, other animals, mathematical laws and many other things. Humans also have far more ability than current model architectures to use memory (particularly in training), flexible context (Model context always directly follows the input distribution in training whereas we can flexibly combine different parts of our input data as we like ) and logic. We can also do live learning in ways current models can’t.
  - cubefox 3y ago
    Humans don't just try to imitate text from some group of humans, like language models. But if we wanted to imitate the text of, say, 18th century novelists, then only those 18th century novels seem to matter. If we also started to imitate our own imitations of 18th century novels (synthetic data) the result would probably get worse, since our imitations are imperfect.
    - cguess 3y ago
      To consider your analogy a bit further: Imagine you had access to only... five 18th century novels, that's it, no other knowledge about the period, the people or anything. Then you try to write your own. You can make up your own stories that are a merging of various parts of those five novels, but they'll always only contain what's in those, or some combination. If all the books deal with France then Germans or English will never make an appearance because it's literally impossible to guess that they exist since they're not in the original books (the books here are training data if that's not obvious).
- kadoban 3y ago
  How does this train of thought apply to something like AlphaZero? There, computation/time seems to be spent creating data. Why can't the same thing in concept apply here?
  - version_five 3y ago
    Good question. I'm not an RL expert, but those kind of models are sampling the possible space of game states, based on the rules of the game, to learn a winning policy. Complex games can have effectively infinite different states. What we're talking about here is approximating a distribution from a finite and comparatively small set of images. There's no "game" you can play to get new images, the training data is fixed. The analogy might be say sampling images from a computer animation, which shows the object your interested in in different plausible poses and backgrounds, lighting, angles. There's a question of making sure this generalizes to "real" images, but that can help. But it relies on the animators knowledge of what the object can look like in different poses. That's how information gets added.
- bheadmaster 3y ago
  > You can add information, by making rules like the class doesn't change if the thing is a different color or size or position, or on a different background. But you can't automatically create new images from a distribution that has been learned and expect them to add information. I am not quite sure I follow. Why would any of those things be true for all models?
- sebzim4500 3y ago
  Here's a scenario where training on the output of a model could improve its own performance: Suppose you want to optimize the ability for an LLM to do arithmetic. You prompt the LLM to generate a bunch of arithmetic questions, prompting it to show its working. Then you take that output, remove the intermediate steps, and train on the results. I think you would improve some benchmarks by doing this, possibly at the expense of others. I'm sure you could do similar things with other types of model, but I think thissimple example shows the point adaquately. In any case, information theory is not especially relevant here. It puts an upper bound on model performance but we are so incredibly far away from that bound it isn't worth worrying about.
  - time_to_smile 3y ago
    > You prompt the LLM to generate a bunch of arithmetic questions, prompting it to show its working. Then you take that output, remove the intermediate steps, and train on the results. Removing or editing the output of the model is providing new information to the model, what's improving the performance of the model in this scenario is that you are explicitly adding new information and fine tuning it on these new cases. > information theory is not especially relevant here It's extremely relevant because people seem to be arguing with about mathematical facts as though they were somehow opinions. You cannot improve the performance of a model without adding new information to that model.
    - sebzim4500 3y ago
      What information is being provided? I'm not suggesting that you would do any kind of check to see if the new training data is correct. Just take the output and remove every line between the first and last. Deterministically removing data is not adding information, unless you are defining information in an extremely unusual way.
      - deleted 3y ago
        [deleted]
      - YeGoblynQueenne 3y ago
time_to_smile 3y ago
> Analogies are great for explaining ideas, but aren't that great as logical arguments. In this case it is hardly an analogy. With respect to Information Theory, compression and machine learning are essentially the same thing [0]. If you can understand why taking a hi-res photograph of a lo-res image does not create more information (and therefore cannot be used to make a hi-resolution poster) then it's essentially the exact same logic explaining why outputs of a model cannot be used to improve that same model. More to this point, which other comments have already pointed out, what's interesting about this post is it asks whether or not very, very large data sets can be reasonably compressed by models such that they can simulate training another model while requiring less data. From an information theoretic perspective this latter case is both completely sound and of potentially big practical importance. I do have to say, I'm a little surprised that basic information theory is no longer common knowledge among the HN community (more-so surprised at a resistance to it), especially among people interested in Machine Learning. Out of curiosity, do you have a background in ML and/or Comp Sci? 0. https://en.wikipedia.org/wiki/Data_compression#Machine_learning https://en.wikipedia.org/wiki/Data_compression#Machine_learn...